# smart-oxe /

Filename Size Date modified Message
contrib
doc
sandbox
smart
375 B
10.9 KB
17.6 KB
390 B
15.1 KB
561 B
71 B
2.7 KB
5.3 KB

# Smart Package Manager

Author: Gustavo Niemeyer niemeyer@conectiva.com $Rev$ $Date$

## Overview

The Smart Package Manager project has the ambitious objective of creating smart and portable algorithms for solving adequately the problem of managing software upgrading and installation. This tool works in all major distributions, and will bring notable advantages over native tools currently in use (APT, APT-RPM, YUM, URPMI, etc).

From The Free On-line Dictionary of Computing:

smart

1. <programming> Said of a program that does the {Right Thing}
in a wide variety of complicated circumstances. (...)


## Project Status

The development of Smart Package Manager started on May 4th, 2004, and the project is currently under beta testing, since all core features are implemented.

## Features

### Modular

Smart has been developed with modularity and flexibility in mind. It's completely backend-based, and package-manager-agnostic. Support is currently implemented for RPM, DPKG, and Slackware package management systems, and porting it to new systems should be very easy.

### Smart Transactions

That's one of the most interesting aspects of Smart Package Manager, and the one who has motivated calling it smart. Computing transactions respecting the relations involved in the package management world may become an unpleasant task when thousands of packages and relations are being considered, or even when just a few complex relations turn the most obvious choice into the unwanted one.

While other softwares try to find a possible solution to satisfy the relations involved in some user-requested operation, and sometimes even fail to do so [1], Smart goes beyond it. In the kernel of Smart Package Manager lives an algorithm that will not only find a solution, if one is available, but will find the best solution. This is done by quickly weighting every possible solution with a pluggable policy, which redefines the term "best" depending on the operation goal (install, remove, upgrade, etc).

This behavior has many interesting consequences. In upgrades, for instance, while precedence is given to newer versions, intermediate versions may get selected if they result in a better global result for the system. Packages may even be reinstalled, if different packages with the same name-version pair have different relations, and the one not installed is a considered better option.

Another important goal achieved with the transaction algorithm is that, even though it is able to check and fix relations in the whole system, it will work even when there are broken relations in installed packages. Only relations related to the operation being made is checked for correctness.

 [1] Check Study Cases for real cases where the algorithm works better than what is implemented in other softwares.

### Multiple Interfaces

Smart has multiple native and completely integrated interfaces:

• Command line interface, with several useful subcommands: update, install, reinstall, upgrade, remove, check, fix, download, search, and more.
• Shell interface, with command and argument completion, making it easy to perform multiple operations quickly using a local or remote terminal.
• Graphic interface, offering the friendliness of visual user interaction.
• Command line interface with graphic feedback, allowing one to integrate the power of command line with graphic environments.

Besides these interfaces, ksmarttray is also included in the smart package. It notifies users about available updates using a KDE tray icon.

### Channels

Channels are the way Smart becomes aware about external repositories of information. Many different channel types are supported, depending on the backend and kind of information desired:

• APT-DEB Repository
• APT-RPM Repository
• DPKG Installed Packages
• Mirror Information
• Red Carpet Channel
• RPM Directory
• RPM Installed Packages
• Slackware Repository
• Slackware Installed Packages
• URPMI Repository

### Priority Handling

Priorities are a powerful way to easily handle integration of multiple channels and explicit user setups regarding preferred package versions.

Basically, packages with higher priorities are considered a better option to be installed in the system, even when package versions state otherwise. Priorities may be individually assigned to all packages in given channels, to all packages with given names, and to packages with given names inside given channels.

With custom priority setups, it becomes possible to avoid unwanted upgrades, force downgrades, select packages in given channels as preferential, and other kinds of interesting setups.

### Autobalancing Mirror System

Smart offers a very flexible mirror support. Mirrors are URLs that supposedly provide the same contents as are available in other URLs, named origins. There is no internal restriction on the kind of information which is mirrored. Once an origin URL is provided, and one or more mirror URLs are provided, these mirrors will be considered for any file which is going to be fetched from an URL starting with the origin URL.

Mirror precedence is dynamically computed based on the history of downloads of all mirrors available for a given origin URL (including the origin site itself). The fastest mirrors and with less errors are chosen. When errors occur, the next mirror in the queue is tried.

For instance, if a mirror http://mirror.url/path/ is provided for the origin ftp://origin.url/other/path/, and a file in ftp://origin.url/other/path/subpath/somefile is going to be fetched, the mirror will be considered for being used, and the URL http://mirror.url/path/subpath/somefile will be used if the mirror is chosen. Notice that strings are compared and replaced without any pre-processing, so that it's possible to use different schemes (ftp, http, etc) in mirror entries, and even URLs ending in prefixes of directory entries.

Smart has a fast parallel downloading mechanism, allowing multiple connections to be used for one or more sites. The mechanism supports:

• Resuming
• Timestamp checking
• Parallel uncompression
• Autodetection of FTP user limit
• Cached file validation

and more.

At that moment, the following schemes are nativelly supported:

• file
• ftp
• http
• https
• scp

Additionally, the following schemes are supported when pycurl is available:

• ftps
• telnet
• dict
• ldap

### Removable Media Support

Smart Package Manager implements builtin support for removable media (CDROMs, DVDs, etc) in most of the supported channel types. The following features are currently implemented:

• Mountpoint autodetection
• Support for multiple simultaneous media drives
• Medias may be inserted in any order
• Installed system is guaranted to maintain correct relations between media changes
• Remote removable media support using any of the supported schemes (ftp, http, scp, etc)

## Running Smart

Smart Package Manager may be run in many different ways, depending on the interface in use and on the intended goal.

The following command would install the foobar package, for instance:

smart install foobar


While the following command would install the foobar package, but with graphic output:

smart --gui install foobar


To open the graphic interface in interactive mode, one may simply run:

smart --gui


Similarly, to open the following command would open the shell interface:

smart --shell


Extensive help is available for all commands, by using the --help switch:

smart --help
smart install --help
smart channel --help
...


## Building Smart

### Dependencies

Core: Graphic Interface: Smart is written in Python, with some core modules rewritten as C extensions for memory savings and performance gains. With that in mind, the core system of Smart depends on Python 2.3 or higher, and a C compiler to build the extensions. The graphic interface depends on pygtk 2.4 or higher. The RPM backend depends on the Python rpm module of RPM 4.4 or higher, due to a limitation which was present in previous versions of the ts.dbMatch() method, and the availability of the readHeaderFromFD() function. In the contrib/patches/ subdirectory there are patches for previous RPM versions including the missing functionality. There are also pre-packaged binary versions which include the patched module without requiring changes in other tools. There are no extra dependencies besides DPKG itself. There are no extra dependencies besides the packaging scripts installpkg, upgradepkg and removepkg.

## Study Cases

In this section will be described real cases showing Smart behavior in comparison with other tools, or handling unusual situations.

### Case 1 - APT

This case happened in a real world environment where a weakness in the algorithm used by APT (which is the same used in APT-RPM) turned a simple operation into a problem of obscure results. Smart Package Manager was used in the same environment to show its results.

The problem starts when an installation of xscreensaver is tried:

[root@damien:/root] apt-get install xscreensaver
Building Dependency Tree... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.

Since you only requested a single operation it is extremely likely that
the package is simply not installable and a bug report against
that package should be filed.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
Depends: libxml2.so.2
E: Broken packages


The error shown makes the user belive that libglade-2.0.so.0 and libxml2.so.2 are not available. That's not the case:

[root@damien:/root] apt-get install libxml2
Building Dependency Tree... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.

Since you only requested a single operation it is extremely likely that
the package is simply not installable and a bug report against
that package should be filed.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libxml2: Depends: glibc-iconv but it is not going to be installed
E: Broken packages


Another misguiding error message. Let's go further:

[root@damien:/root] apt-get install glibc-iconv
Building Dependency Tree... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.

Since you only requested a single operation it is extremely likely that
the package is simply not installable and a bug report against
that package should be filed.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
glibc-iconv: Depends: glibc-gconvdata (= 2.3.3) but 1:2.3.2-586_1cl is to be installed
E: Broken packages


Version 2.3.3 is needed, but 1:2.3.2-586_1cl is to be installed. This message is mostly correct. The only problem is, "1:2.3.2-586_1cl" is already installed:

[root@damien:/root] apt-cache policy glibc-gconvdata
glibc-gconvdata:
Installed: 1:2.3.2-586_1cl
Candidate: 1:2.3.2-586_1cl
Version Table:
*** 1:2.3.2-586_1cl 0
100 RPM Database
0:2.3.3-69473cl 0
500 file: conectiva/all pkglist


The problem was found. A package from another repository (586_1cl shows it's not native, in that specific case) has a higher epoch than the one available in the usual repository. This clearly shows that the APT algorithm marks a single version as candidate, and when this is not the wanted version for some operation, the whole operation is compromised.

When testing Smart Package Manager in the same environment, the expected result is obtained:

[root@damien:/root] smart install xscreensaver
Updating cache...              ######################################## [100%]

Computing transaction...

glibc-gconvdata-0:2.3.3-69473cl.i386

Installed packages (4):
glibc-iconv-0:2.3.3-69473cl.i386
libxml2-2:2.6.13-67598cl.i386
xscreensaver-4.15-69825cl.i386

Confirm changes (Y/n)?


Done.

## Credits

This is the credit section, where people and institutions that have somehow contributed to the project are mentioned.

Conectiva, Inc.:
Conectiva has funded Smart development as the author's employer.
Wanderlei Cavassin:
Conectiva's research & development coordinator, who belived the project was viable and encouraged the author to work on it.
Ednilson Miura & Herton Ronaldo Krzesinski:
Conectiva employees, helped setting up many distributions for tests whenever necessary.
Andreas Hasenack:
Conectiva employee, helped as being the first brave pre-alpha tester, and contributed with many ideas, discussions, etc.
Arnaldo Carvalho de Melo:
Conectiva co-boss, helped with the "channel of mirrors" idea and by encouraging me to build a generic channel information method.
Others @ Conectiva:
Many other people in Conectiva helped with ideas and alpha-testing in general during the pre-release period of Smart development.
APT-RPM & Debian:
Experience on packaging and ideas for a better framework were developed while the author of Smart worked as the APT-RPM maintainer.
Jeff Johnson:Maintainer of the RPM software, employed by RedHat, Inc. contributed as being the RPM maintainer itself, and in many discussions regarding packaging theory in general.
Seth Vidal:YUM author, and member of the Duke University, contributed to Smart with the development of the XML MetaData repository format and discussions about it.
Michael Vogt:Co-maintainer of the Synaptic project (the maintainer, lately), contributed by discussing ideas that could be/were implemented in Synaptic, and ended up being adopted by Smart as well.
Sebastian Heinlein:
Author of the package icons for Synaptic, that were mercilessly stolen to be used in Smart's graphic interface.
TaQ/PiterPunk at #slackware-br:
These guys helped Smart development by explaining details of Slackware practices regarding packaging.
Matt Zimmerman:Debian/Ubuntu developer and co-maintainer of the APT software, helped by shining some light regarding details of the DPKG pre-depends ordering expectations.