1. Neeraj Sharma
  2. python-thetango

Overview

HTTPS SSH
thetango README
---------------

0.  Contact info

Author: Prarit Bhargava
Email: prarit@redhat.com

I can usually be found on oftc:#ia64.

1.  Introduction

The problems associated with bootstrapping an OS on a new architecture have
been around for a long time.  The issues with doing so are numerous -- how
to build, what order should the packages in the OS be built in, how to
identify build dependencies, etc.  Determining a build map, or build tree,
would go a long way in decreasing the time to entirely rebuild or bootstrap
an architecture.

As a starting point, a brute force build method of repeatedly traversing
the F7 SRPM list alphabetically was done.  This resulted in approximately 2500
packages built over a week -- far short of the complete list of SRPMS.
The frequency of successful builds slowed down dramatically after this number
of SRPMS were built.  The perl-* SRPMS are particularily nasty ...

thetango attempts to resolve the build problem by taking a set of SRPMS and
building build trees by evaluating the dependencies between the potential
resulting binary RPMs, and determining the build tree for individual SRPMS.

2.  The Problem of Resolving Dependencies in SRPMS.

"I want to bootstrap an arch.  What order do I build the SRPMS in?"

Given a set of SRPMS, there is no simple method of determining what each
SRPM "provides" as a final binary result (rpm -qp --provides source.rpm
yeilds nothing).  Complicating matters further is the possibility of
circular dependencies and potentially missing packages.

The solution is a multi-step process which starts with examining each SRPM's
source.spec file to decipher the information to determine what each package
will provide as binaries.

Using that information in conjunction with a SRPM's "rpm -qp --requires"
output, a relationship between an SRPM's binary requirements and the SRPM
packages can be drawn.

Finally, by using the SRPM "requirements to packages" information, a "build
tree" for each SRPM can be determined.

3.  A Quick Code Tour

There are five files:

/etc/thetango/thetango.conf
/etc/thetango/example
/etc/thetango/built
/etc/thetango/file_fixes
/usr/lib/[python_dir]/thetango/boneyard.py


built is a list of prebuilt SRPMS required for F7.  See section 4b).

file_fixes is a list of dependencies that the code will not resolve (more on
this in section 5d).

boneyard is thetango's library of python classes.  These classes include:

theTango -- a configuration library for this utility.
reqs - a class of SRPM requirements
provs - a class of SRPM provides and packages
r2p - a class that converts a list of requirements to SRPMS
dfstree - a tree utility that recursively outputs SRPM dependency info as a
          list

thetango.conf is a configuration file which indicates where the final trees
and information gathered from the SRPMS is to be saved.

example is a simple example which involves taking a set of SRPMS and
converting them into build trees.

4.  Current Status

The code is written in python.

For example from F7,

m4-1.4.8-2.fc7.src.rpm

tree is

doxygen-1.5.2-1.fc7.src.rpm
alsa-lib-1.0.14-0.4.rc3.fc7.src.rpm
audiofile-0.2.6-6.fc7.src.rpm
autoconf-2.61-8.fc7.src.rpm
automake-1.10-5.src.rpm
docbook-style-dsssl-1.79-4.1.src.rpm
tetex-3.0-39.fc7.src.rpm
unzip-5.52-4.fc7.src.rpm
jadetex-3.12-13.1.1.src.rpm
docbook-style-xsl-1.72.0-2.fc7.src.rpm
m4-1.4.8-2.fc7.src.rpm

b) builtlist, the list of "pre-requirements" was determined from the list at

 http://fedoraproject.org/wiki/Packaging/FullExceptionList

Due to circular dependencies, additional SRPMS were included in this list.

I agree that many of the packages included in this list are "bogus".
These were included to avoid the circular dependencies.  I admit that I
have not attempted to determine what the "real" cause of the circular
dependencies are. ***

5.  TODO

a) I have identified a few additional enhancements that will come along in
future versions (determine both package and requirements info in the same
loop, etc..)

b) I'm analyzing a few different tree algorithms in order to determine the
best final build procedure. I suspect some version of a minimal spanning
tree algorithm mixed with master-slave process model will do the trick.
However :), I'm open to any other suggestions people may have.

c) Fixing Provides: & BuildRequires: lines in various SRPMS so that we can
elminate the file_fixes junk.

My argument is simple.  We've never needed a requirement on the language
used on the Provides: & BuildRequires: line.  There are several ways that the
information can be written in a spec

All on one line

Provides: foo bar baz

All on one line separated by commas

Provides: foo, bar, baz

The version required line,

BuildRequires: foo >= 1.16

And lastly the file requires line:

BuildRequires: /usr/bin/foo

If we could stick to one language, and possibly eliminate the inclusion of
specific files it would make life so much easier for this code :)

d) No man page.  Okay, that'll be coming in V2 as well. Until then, this doc
will have to suffice.

e) The files listed in the builtfile are not specifically that version of
RPM.  They're meant to be a "base" upon which to build.  Need to clean up
the file to indicate that.

f) There are things that I'm sure I'm forgetting ATM :), so I'll be adding
them here.

6.  Some Examples

a) building a pkglist for a specific package

    TheTango = thetango.boneyard.theTango()
    # Now lets really do some work

    reqs = thetango.boneyard.reqs(TheTango)
    provs = thetango.boneyard.provs(TheTango)
    srpm = 'some.src.rpm'
    provs.makeProvsList(srpm)
    provs.makeExtraProvs(srpm)

b) build a tree for a specific RPM

    TheTango = thetango.boneyard.theTango()
    # Now lets really do some work

    reqs = thetango.boneyard.reqs(TheTango)
    reqs.makeAllReqsLists()
    provs = thetango.boneyard.provs(TheTango)
    provs.makeAllProvsLists()

    r2p = thetango.boneyard.r2p(TheTango, reqs, provs)
    r2p.convert()

    dfstree = thetango.boneyard.dfstree(TheTango, r2p, verbose=0)
    srpm = 'some.src.rpm'
    dfstree.makeTree(srpm)

c) Note the example in the file example builds trees for *everything*

7.  The Tango?

sniff ... sniff ... slobber.