pkgjam: Moving Away from the One True Tree

James K. Lowden

FreeTDS maintainer & pkgsrc whiner

What is pkgjam?

Organization of Talk


As a novice at package creation, I found the guide adequate for dealing with ordinary situations, but lacking on how do deal with the weirder stuff. Worse yet, there's almost always something weird.

— user-pkg@ 21 April 2007

Welcome to pkgjam

Dependency Independence



Dependency Independence (2)

Why hard links?

Database Diagram

Complexity is inherent in the problem domain. We manage this complexity now, but without the benefit of a diagram or a relational database.

Interpolated strings are not a complexity management tool.

Three Databases

bin and site are locally managed. src is built from CVS-controlled sources and is downloaded by users.

The build tool, pkg, consults all three databases to determine what to build and how to build it. (Dependencies may be satistified from bin and options driven by site.)

Interesting Database Columns

integer representing a version of a package, managed by a central registry
user-settable package choice akin to PKG_OPTIONS
canonical version, a lexically sortable package version number
any build-time package setting, large or small

bin Database

site Database

Three kinds of Options

  1. Global options that apply to many packages use id 0
  2. Global options affecting a single package use that package's id
  3. Package-specific options

User can discover options by quering the database.

src Database

Main tables

one id per name+version
define a “package mask”, a range of package versions that would satisify the package's requirement
describe cases in which one of several options satisfy a dependency
are any build-time setting, including e.g. PREFIX
control knobs, sets of knobs, and other options


Describing Dependencies as Relations

Loading src Database

src Database Benefits

The database answers ad hoc queries instantly that either can't be answered today, or that take overnight to answer. For example:

Note: no src.Plist. Instead, we derive a plist (for bin.Plist) by watching what files are installed by the package into its directory.

Exempli Gratia

sqlite> select as pkg, as 'depends on' 
        from Packages p 
        join Dependencies d on = 
        join Packages dp on d.pkgname = 
        where = 'libxml2';

        pkg         depends on
        ----------  ----------
        libxslt     libxml2   
        py24-libxm  libxml2   
        xmlto       libxml2   


We chose SQLite because:

Other features:

Build tool: pkg

Johnny Lam discusses pkg later.


Coming Attractions

Choose pkgjam

[any material that should appear in print but not on the slide]