Foundry

Foundry is a distributed build system that automatically builds Debian packages from git repositories and distributes built packages through apt repositories. I developed foundry to automate the distribution of toolbox and other packages published through deb.m10k.eu.

The build system consists of four workers, called bots, each of which performs one small step of the build process. The bots use pub-sub messaging for communication, reducing coupling between bots and increasing scalability.

The pub-sub topics and their meaning is shown in the following table.

The following diagram gives an overview how the individual foundry components interact. For understandability’s sake each bot is shown only once, but the foundry architecture does not impose a limit on the number of bots. The messaging system that foundry is based on (toolbox’s ipc module) performs automatic load-balancing, allowing bots to be added or removed on demand.

The architecture is explained in more detail in the sources.

Installation using apt

If you are using a Debian-based distribution, you can install foundry through apt. First, import the GPG key used to sign packages in the repository and make sure you have apt-transport-https installed.

# wget -O - -- https://deb.m10k.eu/deb.m10k.eu.gpg.key | apt-key add -
# apt-get install apt-transport-https

Then add the following line to your /etc/apt/sources.lst.

deb https://deb.m10k.eu stable main

If you prefer to use a development build, use the unstable suite instead.

deb https://deb.m10k.eu unstable main

Next, update your package index using the following command.

# apt-get update

Now you can install and update foundry with apt.

# apt-get install foundry

Installation from the sources

If you would prefer to install foundry from the sources, first install toolbox, then run the following commands.

$ git clone https://github.com/m10k/foundry
$ cd foundry
$ sudo make install

For the latest development version, switch to the unstable branch using git checkout unstable or specify the branch with -b when cloning the repository.

Finally, create the foundry group and a directory for foundry to keep build contexts.

# groupadd foundry
# mkdir -p /var/lib/foundry/contexts
# chown -R root.foundry /var/lib/foundry/contexts
# chmod -R 770 /var/lib/foundry/contexts

Configuration

The user that runs foundry has to be member of the toolbox, toolbox_ipc, and foundry groups. You can add a user to these groups using the following command.

# usermod -a -G toolbox,toolbox_ipc,foundry USER

The user further needs to have GPG keys for IPC messaging, package signing, and repository metadata signing. Ideally you should create three keypairs, the default key being used for IPC messaging. Keys can be generated using the following command.

$ gpg --full-generate-key

Starting foundry

The following example shows how to build toolbox with foundry.

Starting watchbot

First, we will start a watchbot to watch the toolbox repository’s stable branch. This can be achieved with the following command.

$ watchbot --repository https://github.com/m10k/toolbox#stable

A single watchbot can watch an arbitrary number of repositories. If you want to watch more than one repository, pass --repository multiple times, as shown below.

$ watchbot --repository https://github.com/m10k/toolbox#stable   \
           --repository https://github.com/m10k/toolbox#unstable

Watchbot supports watching of remote repositories (via git’s dumb and smart HTTP transports) as well as local repositories.

Starting buildbot

This one is really simple.

$ buildbot

By default, buildbot will use the IPC endpoint pub/buildbot to subscribe to commits. This means, by default build jobs will be balanced over all buildbots. If you would prefer multiple buildbots to build the same job in parallel, for example because you are building for multiple architectures, you need to pass the --endpoint parameter. You can think of the IPC endpoint as a load-balancing group. For example, consider you have started four buildbots as shown below.

$ buildbot --endpoint pub/buildbot_i386
$ buildbot --endpoint pub/buildbot_i386
$ buildbot --endpoint pub/buildbot_amd64
$ buildbot --endpoint pub/buildbot_amd64

In this scenario, if a new commit is published, it is sent to either IPC endpoint. This means, one of the buildbots listening on pub/buildbot_i386 and one of the buildbots listening on pub/buildbot_amd64 will see the message.

Starting signbot

Signbots are started as shown below.

$ signbot --gpg-key <keyid>

The purpose of signbot is to sign Debian packages, so you need to tell it which key to use using the --gpg-key option. The key must be a key-id from the default GPG key ring. By default, signbots are load-balanced the same way as the buildbots. If you want to sign packages going to one repository with a different key than packages going to a different repository, you will have to start two signbots with differing --endpoint and --publish-to options.

Starting distbot

Distbot has a few more options, but most of them are fairly self-explanatory.

$ distbot --name deb.example.org        \
          --output /srv/www/deb         \
          --arch amd64                  \
          --arch i386                   \
          --gpg-key <keyid>             \
          --description "My repository"

Like signbot, distbot needs a GPG key id. Unlike the signbot key, which is used to sign packages, this key is used to sign the metadata in the repository. You are strongly encouraged not to use the same key for both. If the path passed via --output contains an existing repository, distbot will reuse it. Otherwise, it will create a new repository in that path.

Note: If you have multiple signbots publishing messages to more than one topic, you can change the topic that a distbot watches by passing --watch and the name of the topic that your signbot is publishing messages on.

Herding the bots

If you want to see a list of the running bots, pass --list to the bot you’re interested in. For example, the following command will list all running watchbots.

$ watchbot --list

If you want to stop a particular bot, use --stop.

$ watchbot --stop 12345

That’s all you need to know to get started. If everything went well, you should be seeing built packages in your Debian repository. If not, the logs within $HOME/.toolbox/log might tell you what’s going on. This is also a good opportunity to mention that you can tell each of the bots to be more talkative by passing --verbose one or more times.