pt-gnupg-builds

Intended for packaging GnuPG and supporting tools for a variety of OS releases.

We deliberately do not create packages which install in the normal system paths. Keep GnuPG packages supported by your OS vendor there. Remember that breaking OS tooling around GnuPG can break your ability to keep your OS up-to-date and secure.

Instead, we install under /opt/gnupg. All packages custom-built for GnuPG install into that prefix. This includes other major projects such as GnuTLS.

Building

We're using Vagrant and/or Docker to build packages. Originally we used Vagrant and much of the design assumed that. We're switching to Docker. There may be some rough edges.

First create site-local.env; things will work without it, but you might not like some details embedded in packages and their names. It should look something like:

export PKG_EMAIL='[email protected]'
export PKG_VERSIONEXT='suf'

I use a Pennock Tech email address and pt for PKG_VERSIONEXT.

You will almost certainly want to change values in confs/deploy.sh; if you instead export PT_SKIP_DEPLOY=true in environ before building, then you can build packages without trying to put them live on the repo server.

Vagrant

This is going away. I don't need a new kernel each time I build and it's easier to pre-cache userlands with installed packages in Docker.

% vagrant status
## see list of machines, typically named for OS
% tools/deprecated.build-vagrant.sh xenial
## if problems: edit, fix, then:
% PT_RESUME_BUILD=t tools/deprecated.build-vagrant.sh xenial
## and repeat until build-vagrant.sh might work.

The contents of the ./in directory are unidirectionally rsync'd towards the VM into /in/ on up and each reload. The build scripts download large assets into that directory, if they don't already exist. ./in is git ignored, so this is an appropriate place to store cached assets between attempts. The environment variable PT_GNUPG_IN can be used to point to a different local location.

You can sync more assets in with vagrant rsync $machine_name.

Unless disabled via env flag, the build script will invoke a deploy script for each machine. The included deploy scripts assume use of aptly for repo management and are tuned via confs/deploy.sh.

Docker

% ./tools/local-fetch.sh
% ./tools/build-docker.rb disco
% ./tools/publish-packages.rb disco

There is no framework for resuming builds, but you can manually invoke Docker with the given command-lines but no command to run inside the container, then manually run the setup/build command-line. You can also pre-generate the baseline OS image with all packages loaded.

For Docker, ./in/ and ./out/${BUILD}/ are volume bind-mounted into the container.

In the confs/machines.json file the docker array for each machine defines an ordered list of preferred base images. If you expect to need to iterate and want to cut down on the start-up time, then try:

% ./tools/generate_docker_images.sh disco

This will install all the OS packages, fpm, etc and do all the pre-setup work; the resulting image will be tagged, and you can insert the name of the built image into the preference list for the machines array. If not already there.

Updating

• Adding a new box (OS) for building on goes into confs/machines.json; the values are parsed in the Vagrantfile and also used elsewhere (jq in scripts). Do not add the repo field until the initial run is complete.
• A new version of GnuPG software should come automatically from swdb.lst
• A new version of non-GnuPG dependent software goes in confs/versions.json
  • A non-swdb version override for GnuPG can also go in here; in the overrides section, keyed by the package basename, include build_version in the option keys, with a value of the overridden version.
• Changing how a package is built goes in confs/configures.json
• Adding an "A needed for B" dependency ordering goes in the confs/dependencies.tsort-in file (use column -t for formatting, tsort to see the resulting order).
• The only items built are those in the dependencies file.
• Changes in PGP signing keys will need to be reflected both in the key-dumps in confs/ and in pgp_ownertrusts defined in confs/params.env
  • ./tools/update-keys.sh might help with some of that
• Patches should be named starting with the product name (eg, gnupg22) and an underscore. The next section may optionally start with v to be a version constraint; use this when the patch is already committed upstream for the next release version. Anywhere in the patch filename may be a sequence _pNNN_ where NNN is a prefix strip level (per patch -p) to be applied.

Adding new repo with aptly

With an S3 bucket set up, S3-website endpoint enabled (for top-level index), CloudFront providing HTTPS from an ACM-managed cert and DNS all handled elsewhere, the aptly steps are:

aptly repo create -comment "Pennock Tech Ubuntu Xenial repo" pt-xenial

jq '.skipContentsPublishing=true' .aptly.conf > x && mv x .aptly.conf

and then here:

./tools/publish-packages.rb --initial xenial

Yields: deb https://public-packages.pennock.tech/pt/ubuntu/xenial/ xenial main

We could move repo creation in-tool too, but currently leaving that out.

Todo

• Audit for XXX, FIXME, TODO, UNIMPLEMENTED