How to package LLVM

This article provides instructions on packaging the LLVM toolchain for Ubuntu.

Setting up

Project repository

  1. Clone the package definitions llvm-toolchain repository:

    $ git clone  https://git.launchpad.net/~canonical-foundations/+git/llvm-toolchain
    
  2. Add a remote tracking the Debian upstream packaging files, not fetching tags:

    $ git remote add debian-pkg https://salsa.debian.org/pkg-llvm-team/llvm-toolchain.git
    $ git config remote.debian-pkg.tagOpt --no-tags
    $ git fetch debian-pkg
    

Branch and tag model

The branch model is intended to work with multiple repositories and notions of upstream simultaneously. So we have several kinds of branches, which take the following forms:

  • upstream/<LLVM_VERSION>: branches that are generated by gbp import-orig, keeping track of the actual LLVM project sources by major version, e.g. upstream/19.

  • upstream-integration-test-suite/<LLVM_VERSION>: an external test suite put together by the Debian developers for testing LLVM toolchain integration. We track it in an upstream branch as well, generated by gbp import-orig on manually downloaded tarballs.

  • debian/<LLVM_VERSION>: copies of Debian’s packaging from their (Salsa repo)[https://salsa.debian.org/pkg-llvm-team/llvm-toolchain]. Should be periodically updated from that remote.

  • ubuntu/<LLVM_VERSION>/<UBUNTU_RELEASE>: branch from the debian/<LLVM_VERSION> branches, and track Ubuntu-specific packaging fixes.

  • pristine-tar is a single branch that keeps track of all the metadata needed to recreate the bit-perfect orig tarballs from the code in upstream/<LLVM_VERSION>, as well as the integration test suite.

  • package-queue/* Used as local workspaces for git-buildpackage when working with quilt patches, described below. These should never be pushed. If you see them in the repo, remove them.

The repo also makes use of tags. There are three kinds (assuming you’re skipping the ones from the upstream Debian remote):

  • ubuntu/<LLVM_VERSION>/<UBUNTU_RELEASE>-base are tags that represent the commit in the matching debian/<LLVM_VERSION> that we branched from. These are useful for when the Git history is complex and git merge-base may return unexpected results. These need to be managed manually.

  • upstream/<LLVM_VERSION.X.Y> these tags are created automatically when importing a tarball. They represent points in the branch history where specific versions were imported, in case there’s a reason to go back and use that.

  • upstream-integration-test-suite/X.Y.Z are just like the upstream tags, but for the version of the test suite used for a specific release. Note that these are not versioned upstream, so it’s just the test suite that’s “current” when we release a package.

  • ubuntu/<CHANGELOG_VERSION>/<UBUNTU_RELEASE> are tags for actual releases headed to the archive, so we can always see the exact code used to build a package.

The main branch you start with just contains some reusable patch files. To progress with building, checkout, for example, ubuntu/19/noble.

Important

This repository sometimes has branches with no file overlap at all, which can mean that switching branches leaves stuff scattered around that Git is afraid to remove. Generally speaking, if you switch branches and see things you don’t expect, you remove them with git clean -fd. But be careful, this deletes files!

Configuring the common LLVM packages

LLVM and its binary packages are versioned, but a few of the shared libraries it installs are not. These are the common packages, listed in debian/packages.common, which have stable ABIs across major versions according to upstream.

Debian controls which version of LLVM builds these unversioned libraries via two variables in debian/rules: SKIP_COMMON_PACKAGES and NEW_LLVM_VERSION. The design is that the newest version in the archive builds the unversioned packages, and every other version links against them.

SKIP_COMMON_PACKAGES set to no

Causes this version to build the unversioned libraries directly.

SKIP_COMMON_PACKAGES set to yes

Skips building them and links against those produced by the version named in NEW_LLVM_VERSION instead.

For example, if LLVM 21 is the newest in the archive, and you are packaging a fix for LLVM 20, you would set SKIP_COMMON_PACKAGES=yes and NEW_LLVM_VERSION=21.

Warning

This section describes an older Ubuntu approach that is being phased out.

It describes changes made with false assumptions, and we will continue to minimize the difference between Debian and Ubuntu versions of the package.

Note

This doesn’t match the Ubuntu support obligations. In Ubuntu, we maintain a single stable version as the default version for that series. For instance, Noble needs LLVM 18 to be maintained as the default for its entire support length. But we still expect to have both newer and older releases in the archive, from backports and software that hasn’t been fully ported to 18+ yet. That means the way we build the common packages is different.

Ubuntu relies on the llvm-defaults source package to handle the unversioned packages. For example, if you are on a Noble system and install the clang package with no version specified, you are actually installing something that is built from llvm-defaults, but which depends upon the versioned packages built from the actual llvm-toolchain-18 package.

Therefore, Ubuntu maintainers need to revert some of the changes made by Debian for this purpose. That means:

  • Ensuring SKIP_COMMON_PACKAGES is always set to no, so we always build everything.

  • Modifying control.in to restore the versioned packages.

  • Modifying debian/rules, so that when dh_makeshlibs is invoked, the major version is appended.

  • Loosening the version requirements for shared libraries, which are often (>= 1:$(LLVM_VESRION)). This means newer versions of LLVM can’t link against the older versions. But as they are ABI-stable, this should not be needed.

  • Renaming the *.install files and friends to ensure they match the new package names.

  • Updating the .gitignore file to ignore the new file names.

  • Never changing the major version that llvm-defaults depends upon.

Here are example changes made to llvm-toolchain-19 on Noble:

Regenerating the configuration files

LLVM contains many templated files that need to be regenerated, not least of which is debian/control. Even if you haven’t made changes, there is no guarantee that the files come in the correct state from upstream, so always default to regenerating them. Many of the generated files (like debian/control) are not always in the right state upstream, so we need to regenerate them after we implement the common package fix. The README says to use the preconfigure target, but it appears to actually be the stamps/preconfigure target. You should just need:

debian/rules stamps/preconfigure

If you get an error when you run that preconfigure target about not having wasi-libc installed, see the solution in Why is it saying I need wasi-libc installed?.

Building the package

The debian/gbp.conf file sets the builder, so you can try gbp buildpackage once you’ve exported the orig tarball.

Exporting orig tarball

This repo uses pristine-tar to avoid needing to constantly download giant tarballs or store them as blobs. It does this by storing the actual files from the tarball in a Git branch, and then setting metadata in the pristine-tar branch that allows it to make the reconstructed tarball bit-perfect.

When building with gbp, the tarballs are reconstructed automatically. However, to debug something or inspect the tarballs, export them with:

gbp export-orig

The other orig tarball

When using the export-orig command, you might be surprised to see more than the typical single orig tarball. The integration test suite used by this package is an external project, meaning it’s not part of the LLVM tarball nor part of the Debian tarball. Instead, this package makes use of the “component” tarball feature introduced in the “3.0 (quilt)” source package format.

The feature is also supported in gbp by using --component=XYZ when importing a tarball. It tracks the component on a separate new upstream branch and automatically exports all the components listed under the [DEFAULT] header in debian/gbp.conf.

If you look at that file, you should see something like:

debian/gbp.conf
[DEFAULT]
upstream-branch = upstream/X
component = integration-test-suite

[component.integration-test-suite]
upstream-branch = upstrema-integration-test-suite/X
upstream-tag = upstream-integration-test-suite/%(version)s

Provided the component is listed under [DEFAULT] and is configured correctly, gbp automatically reconstructs it with pristine-tar alongside the main LLVM tarball whenever you run gbp export-orig.

Building the source package

Run dpkg-buildpackage -S -nc to build the source package.

Building the binary package locally

For basic cases, configure debian/gbp.conf to set the correct build invocation. For example, llvm-toolchain-19 on Noble has this:

debian/gbp.conf
[DEFAULT]
...
builder = sbuild -d noble

Which allows you to invoke a build with:

gbp buildpackage

To use special options, e.g. to skip running lintian for debugging, invoke sbuild directly. See How to build packages locally.

Common tasks

Importing a new LLVM minor release

This describes what to do when LLVM drops a minor release, and you want to ship it on a distribution that already has a previous minor release from the same major version. For example, 19.1.1 already ships on Noble, but we see LLVM is up to 19.1.7 already.

There are two things to do:

  1. Grab the LLVM tarball and import it into our tree.

  2. Decide whether to bring in any new packaging changes.

To get the tarball, use uscan.

Note

At time of writing (Mar 2026), the upstream version of debian/watch seems broken and doesn’t account for the fact that the releases page on GitHub is paginated. However, there’s a reusable patch on main for this and we are attempting to get the fix upstreamed.

uscan --download-version <X.Y.Z>

Double check the naming of the tarball, and then import it to the relevant branch.

gbp import-orig --upstream-branch=upstream/<MAJOR_LLVM_VERSION> \
                --no-merge --pristine-tar ../llvm-toolchain-X_X.Y.Z.orig.tar.xz

The --no-merge flag is important, as gbp otherwise tries to merge the LLVM source into your current branch in addition to adding it to the upstream branch.

You are ready to go, but this is a good time to check what, if any, changes have been made by Debian maintainers. Fetch the latest from your debian-pkg remote and make sure the changes are in the Ubuntu repository:

git fetch debian-pkg
git checkout debian/<MAJOR_LLVM_VERSION>
git merge --ff-only debian-pkg/<MAJOR_LLVM_VERSION>
git push origin debian/<MAJOR_LLVM_VERSION>

Now look through the history using your preferred tooling. Try to use the latest packaging files from Debian in order to grab new patches or packaging fixes. However, due to Debian Sid being a mostly rolling release, sometimes major changes take place that require moving all the packages forward in unison.

For example, Debian transitioned PPC64 to a new floating point format last year, and accordingly updated all the versions they still had in the archive. However, supported Ubuntu releases from before that transition still rely on the old ABI, meaning we need to revert that change. Be on the lookout for anything suspicious.

Usually it’s easier to strip out major features we don’t want than to cherry pick all the fixes that we do want. If you want to rebase your package files on the upstream Debian ones, make sure you also create/update the corresponding Git tag:

git checkout ubuntu/<MAJOR_LLVM_VERSION>/<UBUNTU_RELEASE>
git rebase --onto debian/<MAJOR_LLVM_VERSION> ubuntu/<MAJOR_LLVM_VERSION>/<UBUNTU_RELEASE>-base
git tag -f ubuntu/<MAJOR_LLVM_VERSION>/<UBUNTU_RELEASE>-base debian<MAJOR_LLVM_VERSION>
git push origin --tags

Using LLVM 19 and Noble as an example:

git checkout ubuntu/19/noble
git rebase --onto debian/19 ubuntu/19/noble-base
git tag -f ubuntu/19/noble-base debian/19
git push origin --tags

The tag exists to mark where we are branching from, in case the history gets complex. It’s important to update it and to push that to the repo, so that information isn’t hidden.

Now proceed to make changes and build your package as you normally would.

Importing a new major LLVM release

New major versions are generally synced from Debian, and so importing a new major version is just a merge. However, in order to backport the package or pull in updates, set up our repo to track the new code.

  1. Create the new upstream branch: an orphan branch with a clean slate. Using LLVM 22 as an example:

    $ git checkout --orphan upstream/22
    $ git rm -rf --cached .
    $ git clean -fd
    $ git commit --alow-empty -m "init upstream/22"
    
  2. Import the tarball as described in Importing a new LLVM minor release. Grab it from Launchpad rather than from Github to ensure we have a matching version to the original release, unlike what we do for minor releases.

  3. Create the Debian tracking branch:

    $ git fetch debian-pkg
    $ git branch debian/22 debian-pkg/22
    $ git push origin debian/20
    
  4. Create the Ubuntu packaging branch for future updates:

    $ git checkout -b ubuntu/22/resolute debian/22
    
  5. Make sure we have a good debian.gbp.conf file for the branch. You can grab one from an existing branch as a starting point. The format generally looks like this:

    [DEFAULT]
    upstream-branch = upstream/22
    debian-branch = ubuntu/22/resolute
    pristine-tar = True
    builder = sbuild -d resolute
    
    [buildpackage]
    ignore-new = True
    
  6. Create the base tag and push everything:

    git tag ubuntu/22/resolute-base debian/22
    git push --tags
    
  7. Check if there have been any updates to the llvm-toolchain-integration-test-suite and, if so, go get a new tarball to import.

Working with debian/patches

Because our packaging branches only include debian/ and not the LLVM source, we need a different workflow. To work on the quilt patches, use gbp to import the patches into a Git branch based on upstream tags, with each patch getting converted into a commit. This allows you to use familiar Git tools, like rebasing and amending, to get the patches working for your version of the package.

Important

The branches you create corresponding to patch queues shouldn’t be pushed to the repo. They are hard to keep in sync with patches, and are designed to be just temporary working branches.

$ git checkout ubuntu/<MAJOR_LLVM_VERSION>/<UBUNTU_RELEASE>
$ gbp pq import --pq-from=TAG --upstream-tag='upstream/X.Y.Z'

The tag should be one of the ones generated automatically by gbp import-orig, probably the most recent one for the major version you are working on.

This automatically creates a branch named, e.g. patch-queue/ubuntu/<MAJOR_LLVM_VERSION>/<UBUNTU_RELEASE>. Switch to it, and you can interactively rebase to modify commits, or create new commits that correspond to new patches. Once you are satisfied, turn the commits back into patch files.

# gbp pq --commit export

Now clean up to ensure that you aren’t pushing this work to the repo.

git checkout <anything_other_than_the_patch-queue_branch>
git branch -D patch-queue/ubuntu/<MAJOR_LLVM_VERSION>/<UBUNTU_RELEASE>
git branch -l 'patch-queue/*'  # to ensure you get them all

Shipping a new package version

Building and shipping a new LLVM package is no different than it would be for any other large Ubuntu package. The main thing to keep in mind is that we want to track specifically which commit we ship a release from with a tag. Tagging test builds is not necessary, but when you push a version to a PPA that’s intended to make it to the archive, tag it with the complete version information and the Ubuntu version.

For example, if intending to do an SRU of 19.1.7 for Noble, with the changelog version showing 1:19.1.7-21ubuntu1~24.04.1, tag it as follows (Git doesn’t allow ~ in tags):

git tag ubuntu/19.1.7-21ubuntu1/noble

Running autopkgtests

The tests are a good way to find issues like package version incompatibilities that don’t necessarily break the build itself. You can run them in a PPA as usual using ppa test. As that can take time, doing a local run on your native architecture is a good way to build confidence.

There are many ways to achieve this, but the simplest is to use the same environment you used for building with sbuild. Adjust this to match your dsc file and schroot name (this assumes you only have .dsc and .deb files for your particular build):

autopkgtest ../llvm-toolchain-19_19.1.7-21ubuntu1~24.04.2.dsc ../*.deb -- schroot noble-amd64

To get the integration test suite to run, specify the .dsc file like this. Without it, the test suite code is not available for running those tests as it’s a tarball of source code not contained in the debs.

Creating a merge proposal for SRU

The easiest workflow for doing an SRU involves opening a merge proposal against the git-ubuntu repository for the package. While theoretically we could do something complex to try to track that as a remote, too, it’s simpler to just move your changes over manually.

First, within this package repo, use the git format-patch command to write each commit as a patch file, within a range from the base tag up through HEAD. For example, continuing the above example of doing an SRU of 19.1.7 on Noble, you would probably do something along the lines of this:

git format-patch ubuntu/19/noble-base..ubuntu/19/noble -o /tmp/sru-patches

With the patches created, clone the git-ubuntu repository and apply them.

cd /path/to/git-ubuntu-repo
git checkout -b sru/noble/llvm-19.1.7 ubuntu/noble-devel
git am /tmp/sru-patches/*.patch

If the patches apply cleanly, open your MP.

If they don’t, you’re in a state of partial application. This is often not because of an actual conflict, but because of a generated file changing significantly, or changelogs not quite matching, etc.

Fixing these is more art than science. Fortunately, it’s just to make the review workflow nicer, and what matters is still the package you build. For things like a big mismatch in the debian/control file, it’s usually easier to copy it into place in the git-ubuntu branch rather than using a patch. So, after it fails to apply, and you’re in the interim state with Git, do:

cp /path/to/this/repo/debian/control ./debian/control
git add debian/control
git am --continue

In some cases, if a patch is already applied or doesn’t do anything useful at all, it might make more sense to just git am --skip.

Bootstrapping a new LLVM version

When building a new major version of LLVM for an Ubuntu series that doesn’t have that major version yet, you may need to bootstrap it. The LLVM build process is complex, and in order to support some SPIR-V targets, it depends on an external tool called llvm-spirv-XY. However, that package in turn requires libllvm-XY because it needs to work with the LLVM IR for bidirectional translation. So, to bootstrap the packages, we need to go through 3 steps:

  1. Build a stripped-down version of LLVM without SPIR-V support.

  2. Build llvm-spirv-XY with that stripped-down version.

  3. Rebuild the full version of LLVM with the dependency on llvm-spirv-XY.

The first step is already configured in the packaging scripts, using Debian build profiles. The profile we need is called stage1.

Locally, you can set the DEB_BUILD_PROFILES environment variable to a space-separated list of profiles when you execute your build. So you would ensure that includes stage1 for the bootstrapped version, and does not contain stage1 when doing the final build.

However, at the time of writing, Launchpad does not support build profiles. The information is encoded into the source package locally, but isn’t respected when built in a PPA. That means we need to manually configure the package ourselves.

There are two ways to do this:

  1. Manually modify the control.in/control files to strip out dependencies and packages that are marked as !stage1, and then undo your changes when you are ready to build the full version.

  2. Use a highly-experimental script built for exactly this purpose, available in in the Foundations Sandbox.

For option 2, do:

# copy the Python script to the parent directory of the package repo
python3 ../apply-build-profile.py stage1
dpkg-buildpackage -S -nc

While experimental, it does create a backup of the files it modifies (Git helps, too), and it provides an option to restore them directly after you dput the new package.

Once this stripped-down version of LLVM has built in your PPA, build the llvm-spirv-XY package. Note also that it has some dependencies, like spirv-tools and spirv-headers, which might need to be built too.

In some circumstances, those packages may already exist in another version of Ubuntu, and backports tend to go smoothly. If not, grab the latest packaging files from Debian Salsa:

The final spirv-llvm-translator package can be tricky to build, as the upstream Debian package definition doesn’t always build cleanly on Ubuntu. For example, you might need to adjust the version of GCC it requires, which in turn breaks the symbols file included with the package. You have to fix these things as you would in any other package. For that specific problem, reference the Debian documentation on symbols files

Once you are able to get that package built, do a full build (i.e. without stage1) of LLVM, which should be able to pick up the dependency from your PPA.

Useful gbp features

The git-buildpackage package has lots of features not described here. Check out the gbp documentation. Useful commands include:

  • gbp dch to draft your changelog

  • gbp pull to check for the ability to fast-forward merge before proceeding and updating known branches