A Guide To Build Systems, Debian Packaging & Apt Repositories

A guide to packaging your applications (binaries, automake, cmake & meson) and hosting them in apt repositories

Haven’t you wished you could distribute your package effortlessly? No need to ask the user to copy your binaries and run them from a directory. In first half of this tutorial, we’ll cover building packages for Debian based distributions like Ubuntu. In the second half we’ll give a quick way of creating an apt repository to have these Debian packages available to computers on your local network.

There are two parts to this article:

Part 1: Build Systems & Debian Packaging

We will cover the following types of packages

1. Binary Files Based Packages

2. Build System: Automake

3. Build System: CMake

4. Build System: Meson

5. Preserving configuration files across package versions

Part 2: Hosting packages in a Private Apt Repository

The second part explain how to host the packages on a webserver in a repository for installation using apt on your local computer and computers on your network.

Finally there is a Python script descrbed (and in the repo) to generate the Apt repository.

Download the sources

Let’s start with keeping a copy of the repository cloned for reference as you read through the tutorial. The repository contains a sample application for each build system.

git clone http://www.github.com/mndar/writings

Part 1: Build Systems & Debian Packaging

The core aspect of the Debian packaging system is the debian directory which contains files like control and conffiles. These are text files that describe metadata and various configuration options for the Debian package.

Your Debian based distribution like Ubuntu has various utilties to manage the Debian packages (deb files). Install these utilities with

sudo apt install dpkg-dev

1.0 Binary Files Based Packages

Let’s say you have bunch of binaries or you have compiled you binaries manually. We’ll take the case of of the previous tutorial where we had 2 source and 1 header file and build the binary testclass with manual compilation. All you have with you is the binary testclass. Here’s how you create a Debain package to install the binary to /bin/

Steps

1. Create a directory gobject-test

2. Layout the files in gobject-test directory as if gobject-test is the / of your target system and copy files into the respective directories

3. Create a directory named debian and in it have a file named control. The control file is included below

4. Build the package with dpkg-deb -b gobject-test

The following console session will make it clear

$ tree gobject-test
gobject-test
├── DEBIAN
│   └── control
└── usr
    └── bin
        └── testclass

3 directories, 2 files

$ dpkg-deb -b gobject-test
dpkg-deb: building package 'gobject-test' in 'gobject-test.deb'.

And here is the control file. I have included only the mandatory fields. A complete list of fields can be found here https://www.debian.org/doc/manuals/debian-faq/pkg-basics.en.html#controlfile

Package: gobject-test
Version: 0.1
Maintainer: testuser
Architecture: amd64
Description: Program demonstrating GObjects

2.0 Build System: Automake

Let’s say your build system automake. You don’t need to build the package with ./configure, make to build a Debian package. The steps in this case are much simpler

Steps

1. Unpack the sources into a directory

2. Change into the source directory

3. Run dh_make with the -f option

4. Build the Debian package with dpkg-buildpkg (with options)

Here is a console session to make it clear

$ tar zxvf gobject-0.1.tar.gz 
gobject-0.1/
gobject-0.1/missing
gobject-0.1/.deps/
gobject-0.1/.deps/testclass.Po
gobject-0.1/.deps/myclass.Po
gobject-0.1/depcomp
...
...
...

$ cd gobject-0.1

$ dh_make -f ../gobject-0.1.tar.gz 
Type of package: (single, indep, library, python)
[s/i/l/p]?
Maintainer Name     : mandar
Email-Address       : mandar@unknown
Date                : Mon, 16 Sep 2024 18:19:10 +0530
Package Name        : gobject
Version             : 0.1
License             : blank
Package Type        : single
Are the details correct? [Y/n/q]
Done. Please edit the files in the debian/ subdirectory now.

$ dpkg-buildpackage -rfakeroot -us -uc -b
...
...
...
 dpkg-genbuildinfo --build=binary -O../gobject_0.1-1_amd64.buildinfo
 dpkg-genchanges --build=binary -O../gobject_0.1-1_amd64.changes
dpkg-genchanges: info: binary-only upload (no source code included)
 dpkg-source --after-build .
dpkg-buildpackage: info: binary-only upload (no source included)

$

You should have a package named gobject_0.1-1_amd64.deb in the parent directory.

3.0 Build System: CMake

Building Debian packages is extremely easy with CMake. All you need are few lines at the end of the your CMakeLists.txt. You have to run the cpack command to build the debian package. The debian package will be generated in the current directory.

The CPack lines should be. Adjust the version numebrs and description below.

#build debian packages
SET(CPACK_GENERATOR "DEB")
SET(CPACK_PACKAGE_VERSION_MAJOR  1)
SET(CPACK_PACKAGE_VERSION_MINOR  0)
SET(CPACK_PACKAGE_DESCRIPTION "A Demonstartion of GObject")
SET(CPACK_DEBIAN_PACKAGE_MAINTAINER "ABC") #required
INCLUDE(CPack)

Here is a simple CMakeLists.txt for the program in the previous tutorial. The console sesssion demonstration the building of the debian package is shown in the code block following this one.

cmake_minimum_required (VERSION 2.6)

add_compile_options (-fdiagnostics-color=always)
project (gobject-test)
set (CMAKE_C_FLAGS "-g ${CMAKE_C_FLAGS}")
file(GLOB GOBJECTTEST_SRC
    "src/*.h"
    "src/*.c"
)

add_executable(gobjecttest ${GOBJECTTEST_SRC})
find_package(PkgConfig REQUIRED)
SET (PACKAGES glib-2.0 gobject-2.0)

foreach(PKG_NAME IN ITEMS ${PACKAGES} )
  #MESSAGE (${PKG_NAME})
  pkg_search_module(${PKG_NAME} REQUIRED ${PKG_NAME} )
  target_link_libraries(gobjecttest ${${PKG_NAME}_LIBRARIES})
  target_include_directories(gobjecttest PUBLIC ${${PKG_NAME}_INCLUDE_DIRS})
endforeach(PKG_NAME)

#Install
include(GNUInstallDirs)
set (CMAKE_INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX})
install (TARGETS gobjecttest RUNTIME DESTINATION bin)

#build debian packages
#build debian packages
SET(CPACK_GENERATOR "DEB")
SET(CPACK_PACKAGE_VERSION_MAJOR  1)
SET(CPACK_PACKAGE_VERSION_MINOR  0)
SET(CPACK_PACKAGE_DESCRIPTION "A Demonstartion of GObject")
SET(CPACK_DEBIAN_PACKAGE_MAINTAINER "ABC") #required
INCLUDE(CPack)

$ ls ..
CMakeLists.txt  src

$ cmake ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2") 
-- Checking for one of the modules 'glib-2.0'
-- Checking for one of the modules 'gobject-2.0'
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mandar/code/091624/cmake/gobject-0.1/src

$ make
[ 33%] Building C object CMakeFiles/gobjecttest.dir/myclass.c.o
[ 66%] Building C object CMakeFiles/gobjecttest.dir/testclass.c.o
[100%] Linking C executable gobjecttest
[100%] Built target gobjecttest

$ cpack
CPack: Create package using DEB
CPack: Install projects
CPack: - Run preinstall target for: gobject-test
CPack: - Install project: gobject-test []
CPack: Create package
-- CPACK_DEBIAN_PACKAGE_DEPENDS not set, the package will have no dependencies.
CPack: - package: /home/mandar/code/091624/cmake/gobject-0.1/src/gobject-test-0.1.1-Linux.deb generated.

The current directory will contain the debian file gobject-test-1.0.1-Linux.deb

4.0 Build System: Meson

Building a debian package with a meson build system is similar to the automake process. The steps are (shown without parameters to commands)

1. Run dh_make (with options)

2. Run dh_auto_configure (with options)

3. Run dpkg-buildpackage (with options)

The below session should guide you when you are building your Debian package.

$ ls
meson.build  src
 
$ dh_make --createorig
Type of package: (single, indep, library, python)
[s/i/l/p]?
Maintainer Name     : mandar
Email-Address       : mandar@unknown
Date                : Mon, 16 Sep 2024 20:07:09 +0530
Package Name        : gobject
Version             : 0.1
License             : blank
Package Type        : single
Are the details correct? [Y/n/q]
Currently there is not top level Makefile. This may require additional tuning
Done. Please edit the files in the debian/ subdirectory now.

$ dh_auto_configure --buildsystem=meson
	cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 meson .. --wrap-mode=nodownload --buildtype=plain --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=lib/x86_64-linux-gnu
The Meson build system
Version: 0.61.2
Source dir: /home/mandar/code/091624/meson/gobject-0.1
Build dir: /home/mandar/code/091624/meson/gobject-0.1/obj-x86_64-linux-gnu
Build type: native build
Project name: gobjecttest
Project version: undefined
C compiler for the host machine: cc (gcc 11.4.0 "cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
C linker for the host machine: cc ld.bfd 2.38
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (0.29.2)
Run-time dependency gobject-2.0 found: YES 2.72.4
Run-time dependency glib-2.0 found: YES 2.72.4
Build targets in project: 1

gobjecttest undefined

  User defined options
    buildtype    : plain
    libdir       : lib/x86_64-linux-gnu
    localstatedir: /var
    prefix       : /usr
    sysconfdir   : /etc
    wrap_mode    : nodownload

Found ninja-1.10.1 at /usr/bin/ninja

$ dpkg-buildpackage -rfakeroot -us -uc -b
...
...
...
dpkg-deb: building package 'gobject' in '../gobject_0.1-1_amd64.deb'.
 dpkg-genbuildinfo --build=binary -O../gobject_0.1-1_amd64.buildinfo
 dpkg-genchanges --build=binary -O../gobject_0.1-1_amd64.changes
dpkg-genchanges: info: binary-only upload (no source code included)
 dpkg-source --after-build .
dpkg-buildpackage: info: binary-only upload (no source included)

You will find gobject_0.1-1_amd64.deb in the parent directory

We covered 4 use cases of building Debian packages - binaries, automake, cmake and meson. Next, let’s see how to host these packages in a repository on your local network so that all computers on your network can have access to your newly build packages.

5.0 Preserving configuration files across package versions

If you have configuration files that you want to preserve across installations, the Debian packaging system provides a way. Create a file in the debian directory called conffiles and include the full path of the configuration file, one per line in the file.

# Sample debain/conffiles
/etc/testclass.conf
/etc/testclass_sys.conf

Part 2: Hosting packages in a Private Apt Repository

Say you want to host the packages on an Apache server. Note, these packages will be accessible to everyone on the local network.

Let’s look at what a repository contains

1. The Deb files i.e. the package

2. Packages file: Metadata about the packages

3. Release file: Checksums of the packages

4. Release.gpg: Signature

5. InRelease: A combination of Release and Release.gpg

To create a repositoy here are the steps

1. Create a GPG key

2. Create the Packages file

3. Create a Release file (Python script below)

4. Sign the Release file with the GPG Key

5. Create the InRelease file (Python script below)

6. Host the directory on a webserver or let apt access it via ssh (we’ll take the webserver approach)

7. Edit /etc/apt/sources.list and add an entry for the new repository

The below console session should make it clear

# Manage your GPG Key
$ gpg --full-generate-key
$ gpg --list-secret-keys --keyid-format=long
$ gpg --export (key fingerprint) > username.gpg

# Generate the Packages file
$ dpkg-scanpackages . /dev/null > Packages

# Create a Release File - (Python Script below)

# Sign the Release file
$ gpg -abs -u (key fingerprint) -o Release.gpg Release

# Create the InRelease file - (Python Script below)

The configuration on the client side, your Ubuntu machine is in /etc/apt/sources.list. The relevant line is shown in the code block below. You have the change the URL and path to the GPG key.

# /etc/apt/sources.list
deb [signed-by=/etc/keys/user.gpg] http://localhost:8080/ ./

A Python script to generate an apt repository

The Python scirpt pyaptrepo.py allows you to create a GPG key and create an apt repository in the current directory as well as run a webserver in the current directory for testing.

The Python script can be found in writings/repo/python/. Below is a session with the Python script

pkgdir$ python3 ../pyaptrepo.py 
I want to:
1. List Keys
2. Export Keys
3. Generate Keys
4. Generate Apt Repository in Current Directory
5. Start Webserver/Repo In Current Directory
6. Quit
Enter Choice: 1
[1] C860FEF347CF4675FE73552D0410FD61820F86D9 Mandar Joshi (Test Key) <emailmandar@gmail.com>
[2] 7C96959D09BA29F72BE065CC1F03F82120B56E74 Autogenerated Key <mandar@mandar-Lenovo-ideapad-330S-15ARR>
[3] D03E6B4F1911F33B1B2E3A2C93BD683D538DB364 Autogenerated Key <mandar@mandar-Lenovo-ideapad-330S-15ARR>
I want to:
1. List Keys
2. Export Keys
3. Generate Keys
4. Generate Apt Repository in Current Directory
5. Start Webserver/Repo In Current Directory
6. Quit
Enter Choice: 4
 Scanning For Files
 Generating Packages file
 Generating Release File
 Signature File Release.gpg
[1] C860FEF347CF4675FE73552D0410FD61820F86D9 Mandar Joshi (Test Key) <emailmandar@gmail.com>
[2] 7C96959D09BA29F72BE065CC1F03F82120B56E74 Autogenerated Key <mandar@mandar-Lenovo-ideapad-330S-15ARR>
[3] D03E6B4F1911F33B1B2E3A2C93BD683D538DB364 Autogenerated Key <mandar@mandar-Lenovo-ideapad-330S-15ARR>
Select Key: 1
 Signature File InRelease
[1] C860FEF347CF4675FE73552D0410FD61820F86D9 Mandar Joshi (Test Key) <emailmandar@gmail.com>
[2] 7C96959D09BA29F72BE065CC1F03F82120B56E74 Autogenerated Key <mandar@mandar-Lenovo-ideapad-330S-15ARR>
[3] D03E6B4F1911F33B1B2E3A2C93BD683D538DB364 Autogenerated Key <mandar@mandar-Lenovo-ideapad-330S-15ARR>
Select Key: 1
I want to:
1. List Keys
2. Export Keys
3. Generate Keys
4. Generate Apt Repository in Current Directory
5. Start Webserver/Repo In Current Directory
6. Quit
Enter Choice: 5
Starting Webserver on port 8080
Press Ctrl+C to terminate
127.0.0.1 - - [17/Sep/2024 19:59:44] "GET /./InRelease HTTP/1.1" 200 -
127.0.0.1 - - [17/Sep/2024 20:03:35] "GET /./InRelease HTTP/1.1" 304 -
127.0.0.1 - - [17/Sep/2024 20:03:45] "GET /./gobject-test_0.1-1_amd64.deb HTTP/1.1" 200 -

You will have to run Python script yourself to get a feel of it. And that’s it, a repo is just a directory served with a few special files.

You now have the ability to build and distribute your packages with Apt. Hope you had fun! See you next time.