Page Comparison

...

deb-src lines instruct apt where it should get upstream source packages. In our example it is buster (a codebase of debian). Add an architecture:

Code Block

language	bash
title	add architecture:

dpkg --add-architecture amd64

And update the data:

Code Block

language	bash
title	update existing packages:

apt-get update

...

Code Block

title	update errors

du3p03% sudo apt-get update
Hit:2 https://apt2.dilos.org/dilos dg3-unstable InRelease
Hit:1 http://mirrors.edge.kernel.org/debian buster-updates InRelease           
Hit:3 https://apt2.dilos.org/dilos du3-unstable InRelease                      
Hit:4 https://apt2.dilos.org/dilos dg2-unstable InRelease                      
Hit:5 https://apt2.dilos.org/dilos du2-unstable InRelease                      
Hit:6 http://mirrors.edge.kernel.org/debian buster-proposed-updates InRelease
Hit:7 http://mirrors.edge.kernel.org/debian buster-backports InRelease         
Hit:8 http://mirrors.edge.kernel.org/debian buster InRelease                   
Err:1 http://mirrors.edge.kernel.org/debian buster-updates InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
Err:6 http://mirrors.edge.kernel.org/debian buster-proposed-updates InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
Err:7 http://mirrors.edge.kernel.org/debian buster-backports InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
Err:8 http://mirrors.edge.kernel.org/debian buster InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138 NO_PUBKEY EF0F382A1A7B6500 NO_PUBKEY DCC9EFBF77E11517
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://mirrors.edge.kernel.org/debian buster-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://mirrors.edge.kernel.org/debian buster-proposed-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://mirrors.edge.kernel.org/debian buster-backports InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://mirrors.edge.kernel.org/debian buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138 NO_PUBKEY EF0F382A1A7B6500 NO_PUBKEY DCC9EFBF77E11517
W: Failed to fetch http://mirrors.kernel.org/debian/dists/buster-updates/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
W: Failed to fetch http://mirrors.kernel.org/debian/dists/buster-proposed-updates/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
W: Failed to fetch http://mirrors.kernel.org/debian/dists/buster-backports/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
W: Failed to fetch http://mirrors.kernel.org/debian/dists/buster/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138 NO_PUBKEY EF0F382A1A7B6500 NO_PUBKEY DCC9EFBF77E11517
W: Some index files failed to download. They have been ignored, or old ones used instead.

This means you shoud should add these public keys to the system. Do it as:

Code Block

title	add public keys

du3p03% sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138 EF0F382A1A7B6500 DCC9EFBF77E11517
Executing: /tmp/apt-key-gpghome.0nsKnHHLVF/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138 EF0F382A1A7B6500 DCC9EFBF77E11517
gpg: key DCC9EFBF77E11517: public key "Debian Stable Release Key (10/buster) <debian-release@lists.debian.org>" imported
gpg: key EF0F382A1A7B6500: public key "Debian Stable Release Key (9/stretch) <debian-release@lists.debian.org>" imported
gpg: key DC30D7C23CBBABEE: public key "Debian Archive Automatic Signing Key (10/buster) <ftpmaster@debian.org>" imported
gpg: Total number processed: 3
gpg:               imported: 3

Ok, now these keys have been imported, let's update it once again:

...

How to port a new package

1) Choose a package you want to port. For example, apache2. Go to the debian packages page at https://packages.debian.org/buster-backports/apache2 and copy the correct name of the source package in the right column ("Download Source Package apache2:"). Remember (or copy) "apache2". Install additional packages to manipulate the source code:

Code Block

title	dpkg-dev

du3p03% sudo apt-get install dpkg-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  binutils libdpkg-perl make patch
Suggested packages:
  binutils-doc debian-keyring gcc | c-compiler bzr make-doc ed diffutils-doc
Recommended packages:
  build-essential gcc | c-compiler fakeroot libalgorithm-merge-perl
  libfile-fcntllock-perl liblocale-gettext-perl
The following NEW packages will be installed:
  binutils dpkg-dev libdpkg-perl make patch
0 upgraded, 5 newly installed, 0 to remove and 7 not upgraded.
Need to get 6277 kB of archives.
After this operation, 46.4 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://apt2.dilos.org/dilos du3-unstable/main solaris-i386 binutils solaris-i386 2.31.1-1+dilos3 [4409 kB]
Get:2 https://apt2.dilos.org/dilos du2-unstable/main solaris-i386 libdpkg-perl all 1.19.7+dilos5 [706 kB]
Get:3 https://apt2.dilos.org/dilos du2-unstable/main solaris-i386 patch solaris-i386 2.7.6-3+dilos1 [137 kB]
Get:4 https://apt2.dilos.org/dilos du2-unstable/main solaris-i386 make solaris-i386 4.1-9.1-2+dilos2 [316 kB]
Get:5 https://apt2.dilos.org/dilos du2-unstable/main solaris-i386 dpkg-dev all 1.19.7+dilos5 [708 kB]
Fetched 6277 kB in 8s (740 kB/s)                                               
Selecting previously unselected package binutils.
(Reading database ... 15551 files and directories currently installed.)
Preparing to unpack .../binutils_2.31.1-1+dilos3_solaris-i386.deb ...
Unpacking binutils (2.31.1-1+dilos3) ...
Selecting previously unselected package libdpkg-perl.
Preparing to unpack .../libdpkg-perl_1.19.7+dilos5_all.deb ...
Unpacking libdpkg-perl (1.19.7+dilos5) ...
Selecting previously unselected package patch.
Preparing to unpack .../patch_2.7.6-3+dilos1_solaris-i386.deb ...
Unpacking patch (2.7.6-3+dilos1) ...
Selecting previously unselected package make.
Preparing to unpack .../make_4.1-9.1-2+dilos2_solaris-i386.deb ...
Unpacking make (4.1-9.1-2+dilos2) ...
Selecting previously unselected package dpkg-dev.
Preparing to unpack .../dpkg-dev_1.19.7+dilos5_all.deb ...
Unpacking dpkg-dev (1.19.7+dilos5) ...
Setting up binutils (2.31.1-1+dilos3) ...
Setting up make (4.1-9.1-2+dilos2) ...
Setting up patch (2.7.6-3+dilos1) ...
Setting up libdpkg-perl (1.19.7+dilos5) ...
Setting up dpkg-dev (1.19.7+dilos5) ...
Processing triggers for install-info (6.5.0.dfsg.1-1+dilos1) ...
du3p03%

2) Now download the source code into your build environment:

Code Block

language	bash
title	download source code

du3p03% apt-get source apache2
Reading package lists... Done
NOTICE: 'apache2' packaging is maintained in the 'Git' version control system at:
https://salsa.debian.org/apache-team/apache2.git
Please use:
git clone https://salsa.debian.org/apache-team/apache2.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 10.4 MB of source archives.
Get:1 http://mirrors.edge.kernel.org/debian buster-backports/main apache2 2.4.43-1~bpo10+1 (dsc) [3530 B]
Get:2 http://mirrors.edge.kernel.org/debian buster-backports/main apache2 2.4.43-1~bpo10+1 (tar) [9348 kB]
Get:3 http://mirrors.edge.kernel.org/debian buster-backports/main apache2 2.4.43-1~bpo10+1 (asc) [488 B]
Get:4 http://mirrors.edge.kernel.org/debian buster-backports/main apache2 2.4.43-1~bpo10+1 (diff) [1015 kB]
Fetched 10.4 MB in 6s (1660 kB/s)                                              
dpkg-source: info: extracting apache2 in apache2-2.4.43
dpkg-source: info: unpacking apache2_2.4.43.orig.tar.gz
dpkg-source: info: unpacking apache2_2.4.43-1~bpo10+1.debian.tar.xz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying fhs_compliance.patch
dpkg-source: info: applying no_LD_LIBRARY_PATH.patch
dpkg-source: info: applying suexec-CVE-2007-1742.patch
dpkg-source: info: applying customize_apxs.patch
dpkg-source: info: applying build_suexec-custom.patch
dpkg-source: info: applying reproducible_builds.diff
dpkg-source: info: applying buffer-http-request-bodies-for-tlsv13.diff
dpkg-source: info: applying tlsv13-add-logno.diff
du3p01%

3) Now we can port the package. First of all you should install (possible) all dependencies to build the package. At this stage the build system does now know anything about dependencies, install them manually:

Code Block

title	list of build dependencies

du3p01% cd apache2-2.4.43 
du3p01% dpkg-checkbuilddeps            
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper (>= 10) bison jdupes libapr1-dev (>= 1.6.0) libaprutil1-dev (>= 1.6.0) libbrotli-dev liblua5.2-dev libnghttp2-dev libpcre3-dev libssl-dev (>= 1.1.1) libxml2-dev lsb-release zlib1g-dev libcurl4-openssl-dev | libcurl4-dev libjansson-dev

4) Now you can try to install them:

Code Block

language	bash
title	install dependencies

du3p01% sudo apt-get install debhelper bison jdupes libapr1-dev libaprutil1-dev libbrotli-dev liblua5.2-dev libnghttp2-dev libpcre3-dev libssl-dev libxml2-dev lsb-release zlib1g-dev libcurl4-openssl-dev libjansson-dev
sudo: unable to resolve host du3p01: node name or service name not known
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package jdupes

...

Code Block

title	jdupes

du3p03% apt-get source jdupes 
Reading package lists... Done
NOTICE: 'jdupes' packaging is maintained in the 'Git' version control system at:
https://salsa.debian.org/debian/jdupes.git
Please use:
git clone https://salsa.debian.org/debian/jdupes.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 124 kB of source archives.
Get:1 http://mirrors.edge.kernel.org/debian buster-backports/main jdupes 1.18.1-1~bpo10+1 (dsc) [1878 B]
Get:2 http://mirrors.edge.kernel.org/debian buster-backports/main jdupes 1.18.1-1~bpo10+1 (tar) [109 kB]
Get:3 http://mirrors.edge.kernel.org/debian buster-backports/main jdupes 1.18.1-1~bpo10+1 (diff) [13.0 kB]
Fetched 124 kB in 1s (80.2 kB/s)
dpkg-source: info: extracting jdupes in jdupes-1.18.1
dpkg-source: info: unpacking jdupes_1.18.1.orig.tar.gz
dpkg-source: info: unpacking jdupes_1.18.1-1~bpo10+1.debian.tar.xz
du3p03%

Code Block

title	check jdupes dependencies

du3p03% cd jdupes-1.18.1 
du3p03% dpkg-checkbuilddeps
dpkg-checkbuilddeps: warning: cannot determine CC system type, falling back to default (native compilation)
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 13)

...

Recheck build dependency again to be sure threse there are no unmet dependencies:

...

There are three files (or more) you should check.

debian/changelog

create package directory and copy Makefile from another package

update Makefile with build flags (CONFIGURE_OPTIONS)

Code Block

language	bash
title	build package by:

gmake JOBS=4 install

Create directory 'debs' with structure of files for DEB package build by dpkg-buildpackage.

For example take a look another components

Code Block

language	bash
title	build package(s):

gmake JOBS=4 deb

if you want build packages in your component root directory you can add flag WS_DEBS:

Code Block

language	bash
title	example:

gmake JOBS=4 WS_DEBS=`pwd` deb

as result you can find package at your component directory and be able to install it by 'dpkg -i <your package.deb>'

For more information about environment variables or build flags you can take a look <du2>/make-rules/deb_src.mkThis file controls the target package version and the repo you wnat to place the binary packages to. Let's see at the structure:

Code Block

title	changelog

jdupes (1.18.1-1~bpo10+1) buster-backports; urgency=medium

  * Rebuild for buster-backports.

 -- Joao Eriberto Mota Filho <eriberto@debian.org>  Wed, 15 Jul 2020 17:56:23 -0300

/*** cut ***/

jdupes - the package name
1.18.1-1~bpro10+1 - version of the package
buster-backports - target repo
rebuild for buster-backports - a message which changes have been made (and why)
-- Joao Eriberto Mota Filho <eriberto@debian.org> Wed, 15 Jul 2020 17:56:23 -0300 - who made the changes and when

We add a new record to the file on top (like the LIFO or stack).

Code Block

title	changes in changlog

jdupes (1.18.1-1~bpo10+1+dilos1) unstable; urgency=medium

  * Build for DilOS.

 -- DilOS Team <dilos@dilos.org>  Mon, 10 Aug 2020 12:06:57 +0300

jdupes (1.18.1-1~bpo10+1) buster-backports; urgency=medium

  * Rebuild for buster-backports.

 -- Joao Eriberto Mota Filho <eriberto@debian.org>  Wed, 15 Jul 2020 17:56:23 -0300

As you see we added +dilos1 to the version (to mark our changes), the repo now is unstable (if you plan to contribute the result to the main repo), the message says it is a build for DilOS and the last line shows who did the changes. To generate the date just use date -R command in the shell:

Code Block

title	date -R

du3p03% date -R
Mon, 10 Aug 2020 12:06:57 +0300

Also you can use the template created by gmake debgenlog command, go to your local copy of the du3 repository, and than components/<any_component> (apt for example):

Code Block

# cd <path_to_du3>/du3/components/apt
# gmake debgenlog
/ws/home/akozlov/du3/tools/deb_gen_log.pl
dummy (1.2.3.4) unstable; urgency=low

  * build for dilos

 -- DilOS Team <dilos@dilos.org>  Sun, 31 Jan 2021 22:08:33 +0300

Here you just need to change name and version of the component.

The file is ready and we're preparing next file...

debian/control

This file is used to specify dependencies o build the package(s):

Code Block

title	dependencies

Build-Depends: debhelper-compat (= 13)

You see it ad check stage.

Also the file contains list of the producced packages and their architechtures:

Code Block

title	packages and architectures

Package: jdupes
Architecture: any
Multi-Arch: foreign
Depends: ${misc:Depends}, ${shlibs:Depends}
Description: identify and delete or link duplicate files
/*** cut ***/

Built package will have name jdupes (as the source package). Architecture any means for any platform. It could be:

any - for any architectures (binary package)
all - for all plafroms (documentation, configurations etc)
linux-any - platform specific package (in this example for linux only), we cannot build such packages (as is)

In this example we don't need to change anything, just leave it as is.

debian/rules

Speaking shortly it is a Makefile that is used to build all target packages (from the contol file, using version from changelog). Usualy the file contains valid options to build packages for our platform, but from time to time you need to change something. For example, gnu ld accepts -Wl,-as-needed option, illumos ld does not, just remove this option from the list. Other cases are too specific to describe them as example, just ask us on #dilos at FreeNode or via the mail list.

debian/pre* and post* scripts

Another stuff is he pre/post scripts, probably you should change them (if package has them). Our platform should be able to bootstrap zones, it means the packages could be installed into an alternative root. For this purpose we use BASEDIR environment variable to specify it (it comes from apt/apt-get). Let's take an example from reprepro package:

Code Block

title	postinst

desktop% pwd
/export/home/denis/projects/dilos/du3/components

desktop% cat reprepro/debian/postinst
#!/bin/sh

set -e
if [ "${BASEDIR:=/}" = "/" ]; then
	BASEDIR=""
fi

case "$1" in
    configure)
        if [ -f ${BASEDIR}/etc/bash_completion.d/reprepro.dpkg-remove ] ; then
		echo "Removing obsolete unmodified conffile /etc/bash_completion.d/reprepro"
		rm -f ${BASEDIR}/etc/bash_completion.d/reprepro.dpkg-remove
	fi
    ;;
esac

#DEBHELPER#

exit 0

If we install the package as is (in the current environment) the BASEDIR is "/" and we just clear it.

quilt patch

If during the porting process you need to change something outside the debian folder, then these changes must be placed in the dilos.patch:

Code Block
# quilt new dilos.patch # quilt add <modifying_file> # quilt refresh

du3/components/<package>

When the porting has done and all packages built you should place it into the du3 repo. Create a directory (usualy by the source package name), in our examples it is apache2 and/or jdupes.

Code Block

title	make directory

du3p03% mkdir -p du3/components/jdupes

Create the Makefile to build the package by CI (jenkins):

Code Block

title	Makefile

du3p01% cat du3/components/jdupes/Makefile 
#
# This file and its contents are supplied under the terms of the
# Common Development and Distribution License ("CDDL"), version 1.0.
# You may only use this file in accordance with the terms of version
# 1.0 of the CDDL.
#
# A full copy of the text of the CDDL should have accompanied this
# source.  A copy of the CDDL is also available via the Internet at
# http://www.illumos.org/license/CDDL.
#

#
# Copyright (c) 2020 DilOS Team <dilos@dilos.org>
#
include ../../make-rules/shared-macros.mk

COMPONENT_NAME= jdupes

include ../../make-rules/deb_src2.mk

#
build:
	@echo "Nothing build"

install:
	@echo "Nothing install"

clean:
	$(RM) -r $(BUILD_DIR)
#	$(RM) *.changes
#	$(RM) *.deb

clobber: clean

test: $(NO_TESTS)

Make sure there are no "white spaces" at the of lines:

Code Block

title	debian copy

sed -i 's/ *$//' Makefile

Copy the whole debian directory from the package tree to the component repo directory, add it to git, commit it and create a PR (pull request).

Code Block

title	debian copy

du3p03% cp -R <jdupes-version>/debian du3/componens/jdupes

produce a new source

To build a source package do the command from the main package tree:

Code Block

title	build source package

du3p03% sudo dpkg-buildpackage -d -S -us -uc -sa
dpkg-buildpackage: info: source package jdupes
dpkg-buildpackage: info: source version 1.18.1-1~bpo10+1+dilos1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by DilOS Team <dilos@dilos.org>
 dpkg-source --before-build .
dpkg-source: warning: can't parse dependency tree:native
dpkg-source: warning: can't parse dependency tree:native
 debian/rules clean
dh clean
   dh_auto_clean
        make -j1 distclean
make[1]: Entering directory '/export/home/denis/projects/dilos/porting/jdupes-1.18.1'
rm -f jdupes.o jody_paths.o jody_sort.o jody_win_unicode.o jody_strtoepoch.o string_malloc.o jody_cacheinfo.o act_deletefiles.o act_linkfiles.o act_printmatches.o act_summarize.o act_printjson.o xxhash.o  act_dedupefiles.o jdupes-standalone build_date.h jdupes jdupes.exe *~ *.gcno *.gcda *.gcov
rm -f *.pkg.tar.xz
rm -f -r jdupes-*-win*/ jdupes-*-win*.zip
rm -f -r jdupes-*-mac*/ jdupes-*-mac*.zip
make[1]: Leaving directory '/export/home/denis/projects/dilos/porting/jdupes-1.18.1'
   dh_clean
 dpkg-source -b .
dpkg-source: warning: can't parse dependency tree:native
dpkg-source: warning: can't parse dependency tree:native
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building jdupes using existing ./jdupes_1.18.1.orig.tar.gz
dpkg-source: info: building jdupes in jdupes_1.18.1-1~bpo10+1+dilos1.debian.tar.xz
dpkg-source: info: building jdupes in jdupes_1.18.1-1~bpo10+1+dilos1.dsc
 dpkg-genbuildinfo --build=source
dpkg-genbuildinfo: warning: File::FcntlLock not available; using flock which is not NFS-safe
 dpkg-genchanges -sa --build=source >../jdupes_1.18.1-1~bpo10+1+dilos1_source.changes
dpkg-genchanges: info: including full source code in upload
 dpkg-source --after-build .
dpkg-source: warning: can't parse dependency tree:native
dpkg-source: warning: can't parse dependency tree:native
dpkg-buildpackage: info: full upload (original source is included)
du3p03%

The file *.changes (jdupes_1.18.1-1~bpo10+1+dilos1_source.changes) has all files that should be copied as the source package.

That's all.

Versions Compared

Old Version 4

New Version Current

Key

How to port a new package

debian/changelog

debian/control

debian/rules

debian/pre* and post* scripts

quilt patch

du3/components/<package>

produce a new source