gitpkg - Online in the Cloud

PROGRAM:

NAME

gitpkg - export a Debian source package from nominated git revisions

SYNOPSIS

gitpkg branch [origbranch]

DESCRIPTION

If gitpkg is run in a git(1) repo with a single 'branch' specified, then it will do a git-
archive export of that branch to the DEB_DIR directory. If the package is Debian native
it will simply create a source package from it. If the package has a Debian version, then
an orig tarball will be expected to already exist for it. If an orig tarball does not
already exist then what happens next depends on the value of the gitpkg.create-fake-orig
configuration option (described below).

If gitpkg is invoked with two branches specified, then the first branch will be exported
as the unpacked complete source, while the second branch will be exported for the
orig.tar.gz. This allows all local changes to the source to be recorded in the resulting
diff.gz if a pristine upstream branch exists in the repository. If an orig tarball
already exists for the version at 'branch' then what happens next depends on the value of
the gitpkg.force-overwrite-orig configuration option (described below).

The 'branch' should always have a debian/ dir and may be any tree-ish object that is
accepted by git-archive(1). The 'origbranch', if supplied, should usually not have a
debian/ dir.

CONFIGURATION OPTIONS

Almost all gitpkg configuration is handled using git-config(1) now. The following
configuration options are supported:

gitpkg.deb-dir
May be set to override the default destination directory for exported packages.
Default is '../deb-packages'. Available to hook scripts as DEB_DIR. This may be
overridden on the command line with the --deb-dir=path option.

gitpkg.build-rootcmd
May be set to override the default command used to get root for package build
operations. Default is 'fakeroot'. Available to hook scripts as BUILD_ROOTCMD.

gitpkg.prebuild-target
May be set to a target or targets from debian/rules which will be called after the
debian source tree is exported, but before the source package is built. Some
packages may use this to generate autoconf files or the like, which should
generally not be in the repo, but which should be in the distributed package. The
target is invoked using the BUILD_ROOTCMD. A common use for this hook would be to
call the package's 'clean' target. This hook is unset by default since gitpkg
0.18. Previous to that it defaulted to the clean target. Available to hook
scripts as PREBUILD_TARGET.

gitpkg.orig-compressor
May be set to override the default compression for an exported package.orig.tar.
Default is 'gzip'. For format 3.0 packages, valid values also include xz and
bzip2. Available to hook scripts as ORIG_COMPRESSOR.

gitpkg.orig-compress-level
An optional compression level to use with gitpkg.orig-compressor. 1 is usually the
fastest and 9 is usually the smallest, with the precise details of everything
between being up to the chosen compressor. Default is unset (which will use
whatever the compressor default is). Available to hook scripts as
ORIG_COMPRESS_LEVEL.

gitpkg.deb-compressor
May be set to override the default compression used by dpkg-source(1) for exported
packages. Default (if unset) is to use whatever dpkg-source wants to use. For
format 3.0 packages, valid values also include xz and bzip2. Available to hook
scripts as DEB_COMPRESSOR.

gitpkg.deb-compress-level
An optional compression level to use with gitpkg.deb-compressor. 1 is usually the
fastest and 9 is usually the smallest, with the precise details of everything
between being up to the chosen compressor. Default is unset (which will use
whatever the dpkg-source default is, currently '9'). Available to hook scripts as
DEB_COMPRESS_LEVEL.

gitpkg.dpkg-source
May be set to pass arbitrary options verbatim to dpkg-source(1) when building the
source package. Use with caution and at your own risk. To pass multiple options
to dpkg-source you must set this option multiple times (ie. using git config --add
for each option) due to the otherwise amusing quoting requirements for options such
as "--format=3.0 (native)". Default is empty. Available to hook scripts as the
indexed array DPKG_SOURCE_OPTS. Do not use this to set the dpkg-source(1) -Z or -z
options, they should instead be set using the deb-compressor and deb-compress-level
options respectively. This may be overridden on the command line with the
--dpkg-source=arg option, which likewise must be passed multiple times to set
multiple options.

gitpkg.create-fake-orig
Sometimes both upstream source and debian support really are intermingled into a
single branch of the repo but you'd still like to make a 'non-native' package from
it. gitpkg can fake an orig tarball from such a tree out of everything but the
contents of the debian/ directory. Setting this option to 'true' makes that
behaviour the default if a single treeish is passed to gitpkg and no corresponding
orig tarball is found. Setting this option to 'false' will make gitpkg fail,
reporting an error, if a single treeish is passed and no orig tarball with the
correct version already exists for it (and none was retrieved by a hook script
prior to it being needed). If this option is unset then the user will be prompted
for the correct thing to do if this situation arises. Default is unset. Available
to hook scripts as CREATE_FAKE_ORIG.

gitpkg.force-overwrite-orig
This option controls the behaviour of gitpkg if an 'origbranch' treeish is
specified and the corresponding orig.tar for the 'branch' already exists. If this
is set to 'true', then the orig.tar will be overwritten with the repo source (to
reuse an existing orig.tar simply call gitpkg with only the single debian 'branch'
treeish you wish to export). If this is set to 'false', then it is a hard error to
attempt to export the upstream source again when the orig.tar already exists, and
gitpkg will terminate and scold you if you try. If unset you will be prompted
about whether to overwrite it or not, and the build will continue using whichever
of the two you selected. Default is unset. Available to hook scripts as
FORCE_OVERWRITE_ORIG.

gitpkg.keep-unpacked-source
This option controls whether or not the unpacked source directory is kept after the
package export and exit hook have successfully completed. If this is set to
'true', the unpacked source will be retained. If set to 'false' or unset then that
directory will be removed as the final operation before gitpkg exits if all prior
operations completed successfully. Default is unset. Available to hook scripts as
KEEP_UNPACKED_SOURCE. This may be overridden on the command line with the
--keep-unpacked-source=bool option (where 'no' or 'false' will not keep it, and any
other value, including nothing, will).

SCRIPT HOOKS

User defined scripts can be invoked from a number of points during the package build
process. They are sourced into gitpkg as bash shell snippets, in most cases in a
subshell, so they can read state variables and perform external actions, but cannot alter
the running configuration once a build is in progress. If a hook returns with a non-zero
status, then gitpkg will be terminated. (Hooks that do terminate gitpkg should take some
care not to leave too much of a mess, but also should leave enough clues intact for the
user to diagnose and fix whatever the problem was. Useful and informative error messages
should be barked to stderr before exiting in this way.)

Hook scripts may be installed on the host system outside of the repo tree, or sourced from
version controlled files in the repo itself. Both methods have advantages and risks for
different use cases. Hook scripts are activated by the local admin, by setting each
relevant git-config(1) option with the path to the script to be executed. Paths may be
absolute or relative to the directory which that hook is called from. If a hook is set,
the script must exist when it is called. Care should be taken to only enable them for use
by trusted source trees when hooking into files in the repo itself. Usually you should
enable them on a per-repo basis with git-config(1) rather than at a --global or --system
level.

A brief admonition against getting hooked:
You should avoid complicated in-package hook arrangements becoming essential for exporting
your package source. If you need them to create a particular package correctly, and need
strict version binding with the source being released, and they aren't useful to any other
package at all ... then you're quite probably doing something, or several things, quite
wrong. Else you're in such deep shit working around some broken build system that you
don't need me to tell you about it. Either way, local admin has to enable your hooks
before they can run, so if you want to be friendly to others (and yourself), then keep the
'normal' packaging work strictly inside the usual package building tools, and leave the
gitpkg hooks free for other local admins to wrap whatever automation it is they need
around things. If a particular version of the package source needs some particular
actions performed on it prior to the first source package build, then the PREBUILD_TARGET
option from above is most probably what you want rather than one of these hooks. Other
people can use that again later without needing to have gitpkg around. The aim is for
this to Help You. For some values of All Of You. So do be careful to avoid letting it
screw other people over if the hook isn't called, and/or let them know what they need to
do instead if it isn't. Ok then, there's the barb to watch out for, so back to the point
again:

Hook points
The available hook points are listed below in roughly the order that they would usually be
invoked:

gitpkg.package-config-hook
This hook runs in the top level directory of the repo gitpkg was invoked in, prior
to any operations taking place, with all git-config(1) sourced options available to
it. No detailed information about the package itself is available in the hook
environment yet, not even its name or version, only the tree-ish(es) that gitpkg
was passed by the user, but the hook may run its own self-checks based on the
current (possibly 'dirty') contents of the working tree that gitpkg was invoked in.

This hook is able to modify the gitpkg configuration variables for subsequent
operations. It can perform operations on the repo if needed, but since it needs to
be committed to the repo before it will ever be called, that may not be so useful
here in practice. Basically, it can do anything it pleases, it's just a shell
script, nothing else has really begun yet, and it has been sourced into the topmost
shell level of gitpkg.

Its operation is different from the admin-config-hook in only one respect, the path
to this hook must be relative to the TLD of the repo, and the revision of the file
that will be sourced is checked out from the 'branch' tree-ish that gitpkg was
requested to export. The file must exist in that version at the path given.

Available to hook scripts as PACKAGE_CONFIG_HOOK.

gitpkg.admin-config-hook
This hook is run after the package-config-hook returns, and differs from it in
operation only by reading a static file from the current filesystem rather than
extracting a version controlled one from the repo being exported.

This can be used by the local admin to override any package specific options, that
may have been set by the package-config-hook, with site specific configuration.
This is a policy control, not a security one. Security was all over when you let
the package-config-hook run, this just lets you override it without having to fake
up a new commit changing the package hook.

This is the last hook to run that is able to modify the gitpkg configuration and
set environment options that will be visible to later hooks. Available to hook
scripts as ADMIN_CONFIG_HOOK. This may be overridden on the command line with the
--admin-config-hook=path option.

gitpkg.pre-export-hook
This hook runs in the top level directory of the repo, after the package name and
version have been determined, and with the final package configuration including
any tweaking by the previous hooks. It cannot alter any configuration options,
only act upon them or terminate gitpkg.

This can be used to do things like invoke pristine-tar or prefetch an existing orig
tarball from some foreign source. It may perform operations on the repo if any
such are desired, or any other last minute check that needs to be done before we
actually get about the task of exporting the source we want packaged.

Available to hook scripts as PRE_EXPORT_HOOK.

gitpkg.deb-export-hook
This hook runs in the top level directory of the exported debian source,
immediately after the source has been exported from the requested tree-ish, and
immediately prior to the PREBUILD_TARGET being invoked (if provided). It cannot
alter any configuration options, only act upon them or terminate gitpkg. If this
hook terminates gitpkg, the exported source directory will be left on the system
for the user to inspect. Subsequent invocations of gitpkg for the same release
version will overwrite it though. Available to hook scripts as DEB_EXPORT_HOOK.

gitpkg.orig-export-hook
This hook runs in the top level directory of the exported 'upstream' source,
immediately after the source has been exported from the provided tree-ish, and
prior to it being compressed into a tarball. It cannot alter any configuration
options, only act upon them or terminate gitpkg. If this hook terminates gitpkg,
the exported source directory will be left on the system for the user to inspect.
Subsequent invocations of gitpkg for the same release version will overwrite it
though.

This hook is only invoked if the upstream 'origbranch' actually is exported from
the repository. If an existing orig.tar is found or has been created by some
earlier hook (and it is not being overwritten, see force-overwrite-orig above),
then the operations this hook would perform are presumed to have already happened
for this tarball and it is skipped.

It is not safe to assume that this hook will be executed before or after deb-
export-hook, and it may in fact be run in parallel with it at some point in the
future. They both will be entered after pre-export-hook returns, and exit-hook
will not begin until (at least) after both have returned. What else happens in the
middle of all that we make no firm promises about at this stage.

Available to hook scripts as ORIG_EXPORT_HOOK.

gitpkg.exit-hook
This hook runs in the directory where the package .dsc was deposited by dpkg-
source(1), after all internal gitpkg operations have successfully completed. It's
too late to alter any configuration options, or even to terminate gitpkg really.
You can pretty much do what you like from this one, anything that goes wrong from
here on is your own doing. Available to hook scripts as EXIT_HOOK. This may be
overridden on the command line with the --exit-hook=path option.

Hook Environment
The following variables are made available for hook scripts, in addition to those already
listed as shadowing a git-config option from above. Not all of them are valid/useful at
all hook points, see the hook documentation above for the exceptions applying to specific
hooks.

GITPKG_HOOK_API
Permits hook scripts to query what interfaces are available to them. Has only two
numeric components separated by a '.' of which the number to the right of point
will get incremented every time we add some new variable a hook might access, or
add some new knob it might tweak where existing interfaces have not have changed.
If we do screw up and need to change some current interface, the number to the left
will get bumped. The current API version is 0.2

Available in API version 0.1
These variables have been available to hooks since gitpkg version 0.13

GITPKG_TREEISH
The user-passed debian 'branch' tree-ish that gitpkg was invoked to export.

GITPKG_ORIG_TREEISH
The 'origbranch' tree-ish that gitpkg was invoked with. This will be empty if only
a single 'branch' tree-ish was specified.

DEB_SOURCE
The name of the source package to create, without any versioning. As seen in the
Source: field of dpkg-parsechangelog(1).

DEB_VERSION
The version of the source package to create, without any epoch. As seen in the
name of the .diff.gz and .dsc files.

UPSTREAM_VERSION
The version of the source package to create, without any debian version. As seen
in the name of the orig tarball. For native packages this will be the same as
DEB_VERSION.

DEB_ORIG
The full versioned filename of the orig tarball to use or create. This variable is
empty for native packages without a Debian version part.

DEB_DSC
The full filename of the package .dsc that will be or has been created.

DEB_PACKAGE
The directory name of the debianised source tree to pass to dpkg-source(1).

REPO_DIR
An absolute path to the top level directory of the git repo we are exporting from.
Usually, if you need to look out of the tree that you were dropped in, you're
probably doing something (at the) wrong (time), but there are exceptions, and being
able to query git-config options is one of them. That's mostly what this one is
for right now. See the repo-config-helper documented below. Be careful if you do
use it for much else.

Available in API version 0.2
These variables have been available to hooks since gitpkg version 0.24

GITPKG_AOPTS
An associative array containing the command line options not parsed by gitpkg
itself which can be used to override the behaviour of a hook. The array is keyed
on the names of the options with the '--' removed. If an option was passed
multiple times, only the last value passed will be stored in this array. For
example:

$ gitpkg --my-option=foo --option2 --opt=oops --opt='bar baz'

Will give:

${GITPKG_AOPTS[my-option]} = "foo"
${GITPKG_AOPTS[option2]} = ""
${GITPKG_AOPTS[opt]} = "bar baz"

GITPKG_IOPTS
An indexed array containing the command line options not parsed by gitpkg itself
which can be used to override the behaviour of a hook. The array contains the
literal option strings passed and so can be used to access options which are
intended to be passed multiple times. For example:

$ gitpkg --my-option=foo --option2 --opt=oops --opt='bar baz'

Will give:

${GITPKG_IOPTS[0]} = "--my-option=foo"
${GITPKG_IOPTS[1]} = "--option2"
${GITPKG_IOPTS[2]} = "--opt=oops"
${GITPKG_IOPTS[3]} = "--opt=bar baz"

The extract_values_for_option function in repo-config-helper (see below for details
of it) can be used to further parse this array to obtain all the value(s) for a
specific option.

Hook Library
There are some canned hook scripts for various tasks available in /usr/share/gitpkg/hooks
which currently include:

cowpoke-exit-hook
A simple exit hook which sends the exported package off for building using cowpoke(1).
To enable it:

$ git config gitpkg.exit-hook /usr/share/gitpkg/hooks/cowpoke-exit-hook

Additional git-config(1) configuration options:

gitpkg-cowpoke-exit-hook.ask-first
If 'true' prompt for confirmation before calling cowpoke. Default is to just
go ahead and do it.

gitpkg-cowpoke-exit-hook.options
May include any other options to pass verbatim to cowpoke. To pass multiple
options, set this multiple times, once for each option. This may be
overridden on the command line with the --cowpoke=arg option, which likewise
must be passed multiple times to set multiple options.

dpkg-buildpackage-exit-hook
A simple exit hook to build binary packages locally with dpkg-buildpackage(1). To
enable it:

$ git config gitpkg.exit-hook /usr/share/gitpkg/hooks/dpkg-buildpackage-exit-hook

Additional git-config(1) configuration options:

gitpkg-dpkg-buildpackage-exit-hook.ask-first
If 'true' prompt for confirmation before calling dpkg-buildpackage. Default
is to just do it.

gitpkg-dpkg-buildpackage-exit-hook.options
May include any other options to pass verbatim to dpkg-buildpackage. To pass
multiple options, set this multiple times, once for each option. This may be
overridden on the command line with the --dpkg-bp=arg option, which likewise
must be passed multiple times to set multiple options.

gitpkg-dpkg-buildpackage-exit-hook.build-log
If set 'false' don't save a log of the build process, the default is to record
one.

pristine-tar-pre-export-hook
A hook to extract an orig tarball using pristine-tar. Which orig to extract is
determined by the package version of the 'branch' tree-ish. To enable it:

$ git config gitpkg.pre-export-hook /usr/share/gitpkg/hooks/pristine-tar-pre-export-hook

If a pristine-tar branch is not found in the repo, then gitpkg will be terminated.

quilt-patches-deb-export-hook
This hook reads a list of revision ranges suitable for git-format-patch(1) from the
file debian/source/git-patches, one per line, and exports them to the debian/patches
directory in a form suitable for (format 3.0) quilt packages. It is not required for
creating such packages, but permits you to separate out individual patches however you
please from the default single patch that is otherwise created by dpkg-source.

To enable it:

$ git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/quilt-patches-deb-export-hook

The contents of debian/source/git-patches may include comments (on any line beginning
with a #), empty lines, and expressions of a range of commits. The revision ranges
may include $DEB_VERSION, $UPSTREAM_VERSION, $DEB_REF or $UPSTREAM_REF. The first
pair will be substituted with the version of the package being exported, the second
pair with those version strings after mangling by sanitise_git_ref to remap them to a
legal git refname. Using the sanitised versions is to be preferred in most cases.
For example:

# Export all commits between these two treeishes,
# based on the version of the package being exported.

upstream/$UPSTREAM_REF..patches/$DEB_REF

debcherry-deb-export-hook
This hook invokes git-debcherry(1) to find and export patches to the upstream source
in a form suitable for use with (format 3.0) quilt packages. It allows for a more
natural (and automatic) workflow than the quilt-patches hook above by searching for
patches made to the packaging branch that have not yet been applied upstream.

To enable it:

$ git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/debcherry-deb-export-hook

In order to use this hook, a ${DEB_ORIG}.commit file must be created which contains
the treeish of the exported upstream source in the repository. This will be created
automatically (if this hook is enabled) when you export an upstream tarball by passing
both branch and origbranch to gitpkg, or if you use the pristine-tar-pre-export-hook,
which determines an appropriate commit corresponding to the tarball. If your upstream
tarball is created using some other mechanism you will need to ensure that file is
created yourself.

If using this hook, you may wish to document that in your repository with something
similar to the text in /usr/share/doc/gitpkg/examples/README.debcherry-export as a
convenience to other users. Your package will still be exportable without this hook
enabled, it just won't have the upstream patches individually separated out into a
quilt series.

Hook Library Helpers
These are even more trivial snippets, for operations which may be shared by several
scripts. Also found in /usr/share/gitpkg/hooks. Usually these would be sourced by other
scripts rather than being hooked to directly.

repo-config-helper
Provides a simple wrapper around `git config`, which ensures it is called from the
repo tree where any repo-specific config options may be stored. Useful to scripts
which aren't called from inside the repo tree, but which do have git-config options of
their own to query.

Provides the sanitise_git_ref shell function which remaps character strings that are
illegal to use in a git refname.

Provides the extract_values_for_option shell function which can be used to extract an
array of the values for a particular option from GITPKG_IOPTS.

See the content of that file itself for more detailed documentation on the functions
that it provides.

INTERACTIVIY

If you intend to call gitpkg from your own scripts, then you should note that there are
two situations when it may prompt interactively by default. There is no One True Sane
Default for these cases, so it's better to just ask the user and continue than to make
them start the whole process again in the likely case where they have called gitpkg
directly. For details, see the gitpkg.force-overwrite-orig and gitpkg.create-fake-orig
config options above. You should set both explicitly to the behaviour that you desire
from them if gitpkg should never become interactive.

WORKFLOW

Though gitpkg explicitly does not try to force any particular workflow procedure upon you
in order to make full use of it, it probably is worth making quick mention of at least one
simple way to manage Debian packages in git.

One common repo structure is to keep pristine upstream source on one branch, which is
updated either directly from an upstream repo or by importing tar archives to it
periodically, with the Debian patched source on another branch. In this situation the
task of preparing a new upstream release from a tarball might look a bit like this:

Check out the upstream branch
$ cd myrepo
$ git checkout upstream

Remove all old upstream files from the repo
$ rm -rf $(all_files_except .git)

Unpack the new tarball in their place
$ tar zxf $new_upstream.tar.gz

Let git figure out what is renamed/new/gone by itself.
Make sure you don't have things like vim .swp files lurking
in the tree still at this point.
$ git add .
$ git commit -a
$ git tag v$upstream_version

Prepare the Debian branch
$ git checkout debian
$ git merge upstream
$ $(update changelog and other debian patches etc.)
$ git commit -a
$ git tag v${upstream_version}-$debian_version

Make a release
$ gitpkg v${upstream_version}-$debian_version v$upstream_version
$ cd ../deb-packages/mypackage && dpkg-buildpackage ...

Use gitpkg online using onworks.net services