How to handle packaging for complex situations

Assume you've managed to build the sofware, provide required patches, and you want to finish the port.

Know the software

Identify options: The first step is usually to identify build options. You will often have to read the configuration log, see what stuff your port auto-detects. Read the configure script options. Read the port documentation for extra stuff.
Make options work: Recompile your port with various options. Install extra dependencies. Make sure your port detects them correctly. Add supplementary patches to ensure compilation. Test the result, and verify extra stuff does work.
Identify missing software: Some dependencies won't be fullfilled because the missing software has not yet been ported. It is highly recommended to explicitly disable those options. Failure to do that breaks bulk builds all the time: people port new software and import it, and soon after, old ports stop building because they detect the dependency, try to use it, and fail to build or package.
Check run-time dependencies versus build-dependencies: Update your packing-list with make plist. Use port-lib-depends-check to see what libraries your software needs (that will end up in LIB_DEPENDS or WANTLIB, usually). Identify various files and binaries in the dependencies that have to be present for the port to work.

By this point, you should have a fair understanding of your ports' working.

Figure out important options

You won't care about some options. It makes no sense to disable some stuff if it always work, and if the dependencies are quite small. Take special notes of licences on dependencies, especially the PERMIT* stuff. As a rule, even if a dependency is very small, if it affects the licensing of the resulting package, you will have to explicitly take care of it.

Considering all possibly options, you should be left with a much smaller set of options for your port, mostly depending on what packages are needed to run the software. For now, do not worry about build dependencies. Remember that the OpenBSD ports system is focused on the end user, and the end user will want to install binary packages, so it doesn't matter if you need a huge piece of software to build your port if it doesn't show up as a library or runtime dependency.

The ideal case: MULTI_PACKAGES and PSEUDO_FLAVORS

By now, you should have a fairly good idea of:

which new files show up when you activate each option
which libraries/runtime files are needed when you activate each option

In the ideal case, build options will simply create new files, with new dependencies, and not affect other stuff. This is a fairly common scenario for plugin frameworks: you add one library, you end up with a new plugin. This also happens fairly often for core applications with a graphics front-end: the console application is built every time, and the x11 interface shows up as a separate binary.

In this case, try a MULTI_PACKAGES: set the MULTI_PACKAGES variable to a list of -sub packages, add COMMENTS, and look at your packaging. Basically, MULTI_PACKAGES only affects the packaging: if you have MULTI_PACKAGES=-s1 -s2 all stuff relevant to the package will exist in two variants: COMMENT-s1 for the first package, COMMENT-s2 for the second package, PLIST-s1, PLIST-s2, DESCR-s1, DESCR-s2. You need to write those COMMENT-s1 and COMMENT-s2 in the Makefile, and to split your PLIST into two parts, and to create DESCR-s1/DESCR-s2. You will also need to specify separate PKGNAMES for all subpackages.

It is a good idea to start with the minimal framework work required: just copy the existing description and comments, because you will have to fiddle with MULTI_PACKAGES and stuff before you polish this.

Once you've separated the files properly, you will need to check dependencies: LIB_DEPENDS, WANTLIB, and RUN_DEPENDS will be split for each subpackage. It is usually time to check that your multi packaging "works", and that those nasty dependencies you don't want to force on the user are indeed relegated to a specific subpackage.

Assuming everything works, you're mostly done. Just pick reasonable names for the various packages, and fill in the comments and descriptions. The end-user will be able to just install the package(s) they want.

But wait. What about the build, you say ? Well, having a lot of dependencies during build is not a problem. Most packages are built by the OpenBSD team using special build runs (known as bulk-builds) where a developer just builds all possible packages on a dedicated machine (or several, for slow architectures). Since everything will get built, having big dependencies is not an issue. Building the same thing several times, is an issue, though, which is why MULTI_PACKAGES are the best way to handle options (when possible): one build, one set of packages to test, better quality overall...

If you also want to help people who build packages themselves, you may consider adding PSEUDO_FLAVORS. A pseudo-flavor is a way to tweak an option (say, disable the graphical interface) that's very similar to actual flavors. In fact, the biggest difference is a functional difference: a pseudo flavor should only affect the set of packages that get built, but it is never allowed to modify the actual package contents.

For instance, assuming you separated the graphical interface into a separate subpackage (MULTI_PACKAGES=-core -x11), you could create a pseudo flavor no_x11 that avoids building the -x11 subpackage. The crucial point is that this flavor should NOT affect the -core package in any way.

You would end up with a Makefile that looks something like this:

CATEGORIES = app
COMMENT-core = foo core application
COMMENT-x11 = foo graphical interface
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-core-$V
PKGNAME-x11 = foo-x11-$V
PSEUDO_FLAVORS = no_x11
FLAVOR ?= 
CONFIGURE_STYLE = gnu

MULTI_PACKAGES = -core
WANTLIB = c m crypto ssl
WANTLIB-x11 = ${WANTLIB} X11 Xt
RUN_DEPENDS-x11 = ::${BASE_PKGPATH},-core

.if ${FLAVOR:L:Mno_x11}
CONFIGURE_ARGS += --disable-x11
.else
MULTI_PACKAGES += -x11
.endif

.include <bsd.port.mk>

Notice that you only have to write a very small conditional section in the Makefile: the system doesn't care at all that you define extra variables.

Interdependencies between subpackages

MULTI_PACKAGE setups used to be asymetric, with a -main subpackage and other subpackages, with the -main subpackage always built, and other subpackages possibly depending upon it. The current situation is totally symetric: any subpackage can depend on any other. The infrastructure has specific provisions to avoid looping indefinitely.

Specific attention must be provided to library inter-dependencies: they cannot be specified as WANTLIB, since WANTLIB do not contain any marker that says which package they come from. Rather, they must be specified as LIB_DEPENDS. Normal LIB_DEPENDS are checked at the start of build, and during packaging. If a LIB_DEPENDS specifies one of the subpackages currently being built, then the infrastructure will detect this and only check the dependency at packaging time (and thus packages may be created in a specific order to satisfy interdependencies).

The infrastructure provides specific variables to help in writing inter-dependencies: BUILD_PKGPATH contains the PKGPATH used during building the current packages, taking flavors and pseudo-flavors into account. It is highly recommanded to use this variable instead of rolling your own: failure to do so will often trigger rebuilds in interesting flavors situations. For instance:

...
FLAVORS = a b
FLAVOR ?= 
MULTI_PACKAGES = -p1 -p2
LIB_DEPENDS-p1 = foo::some/pkgpath,-p2
...

If you go on and build in some/pkgpath with FLAVOR=a, then creating the subpackage for -p1 will trigger a rebuild with FLAVOR=''. You would write

LIB_DEPENDS-p1 = foo:${BUILD_PKGPATH},-p2

instead.

There is also a BASE_PKGPATH variable, which does not take pseudo-flavors into account. This variable has limited applicability: it corresponds to a transition between old MULTI_PACKAGES and newer ones, where the old "main" subpackage did not have any marker in its pkgpath, and thus the corresponding package needs a @pkgpath ${BASE_PKGPATH} in its packing-list. (In general, pseudo-flavors are build information, and should not make their way into packages and packing-lists).

True FLAVORS, and PKGNAMES

There are some cases where configuration options are too invasive, and you will have to add true flavors to the Makefile: those flavors will command some configuration options, and usually additions to various depends. Note that package naming is mostly automatic: the PKGNAME will have an extension built by appending the specified flavors to its name. So, if

PKGNAME = foo-1.0
FLAVORS = f1 f2 f3

and you build the port with FLAVOR='f3 f1', then FULLPKGNAME=foo-1.0-f1-f3 (FLAVORS is used to reorder specified FLAVORS in a canonical way).

There are sometimes mixed situations, where some packages do depend on the FLAVOR, and some don't. For instance, some ports include a large set of documentation that does not depend on the FLAVOR, and some actual programs that depend on the FLAVOR. In such cases, you can specify the FULLPKGNAME for the documentation subpackage explicitly. E.g., something like this:

CATEGORIES = app
COMMENT-core = foo application
COMMENT-doc = foo documentation
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-$V
FULLPKGNAME-doc = foo-doc-$V
FLAVORS = crypto

MULTI_PACKAGES = -core -doc
WANTLIB-core = c m 

.if ${FLAVOR:L:Mcrypto}
WANTLIB-core += ssl crypto
CONFIGURE_ARGS += --enable-crypto
.endif

As mentioned in the documentation, all package names have the same structure: stem-version-flavor_extension.

By default, packages with the same stem do conflict, and update paths will look at candidates with the same stem. The right package will be the one coming from the exact same PKGPATH, or matching @pkgpath annotation in the packing-list.

Usually, MULTI_PACKAGES should not conflict, so they must have different names (and the infrastructure has no way to build those names). On the other hand, flavors should conflict, and thus have the same name. The flavor information should end at the end of the package name, except for pseudo-flavors, which do not change the way a package is built.

As far as dependencies go, by default, specifying a PKGPATH will just create a stem-* dependency, meaning any package with the right stem will match the dependency. By default, any flavor will match. If only specific flavors are desired, you must include them in your specification, e.g., stem-*-flavor. If some flavors are unwanted, you can remove them from matching packages, e.g., stem-*-!flavor.
$OpenBSD: packaging.html,v 1.2 2009/06/09 10:37:42 espie Exp $