Single-spec Guidelines

Status: Provisional

The contents of this page should be integrated to the main Packaging Python guidelines. This is more work than putting down the singlespec guidelines separately, so for now (2017-03-07) I am doing the latter.

What is single-spec?

'Single-spec' (or 'Singlespec') is an approach for packaging both python2 and python3 variants of a Python module from a single source spec file. See also Packaging Python guidelines.

Compatibility shims

If your packages are targeted for anything other than Factory, the spec file must include two things:

Redefinition of %python_module macro.

%{?!python_module:%define python_module() python-%{**} python3-%{**}}

Build requirement on python-rpm-macros:
```
BuildRequires:  python-rpm-macros
```

BuildRequires

BuildRequires are not conditional and apply for the whole spec file. There is no such thing as "if I build for Python 2, I require package foo"; you simply require package foo always.

If you need a package for all available flavors of python, use the %python_module macro. Convert this:

BuildRequires: python-setuptools
BuildRequires: python-py >= 1.4

to this:

BuildRequires: %{python_module setuptools}
BuildRequires: %{python_module py >= 1.4}

Note that the version requirement goes inside the %python_module call.

This is not intended for Requires tag. The %python_module macro is only for BuildRequires

If you only need the package for one flavor of python, simply don't use the macro.

BuildRequires: python2-enum34
BuildRequires: python3-astroid

If you need to BuildRequire "python", you can instead use %pythons:

BuildRequires: %{pythons}

Requires, Provides and similar

In many cases, you don't need to do anything. The single-spec rewriter will convert your Requires to match the generated package.

Warning: In particular, do not use %python_module for Requires.

Package names in tags Requires (as well as "Requires(pre)" and all the others), Provides, Recommends, Suggests, Obsoletes, Conflicts, Supplements, and Enhances are automatically converted.
The converter takes into account the packageand expression, and %requires_ge and %requires_eq macros. However, support for these must be coded explicitly. If you find an expression that is not converted, please either post it on the packaging ML or file a bug against python-rpm-macros.

If the requirement name starts with "python-", or with the same name as your package (so "python3-bar" in package "python3-foo"), the python name is changed to match that of the generated package.
This also works for "python" itself.
Packaged listed in packageand expression are also converted.

Requirements specific for one python flavor

You can specify that some packages should only be included for some python flavors, by wrapping them in conditionals using the %python_flavor variable

Requires: python-idna
%if "%{python_flavor}" == "python2"
Requires: python2-enum34
%endif

As a shortcut, for every flavor, there is a %ifpython macro: %ifpython2, %ifpython3 or %ifpypy3.

%ifpython2
Requires: python2-enum34
%endif

Note that the shortcuts must not be nested in other conditionals, otherwise you can get "%endif without %if" error message. If you need to nest conditionals, use the %python_flavor conditional.

Warning: This does not work for %prep, %build, %install, %check.
The %ifpython sections are copied (or not) to subpackage definitions. Subpackages have their own requirements, file lists and %pre/%post/etc. scriptlets. The scripts from %prep and others is shared. %ifpython will do the wrong thing.

Obsoleting and Providing old symbols

The following sequence:

Obsoletes: python-distribute < %{version}
Provides:  python-distribute = %{version}

would mean that your python2-package will obsolete/provide python2-distribute and your python3-package will obsolete/provide python3-distribute and your pypy3 package will obsolete/provide pypy3-distribute.
Often, this is not what you want.

First of all, in many cases, this is only applicable for Python 2, so the sequence should be wrapped in %ifpython2 conditional.

Second, you will note that neither package obsoletes/provides python-distribute.
Here is how you do that:

%define oldpython python
%ifpython2
Obsoletes: %{oldpython}-distribute < %{version}
Provides:  %{oldpython}-distribute = %{version}
%endif

`%python_module` in Provides

If you are creating a subpackage that will be common for all flavors (could be a -doc subpackage), sometimes you need to provide a symbol for all flavors.

E.g., your package python-foo-doc should also provide python2-foo-doc, python3-foo-doc etc.

In such case, you use the %python_module macro:

%package -n python-foo-doc
Provides: %{python_module foo-doc = %{version}}

This is also the one case where you could use %python_module in Requires.

(See below on declaring packages with -n to prevent autogeneration.)

`%python_subpackages`

Easy enough: place the %python_subpackages macro on a separate line at the end of the spec preamble (the part that ends where your package's %description begins).

Provides: pylint
BuildRoot: %{tmproot}/blabla
BuildArch: noarch

%python_subpackages

%description

This macro emits all the subpackage descriptions, %files and scriptlet sections for the autogenerated parts.

Subpackage declarations

Subpackages are converted automatically. If you have a subpackage %package foo, singlespec will create python3-yourpackage-foo and all the rest from it.

If you want to prevent this, use %package -n %{name}-foo. Or the full name, %package -n python-yourpackage-foo. This will ensure that the subpackage will be skipped.

This also means that packages named %package -n yourpackage-python will not be processed. There will be a mechanism for these in the future.

Common documentation packages

It is very common that the shipped documentation and probably included examples are quire big and should be in a separate sub-package. The package could have - for example - these sections:

%package -n %{name}-docs
Summary:        Documentation files for %name
Group:          Documentation/Other

%description -n %{name}-docs
HTML Documentation and examples for %name.

%files -n %{name}-docs
%doc examples docs/_build/html/

Build macros

Instead of python setup.py build, use %python_build.
Instead of python setup.py install, use %python_install.

These macros already contain the usual options (--root, --prefix), so in the typical case, you don't need to supply any options. If you have something specific, you can add it: %python_build --enable-specific-feature.

If you set environment variables, export them first. Instead of:

CFLAGS="-fwrapv" python setup.py build

use:

export CFLAGS="-fwrapv"
%python_build

For any other commands, you can use %python_exec. That is also a good way to run pythonic executables. For example, convert this

%check
python setup.py test
nosetests-%{py_ver}
PYTHONPATH=%{buildroot}%{python_sitelib} py.test

to this:

%check
%python_exec setup.py test
%python_exec %{_bindir}/nosetests
%python_expand PYTHONPATH=%{buildroot}%{$python_sitelib} py.test-%{$python_version}

`%python_expand`

For anything more complicated than executing all the interpreters, use the %python_expand macro. This will repeatedly expand the %{$python}, %{$python_sitelib} etc. strings to the currently used flavor.

%python_expand rm -r %{buildroot}%{$python_sitelib}/file.txt

results in (apart from some build dir manipulations):

rm -r %{buildroot}%{python2_sitelib}/file.txt
rm -r %{buildroot}%{python3_sitelib}/file.txt
rm -r %{buildroot}%{pypy3_sitelib}/file.txt

IMPORTANT: you can use %python_expand to replace macro definitions, but make sure you use, e.g., %{$python_sitelib}, with the $ sign.
If you use plain %{python_sitelib}, the macro will be expanded before %python_expand can modify it.

A common use for %python_expand is with fdupes, as in:

%python_expand %fdupes %{buildroot}%{$python_sitelib}/mymodule

You can use multiline %python_expand if you enclose the lines in {}. The only technical limitation is that the first line must not be empty (but can be a comment starting with #):

%{python_expand # this will expand the following section
%$python_install
mv %{buildroot}%{_bindir}/exename %{buildroot}%{_bindir}/exename-%{$python_bin_suffix}
}

This is eg useful when running tests and some pre-step is needed:

%check
%{python_expand rm -rf .testrepository
$python setup.py test                                                                                                                                                                                                                                                                     
}

Naming flavor-specific files

Executables specific to a particular flavor should use %python_bin_suffix instead of %python_version for names. This expands to %python_version for CPython, pp%{python_version} for PyPy. If we support Jython, it will get a specific bin_suffix too.

Build directories specific to a particular flavor should use %python_prefix. This expands to the flavor name.

Filelists

If your package is called python-something (that is, the name prefix is "python" and not a specific flavor), you must mark your %files sections with %{python_files} macro.
Convert this:

%files
%{python_sitelib}/foo

%files plugins
%{python_sitelib}/foo-plugins

to this:

%files %{python_files}
%{python_sitelib}/foo

%files %{python_files plugins}
%{python_sitelib}/foo-plugins

You can use %ifpython2 and similar to conditionally include some files only in some flavors. In addition, you can use shorthands:

%files %{python_files}
%{python_sitelib}/foo
%{python_sitelib}/foorun.py*
%python3_only %{_bindir}/foorun
%pycache_only %{python_sitelib}/__pycache__

Use %pycache_only or %ifpycache to mark __pycache__ directories.

Executables

For new packages that carry executables, it is preferred that these are marked %python3_only. That ensures that the executables are only installed for the preferred python flavor, which is Python 3.

If your package functionality depends on version of Python (e.g., test harnesses, source code processors, installers, etc.), you should package executables for all flavors. This is easy if the package itself installs versioned executables. If not, use the %python_clone macro.

%python_install
%python_clone %{buildroot}%{_bindir}/executable

This macro will copy the executable as executable-%python_bin_suffix for all flavors, and change the shebang line to the appropriate interpreter.

%python_clone works for all kinds of files. If it recognizes a man page extension, it will correctly rename manpage.1 to manpage-%python_bin_suffix.1.

When using %python_clone, you need to list both the original executable *and* the copied executables in your filelist:

%{_bindir}/executable-%{python_bin_suffix}
%python3_only %{_bindir}/executable

If changing the shebang line is not appropriate or sufficient, you will need to catch the executable after every installation, like this:

%{python_expand %$python_install
mv %{buildroot}%{_bindir}/executable %{buildroot}%{_bindir}/executable-%{$python_bin_suffix}
}

Python 2 stacks

The recommendation for preferring Python 3 is intended for new code.

Existing packages that are parts of Python 2 stacks (such as the current (2017-03) OpenStack Cloud) need executables for Python 2 as well. For these, it is preferable to use the alternatives approach outlined below.

update-alternatives

Sometimes it is useful to let the user switch the unversioned executable name to one version of the package or another.

Also, if you can't be sure that the user has a Python 3 stack, and want to provide unversioned executable names regardless, update-alternatives is the way to go.

update-alternatives allows you to switch other things (usually man pages) along with executables.

As a prerequisite, you need to have version-specific file names available -- that is, for every file you provide, file-%python_bin_suffix should exist for all flavors. (for manpage.1, the appropriate name is manpage-%python_bin_suffix.1)

First, you need to set up the alternatives in %install section.

If you're using %python_clone to create the executable, simply pass -a option: %python_clone -a %{buildroot}%{_bindir}/executable
If not, and the file in question is %{_bindir}/something, call %prepare_alternative something
If the file is not in %{_bindir}, you have to specify the path: %prepare_alternative -t /path/to/file file. The path is without %buildroot

Second, create the appropriate %post and %postun sections. Make sure that you have the proper requirements:

Requires(post):   update-alternatives
Requires(postun):  update-alternatives

Then use %python_install_alternative and %python_uninstall_alternative respectively.

%post
%python_install_alternative exename

%postun
%python_uninstall_alternative exename

Third, mention the alternative, unversioned, in file list

%python_alternative %{_bindir}/exename

You can examine a full spec file at [1].

Grouped alternatives

The update-alternatives system allows for multiple files in the same group, to be switched together. This is useful if you want to install an executable along with its manpage, or multiple executables belonging to the same function group.

To make use of this, simply specify the files as multiple arguments to %python_install_alternative:

%{python_install_alternative pylint pylint.1 epylint epylint.1}

The macro can recognize manpage names and handle them correctly, but the first arguments needs always to the an executable. Or you can specify a full path to the file in question.

The first argument to %python_install_alternative is the group name. This is the only argument for %python_uninstall_alternative; you uninstall the whole group by the one name.

Packages for single Python version

Packages that only exist for Python 2 should be left as is. New packages should be called python2-foo. Old packages should be left as python-foo, but you should add Provides: python2-foo. You should also make sure that all BuildRequires and Requires are for "python2" and "python2-foo".

Packages that only exist for Python 3 can be left as is, or they can be converted to singlespec, with %define skip_python2 1. This is to ensure that they can build for PyPy or other flavors in the future. The definition should be made at the top of the spec-file.

Backport packages should not be converted to singlespec. They should never even be built for Python 3 and kept for Python 2 only.

These include: python-enum34, python-singledispatch, python-backports.*, python-funcsigs and possibly others.

The known exception is python-backports_abc which is relevant even in Python 3.4 and 3.5.

Conditionals on Python versions

If you need to differentiate between python versions you can use the following macros:

Macros	Example
`%python2_version_nodots`	27
`%python3_version_nodots`	34

For example:

%if %python3_version_nodots > 34
...
%endif

Submitting to Factory

For Factory submissions, first make sure that all dependencies of your package are already converted to singlespec and submitted to Factory.

Also make sure that you're submitting the most recent version of the package. Don't forget to check both d:l:python and d:l:python3.

If possible, ensure that you have a %check section that runs tests for all flavors of Python.

If the package is already present in Factory, check if its Python 3 variant also exists. If it does, send a delete request for it along with your submission.

If your package is backporting new features of Python 3.4 or newer to Python 3.3 or older, including Python 2, do not convert it to singlespec and only submit a Python 2 version of the package.

Common Gotchas

Things to look out for (which can also get your submission declined):

If the %python_module redefinition is present, check that it has "python-%{**}" and not "python-%1"
use %python_module for BuildRequires
do NOT use %python_module for Requires
make sure %python_subpackages macro is present
%files sections should be marked as "%files %{python_files}"
in %python_expand, make sure you use %$python_macros instead of %python_macros
use %python_bin_suffix instead of %python_version for distinguishing executables or build directories

All Macros and Examples

The full documentation for all macros defined for singlespec can be found at the GitHub page for python-rpm-macros package.

Here is an example of a simple spec file using singlespec: [2]

Here is an example using update-alternatives: [3]

Navigation

Help

Tools

Sponsors

openSUSE:Packaging Python Singlespec

Contents

Single-spec Guidelines

What is single-spec?

Compatibility shims

BuildRequires

Requires, Provides and similar

Requirements specific for one python flavor

Obsoleting and Providing old symbols

`%python_module` in Provides

`%python_subpackages`

Subpackage declarations

Common documentation packages

Build macros

`%python_expand`

Naming flavor-specific files

Filelists

Executables

Python 2 stacks

update-alternatives

Grouped alternatives

Packages for single Python version

Conditionals on Python versions

Submitting to Factory

Common Gotchas

All Macros and Examples

Navigation

Help

Tools

Sponsors

openSUSE:Packaging Python Singlespec

Contents

Single-spec Guidelines

What is single-spec?

Compatibility shims

BuildRequires

Requires, Provides and similar

Requirements specific for one python flavor

Obsoleting and Providing old symbols

%python_module in Provides

%python_subpackages

Subpackage declarations

Common documentation packages

Build macros

%python_expand

Naming flavor-specific files

Filelists

Executables

Python 2 stacks

update-alternatives

Grouped alternatives

Packages for single Python version

Conditionals on Python versions

Submitting to Factory

Common Gotchas

All Macros and Examples

`%python_module` in Provides

`%python_subpackages`

`%python_expand`