openSUSE:Git Packaging Workflow

Jump to: navigation, search

Important: This document assumes you have also read the OBS to Git page. For command-line contributors, see the git-obs workflow section for streamlined package management. To learn about known issues see the dedicated page.

Prerequisites

OSC Version Requirements

Ensure you're running osc version 1.15.0 or later. Earlier versions may contain bugs in the osc fork functionality. For example, Issue #1767 or Issue #1760.

For Leap users: Add the openSUSE:Tools repo to get the latest osc:

zypper ar -f obs://openSUSE:Tools tools

Recommended tools:

zypper in -y git-lfs, gitea-tea, gitea-tea-bash-completion

Setting up your account at src.opensuse.org

Before running any of the commands below you need to ensure your account is set up with a token and an ssh key.

Gitea token

Log into Gitea and go to https://src.opensuse.org/user/settings/applications and create a new token. The token needs a name (like 'git-obs'), and permissions. Per default all permissions are disabled in the web interface. To make the token usable with 'osc fork', set permissions type 'issues' and 'repository' to 'read-write'.

SSH Key generation and registration

ssh-keygen -t ed25519 -C "your_email@example.com"
ssh-add ~/.ssh/id_ed25519 # or use a different filename as required

You can either add your new key via webui or commandline:

  1. For webui navigate to https://src.opensuse.org/user/settings/keys and click Add Key, so you can search for your key within ~/.ssh/. Ensure that you upload the contents of the file ending in .pub
  2. On commandline run:
git-obs ssh-key add --key-path ~/.ssh/id_ed25519.pub # your filename may vary

Some hardware keys don't work (Nitrokey Start). Workaround is to generate another SSH key for gitea stored in a file or to use HTTP auth.

git-obs login

Finally configure git-obs to login

# Add your login and enter the token created above
git-obs login add opensuse --url https://src.opensuse.org --user [username] --set-as-default
# Use the following command to check that your login has been successful, if it is you will see your username and the url along with other data.
git-obs login list

Command comparison with the old workflow

Package sources are maintained in Git. Make sure to clone with SSH and have the SSH key in your Gitea profile to be able to push changes.

Command comparison
Old command New command Explanation
osc add file git add file Add a file
osc vc osc vc Changelog
osc ar git add -A Add and remove all files to local tracking. WARNING: it would also add directories as we support it now.
osc ci git commit; git push Push changes to remote
osc build osc build Build package
osc branch osc fork Create fork on gitea and OBS build setup
osc mkpac obs-git-init Create or update git repo
osc meta prjconf vi _config Edit the “_config” file in your project git

Example update workflow

The following example shows the workflow for updating a package as a maintainer:

### As a maintainer, you may want to work directly in your repository:
git clone gitea@src.opensuse.org:xmpp/libstrophe.git

### If you are not the maintainer, you will not have write permission later on. So you may want
### to fork instead the repository and setup a build project. You can create this with one step:
osc fork network:messaging:xmpp libstrophe
osc co home:username:branches:network:messaging:xmpp/libstrophe # this is actually doing a git clone

cd libstrophe
### Should use main branch for Git devel projects.
git checkout -b main
vi *.spec
git rm *tar.xz
osc service localrun download_files
git add *tar.xz
osc vc
### Note: by default, this builds against openSUSE:Factory and not project
osc build --alternative-project=network:messaging:xmpp
git checkout main
git add *.spec *.changes *.tar.xz
git commit -m "Update to version ABC"
git push
osc results network:messaging:xmpp/libstrophe
osc sr network:messaging:xmpp/libstrophe openSUSE:Factory

Running osc fork will print helpful output:

osc fork network:messaging:xmpp libstrophe
Using the following Gitea settings:
 * Config path: /home/username/.config/tea/config.yml
 * Login (name of the entry in the config file): obs
 * URL: https://src.opensuse.org
 * User: user1

Forking git repo xmpp/libstrophe ...
 * Fork created: user1/libstrophe

Forking OBS package network:messaging:xmpp/libstrophe ...
 * OBS apiurl: https://api.opensuse.org
 * Fork created: home:user1:branches:network:messaging:xmpp/libstrophe
 * scmsync URL: https://src.opensuse.org/user1/libstrophe#main

Here we can see that it actually creates a fork on Gitea, and a project branch on OBS so you can test build it.

As a maintainer we could also just:

git clone https://src.opensuse.org/xmpp/libstrophe.git
cd libstrophe # and do the changes as above
osc sr network:messaging:xmpp/libstrophe openSUSE:Factory # and still submit by mentioning the OBS project

We could also use tea or git-obs to just create a fork on Gitea without a project branch on OBS for building. Or we could follow the AGit workflow. Which will be explained below.

Example submission workflow

Upon trying to submit packages from an OBS home: project to a Git managed devel project you will receive an error message like the following:

$ osc sr home:mnhauke dnscontrol server:dns
Server returned an error: HTTP Error 403: Forbidden
The target project server:dns is managed in an external SCM: https://src.opensuse.org/dns/_ObsPrj.git#master

Follow these steps instead:

1. Log in to https://src.opensuse.org/ and use the web UI or tea command line to create a blank repository under your user

2. Clone the blank repository to your local machine: git clone gitea@src.opensuse.org:$USER/$package.git

3. Check out the package from your home: project: osc co home:$USER $package

4. Use something like cp or rsync to copy the files from the osc checkout directory into the empty Git repository directory

5. Use the regular Git process to track and push your initial import:

git commit -a -m 'Initial import'
git push $remote main

6. Find the maintainers of the relevant devel project: osc maintainer server:dns

7. Contact the maintainers (for example using a mass email) and ask them to please transfer the repository from your user space into their organization

Contributing to a package as a non-maintainer

Contributors now send pull requests on Gitea instead of submit requests to devel projects on OBS. You can handle pull requests either via the Gitea Web user interface on src.opensuse.org or use git-obs (which is installed automatically via osc).

Contributors can use the workflow to "fork" a repository to their home organization, make changes and send a pull request. This workflow is already familiar to GitHub users. Another way to submit changes to a package where you are not a maintainer is to use the AGit workflow.

To sum it up: If you want to contribute to a package for which you are not set as maintainer, you have several options:

  • Gitea
    • Fork in Gitea
    • Gitea Web UI
    • Gitea command-line tool tea
    • git-obs
  • Make your changes and git push
  • Create a pull request
  • AGit workflow
    • Clone the package
    • Push to a special branch, and a PR will be created automatically

When a user creates a pull request to the package repository, the autogits-devel bot automatically creates a second pull request to the project repository _ObsPrj. In the _ObsPrj pull request, the autogits_obs_staging_bot bot provides a link to an OBS build project where the packages are built. The bot also posts the build results as a comment on the original PR. If the title of the pull request is prefixed with WIP:, the automated workflow is not triggered.

The maintainer can then review the submitted changes. After the maintainer approves the PR, the bot automatically merges it. Pull requests from maintainers are automatically merged if the build is successful.

Removing a package

A simple project without submodules can be simply removed by removing a directory.

git rm

When you need to remove a package with a submodule, then first pool. Then delete the submodule in question, and make a PR with that submodule removal.

Git-obs workflow

Initial Setup

Log into Gitea and go to https://src.opensuse.org/user/settings/applications and create a new token. The token needs some name (like 'git-obs'), and permissions. Per default all permissions are disable in the web interface. To make the token usable with 'osc fork', set permissions type 'issues' and 'repository' to `read-write`.

Configure git-obs with your credentials:

# Configure login for OBS
git-obs login add --url=https://src.opensuse.org --user=... obs --set-as-default
# Configure login for IBS
git-obs login add --url=https://src.suse.de --user=... ibs
# Upload an existing ssh key to Gitea (if the used gitea token has read-write permissions for 'user')
git-obs -G obs ssh-key add --key-path=~/.ssh/id_rsa.pub
# List ssh keys associated with Gitea account (if the used gitea token has at least read permissions for 'user')
git-obs -G obs ssh-key list

Submit a pull request:

# Fork a package:
git-obs repo fork <develprj>/<package>
# Clone the forked package:
git-obs repo clone <owner>/<package> [--no-ssh-strict-host-key-checking]
# Make change
cd <package>
git remote -v
git switch <branch>
git branch <new-branch>
# Alternatively: git branch for/<target-branch>/<feature-name>
# Make changes
git commit
git push
# Create a pull request
git-obs pr create [--title=...] [--description=...] [--target-branch=...]

Review a pull request:

git-obs pr review [<owner>/<repo>#<number> ...]

AGit workflow

The AGit workflow doesn't work with LFS for now. This is supposed to be fixed in the future. It is recommended not to use this workflow if you need LFS.

This has the advantage that you don't need to fork a package before contributing. This also means we don't waste space in case you forget to clean it up after your contribution. See AGit documentation for details.

git clone https://src.opensuse.org/lxde/parcellite
vi *.spec *.changes
git commit -m "Update"
git push origin main:refs/for/main/my_update -o title="Update to 123"

Creating a new package in Gitea

To create a new package repository, visit https://src.opensuse.org/repo/create

When creating the repository, select new pool/new_package from the Template dropdown, if it's not preselected already.

Git Content should be also selected from the Template Items, to preconfigure the repository with the default metadata from https://src.opensuse.org/pool/new_package

including files such as:

  • .gitignore
  • .gitattributes

These files are required for proper handling of Git LFS, especially when working with source archives.

Please make sure you have git-lfs installed.

Proper way to create a completely new package will be defined after delivery of https://github.com/openSUSE/openSUSE-git/issues/146

For now, please use these steps https://en.opensuse.org/openSUSE:Packaging_for_Leap#Adding_a_new_package_to_Leap_from_Factory

Reviewing build results

Build results can be viewed on OBS:

osc results network:messaging:xmpp/libstrophe

Additionally, the autogits_obs_staging_bot bot will post the build results as a comment on the created pull request.

Or build results can be set up by one of these methods:

  1. You have to create a pull request in gitea, then the autogits bot will create a project in OBS for build results. If you want faster feedback without having to create a PR already, the only way is to `osc linkpac` the package in OBS and make your changes there, and then create the PR later.
  2. Or you can create an empty project with some build repository and scmsync the package in there. This is what `osc fork` does.
osc meta prj -e home:....:testbuild -- add <repository>
osc meta pkg -e home:.....:testbuild package_name  -- add <scmsync>https://src.opensuse.org/user/package_name#main</scmsync> 

The documented path would be:

osc fork devel:languages:erlang erlang27

Submitting to Factory

Submissions to openSUSE:Factory are not yet managed through Git/Gitea. This process is still handled by osc. Therefore, after your changes are committed to the Git project or a PR is accepted, you must create a submit request to Factory. For example, to submit the libstrophe package, execute the following command:

osc sr network:messaging:xmpp/libstrophe openSUSE:Factory

Finding a package's devel project in Factory

Currently, a package in openSUSE:Factory doesn't show where the devel project for that package is. This will be improved in the future. The interim solution is to locate the development project by searching for the package name in pkgs/_meta/devel_packages.

Visualizing build results

Use https://br.opensuse.org/status/PRJ/PKG/<repo>/<arch> where repo and arch are optional.

For example:

This can be added to your packages README.md:

Devel project build status:

![devel:Factory:git-workflow](https://br.opensuse.org/status/devel:Factory:git-workflow/autogits)

Tips and best practices

Commit messages and changelogs

When you update a package, you can reuse the text you wrote in the .changes file in the git commit message. To do so, follow the standard format described here, which looks like this:

-------------------------------------------------------------------
Sat Jan 11 16:37:07 UTC 2025 - Your Name <your@emailcom>

- Update to new upstream release x.y.z:
  * bling and changes from upstream for that version
  * just the relevant parts, no info about other OS
  * and keep it as short as possible

-------------------------------------------------------------------
Sat Jan 10 16:37:07 UTC 2025 - Your Name <your@emailcom>

- Initial package of packagename 1.0 for openSUSE:

You have various ways to reuse your changelog.

Changelog alias

Use the following line:

git diff -U0 --cached *.changes | grep '^[+]' | grep -Ev '^(--- a/|\+\+\+ b/)' | tail -n +4 | cut -c4- | git commit -F -

Add it as an alias to your shell:

vi *.zshrc

alias gitum="git diff -U0 --cached *.changes | grep '^[+]' | grep -Ev '^(--- a/|\+\+\+ b/)' | tail -n +4 | cut -c4- | git commit -F -"

After you add a file to be contained in the next commit with git add *.spec *.changes *.tar.gz, you can now run gitum to take the commit message from the staged changes file, remove the boilerplate, and commit.

insteadOf

If tired of writing src.opensuse.org, you can use insteadOf in your ~/.gitconfig file. Add this to the config:

[url "gitea@src.opensuse.org:"]
  insteadOf = soo:

Now you can simply use git clone soo:user/repo.

sha256 repos with Gitea client

Most of the time you actually don’t want to create a new repository from scratch, but use prepared template, for that you could run

tea repo ct --name appropriateName --template pool/new_package

(yes, the name of the template is literally pool/new_package).

If you actually really need to make a fresh blank repository, and because we require SHA256 repositories, you can run (from version 61.2.0 onwards):

tea repo create --name appropriateName --object-format sha256

Build a bunch of packages together as a test

  1. Get needed repositories into your home repository for example: https://src.opensuse.org/mcepl
  2. In the home project setup an empty package with manual setup <smsync> in meta for example: https://build.opensuse.org/package/show/home:mcepl
  3. Do for all needed packages...

This need improvement, tracked as [1]

Project Setup

It is possible to setup a git project for group of packages like PythonX.Y. Then the project can have it's own workflow bot running with their own workflow definition. The request can be opened here: https://progress.opensuse.org/projects/opensuse-admin/issues/new

Maintainer of the project needs to provide definition for:

  1. Setup _ObsPrj
  2. Workflow_bot running (bots must run in BuildOPS machines)
  3. Merges can be also defined: automatic, wait for peer review or manual merge.

Example from existing project:

  1. main file https://src.opensuse.org/python-pytest/_ObsPrj/
  2. https://src.opensuse.org/python-pytest/.profile
  3. https://src.opensuse.org/python-pytest/_ObsPrj/src/branch/main/_config
  4. Currently, it's used a file for defining maintainers of the project https://src.opensuse.org/python-pytest/_ObsPrj/src/branch/main/_maintainership.json
  5. https://src.opensuse.org/python-pytest/_ObsPrj/src/branch/main/_manifest
  6. https://src.opensuse.org/python-pytest/_ObsPrj/src/branch/main/workflow.config
  7. https://src.opensuse.org/python-pytest/_ObsPrj/src/branch/main/staging.config

Permission for Maintainers

Go to the repo setting in Gitea and check "Allow edits from maintainers", then PR authors only need to do normal git obs pr create, then the created PR will inherit the repo's "Allow edits from maintainers" setting.

It will be available in gitea 1.26.

Missing Parent Project

The osc service dr fails with the git workflow. It reports:

Git SCM package working copy doesn't have the following metadata set: apiurl, project
 - The package has no parent project checkout

Solution is to set those variables: either in the projects _manifest (e.g., https://src.opensuse.org/python-pytest/_ObsPrj/src/branch/main/_manifest) or via git obs meta. It's better to use git-obs meta set --apiurl=... --project=.... The settings apply per branch, so you can have different settings for different branches.

Can't create PR on the pool branch

People complained they are trying to push while a branch has a gray hook in gitea webui. Well, it’s not possible to push to a pool. Pool doesn’t have permission for everyone, so users see it gray. The proper way is to fork from the pool, make your own branch, do the work there, create PR for the correct branch. Careful, when you are doing PR, gitea will propose some branch for merging, but not necessary the one you wish to use.

Some maintainers are creating PRs using git-obs, which doesn't have the issue.

Tool reference

A quick overview of tools that are used in this guide: