openSUSE:ALP/Workgroups/Git-Packaging-Workflow/WorkPackages/Submission-Workflows-Protoype
Overall goals of the Git-Workflow Prototype
- Prove there is a improvement for day-to-day Packagers life by avoiding Open Build Service specific workflows (submitrequests, maintenancerequests, osc, the webui etc) and switch to a git based approach with a git webui/PR workflow
- Provide an easier contribution experience for drive-by contributors
- Improve maintenance of a package in many distributions by switching to a package source repository centric approach, which means git branches are the primary element to manage many distribution versions of a package.
- Keep the entry barrier and learning curve low or lower it over today's level. git is a a swiss army knife toolkit and can do many complex things in very complicated ways. however the majority of the user base is likely going to be git basic knowledge users only. We want to avoid a large ramp up before the first contribution can happen. rpm is complicated enough by itself.
Decisions to take to have a successful git workflow
- Where to store bugowner and maintainership information
- how to enforce proper access control
- how to notify maintainers of packages on actions to take (issues filed, build failures, PRs to review..)
- how to enable local development (e.g. the domain of osc and specifically osc build) with a pure git workflow
- how to replace or convert obs source services
- how to replace or convert remote bots that participate in the distribution development (legal, factory-auto)
- how to replace or convert the osc staging / staging project setup for Tumbleweed and other distributions
- how to maintain security multiple branches of a package including those that need to be handled in the certified "internal" build service instance
Phasing of a git roll out
Package sources history as git repository
- Goal: Have the existing openSUSE Factory source history converted into a package dist git repository
- Status: Completed, available at https://src.opensuse.org/pool (and continuously updated at /rpm)
Be able to maintain a package in Factory
- Goal: maintain the package distribution in git (+LFS) on src.opensuse.org, be able to receive contributions and forward them to factory
- Status: Completed, via a src-o-org-forwarder bot that converts PRs to the package gits into a "legacy SR" for openSUSE Factory integration
Workflow of a Package modification
Concept picture
Phase 1
In this first phase, the package maintainer would be receiving a contribution to an existing package in the form of a Pull request. For the prototype an on-premises hosted Gitea instance is being used because it was considered a more certification-ready approach than going with a Software-as-a-Service approach like GitHub, although in the design nothing is depending on Gitea-specific features so the implementation can be replaced by another Git Hoster/Frontend later.
This Pull request would undergo the following steps:
- The package maintainer is getting notified about the existence of a new PR
- The package specific tests including a build test are triggered an the test results are reported alongside the PR
- The package maintainer is reviewing the change and submits a vote (but does not merge it)
- Once maintainer voted, the pull request gets picked up by the staging bot and will undergo a staging-integration test and the additional reviews needed for primary-distribution integration (source-review, legal checks, openqa)
- The release engineer validates that all conditions have been met and merges the PR into the factory branch
This workflow assumes that all the steps needed to integrate a change into the "factory" branch of the package are attached to one PR, and the package maintainer as well as the contributor can track their status from the PR alone without having to visit any other system or webpage to get an overview on the status.
For packages that are developed in a coordindated team/subset, it is possible that changes are tested in a "devel" branch, which would be outside the scope of this description.
In this Phase 1, the changes are handled package-by-package and integration tested against a "factory" (following the "Factory first" development workflow paradigm). Once a change has entered the factory branch, derived distributions (like in this picture the ALP distribution) can decide to cherry-pick the change, potentially at a lower integration threshold as the particular change has been already integration tested.
In this Phase 1, the release engineer would continue to operate using the existing Build Service Staging mechanics and project maintenance. This also means that the existing issues with e.g. prjconfig remain unresolved. These are intended to be addressed in the later stage when the distribution maintenance moves into a git project as well.
Phase 2
In this phase the project is also maintained in the git repository, and the distribution release engineer is also operating purely outside the OBS UI/mechanics in Pull requests. This has a larger impact on the current project handling and build resources needed so it will require further thought on how to implement that in a scalable fashion. One could imagine that this is similar to running "pbuild" as a CI / Action on a PR and collecting the test results (+ results of further testing from means like e.g. openQA) and adding that as a buildresult to the PR of the distribution git.
Consequently every PR would be build in isolation, which is similar to how currently there are multiple "Letter-Stagings" in the distribution development in the Open Build Service. However, each change to a PR would likely cause a full rebuild as previous build results cannot easily be reused.
Concept Picture
In this model, the build service project would be entirely defined by a git repository and every distribution increment would consider of one or more package updates that are being tested together in a single PR.