Archive:Build Service Git Backend

Jump to: navigation, search



GSoC 2009 Project

student: Peter Libic

mentors: Brandon Philips, Pavol Rusnak, Andreas Gruenbacher


Proposal

My project proposal is to create a prototype git backend for the OpenSUSE Build Service. Currently OBS uses a custom MD5 based relational database called SRCREP for storing project revisions. It works well for OBS but using a git backend could improve entire service.

Git is one of the best revision control systems currently available. Present OBS makes extensive pressure on the backend when using basic revision control commands like diff or log. Git can move these actions to the client what may lower the server load.

Git can easily import/export data from multiple repositories what could make the process of import and export of projects from different Service instances.

Since git is distributed system it is possible to use all its features also when the network connection is not available. The changes can be integrated to the main repository after the network access is available again.

The source files could be accessed by anonymous users more easily. Currently, OSC is needed for such access which is not so standard and well-known as git is.

In the present system the changes can be tracked only if the full tree is checked out. For git there exists frontends that make browsing through revision and changes in the repository simple and fast (through web, for example).

This project would build a prototype that would replace the usage of SRCREP with git on the backend of OBS. This prototype could then be used to see what else could be done with a git based OBS and an evaluation could be made from there.

The project will create a git repository for every package. For example, packages named "konversation" will have repositories KDE/KDE4/STABLE/konversation.git, KDE/KDE4/STABLE/konversation.git and openSUSE/Factory/konversation.git.

Current state

  • OBS uses a custom MD5 based relational database called SRCREP for storing project revisions.
  • Command like log and diff are remote, which makes them very slow for remote users (i.e., "the community").
  • Source links are less powerful than they could be, as well as harder to understand than necessary.
  • It is hard to understand / reproduce how a specific revision of a source link (i.e., a merge) was created.

Goals

  • Allow packages to use git as their repository format on the backend.
  • Convert the data in SRCREP into GIT (Note: this is already supported by bsgit.)

Advantages / Use cases

  • Operations like diff and log/history are offline; this lowers the server load. (Osc only supports diff against the latest known version and the status command locally; operations like log are always remote.)
  • Arbitrary versions of a package can be compared efficiently (in log as well as in diff form) even across branches or repositories.
  • Commits can be done locally and pushed to the server "in one go". This makes it much more practical to commit one logical change at a time, the results are easier to review.
  • Commits can be exchanged between repositores very easily, including proper tracking so that merging will work.
  • Simpler submit requests.
  • Very powerful support for merging (in general, but also of submit requests).
  • The native git network protocol is much more efficient than the build service ABI.
  • Multiple desktop GUI clients are available for browsing git repositories.
  • Multiple web interfaces for browsing repositories online are readily available.
  • Source links turn into branches from which to merge; applying patches in source links turns into (three-way) merging.
  • The repository format is open rather than "homegrown"; the data won't be locked into the build service.
  • Support for packed (i.e., compressed) repositories, which are (somewhat) smaller.
  • Merge a patch from a contributor, including tracking who the patch is from (i.e., author vs. committer).
  • Changes (even sets of changes) can be sent via an e-mail. Such emails can trivially be imported.
  • Merge commits from another repository (i.e., pull request).
  • Track which changes came from where (i.e., cvs annotate / git blame).
  • Browse a repository (on the console, GUI, on the web).

Meetings

  • 06 May 2009 - Meeting #1 - IRC #obs-git (P. Libic, P. Rusnak, A. Gruenbacher, M. Mohring)
  • 13 May 2009 - Meeting #2 - Prague SUSE office (P. Libic, P. Rusnak, A. Gruenbacher)
  • 13 Jul 2009 - Meeting #3 - IRC #obs-git (P. Libic, A. Gruenbacher, P. Rusnak)
  • 27 Aug 2009 - Meeting #4 - IRC #obs-git (P. Libic, A. Gruenbacher)

Presentations

Source Code Repository

Related Project

Interesting (and unsorted) links


Retrieved from "https://en.opensuse.org/index.php?title=Archive:Build_Service_Git_Backend&oldid=171688"