openSUSE:Mirror infrastructure
What does a server need to become a public mirror?
- It needs at least 40GB to 60GB of disk space, depending on what is being mirrored.
- It needs a lot of bandwidth bandwidth; The actual amount is hard to predict but e.g. in Germany 1 TB per month should be considered the minimum and would be quite easily reached. The mirror should ideally be able to afford 2 TB per month to avoid bandwidth caps. 10mbit is minimum, 100mbit is better. The situations and requirements will vary from location to location and may different than expected. In general, the more content is being mirrored, the more traffic is attracted, on the other hand we can control the number of redirects quite well. The presence of ISO images is the largest determinant for caused traffic.
- The hardware or OS doesn't really matter.
The current sizes of the rsync modules listed on this page but note that it is entirely possible to mirror only parts of a module.
Rsync servers
Access for the public
rsync.opensuse.org
This rsync server is open to anyone. It offers public access via rsync protocol to the download.opensuse.org content. Access is usually limited to 50 concurrent connections, so you might not always be able to access it. Some of the mirrors listed here might also offer rsync services.
stage-main-repos.opensuse.org aka stage3.opensuse.org
This rsync has the same rsyncd config as the main stage.opensuse.org, but it has a 5-minute delay.
Note, that the "ports" and "repositories" dirs on stage-main-repos.o.o have only a small collection (150GB each) of repos that are more likely to be requested by users. Use stage.o.o if you want everything.
provo-mirror.opensuse.org
This rsync server is available for the Americas in case you notice network bottlenecks with other regions.
Access for registered mirrors
stage.opensuse.org
Registered mirrors get access to stage.opensuse.org. This server provides the updated content of download.opensuse.org before the official release and has a higher transfer rate than the public servers. You may want to register for access at mailto:admin@opensuse.org, if your mirror has at least a 100MBit connection, and if the conditions outlined in the following paragraph are met.
Conditions for access to stage.opensuse.org
A few words about "staged content" up front. Staged content is content that is not meant to be public yet -- but which we still would like to spread to mirrors already, so that at the time of the public release it is already mirrored, and thereby accessible for many people. So how can that be achieved? We set the permissions of the directory to be protected to 'rwxr-x---' (0750). The directory is then served as part of the tree which is hosted on the stage rsync server. When mirrors sync from it, they will replicate those permissions. And when the time to release has come, the directory permissions are changed to rwxr-xr-x (0755), and when the mirrors sync the next time, they catch up with it and the directory becomes accessible on their HTTP servers as well. This process of release by permission change is sometimes called "bit flip release".
There are some caveats with that, which you (as mirror admin) need to observe:
- run rsync with -p (--perms), so that the permissions are reproduced on the target machine.
- if you run a public rsync server: make sure that your rsync daemon runs under a different user id than the script which pulls the content. Otherwise you might be publicly serving the staged content. You can achieve this, for instance, by setting uid = nobody and gid = nogroup in the respective rsync module.
- run your mirror scripts under a user id different from the one which your HTTP server runs as. An identical user id would make all files readable for the the HTTP server. The same effect happens if you run the server as root.
- never run your web/rsync server as root. A somehow recurrent misconfiguration is, if lighttpd is used, that it is run as root, because the configuration which causes it to run as a different user/group has been forgotten.
You should be subscribed to the mirror mailing list (see bottom of this page), so we can keep you up to date with regard to ongoing release activities. We will inform you of the release schedule, and exact timing of public release -- and you can actively support us in fact.
Registering your mirror
In order to redirect clients to your mirror, we need the following:
- Email address and the name of the administrator for contact. As mirrors tend to run for years and people move around, we strongly recommend using a group address, e.g. mirror@ or ftpadmin@ or similar.
- a name and URL of the operator or sponsor of the mirror, for display in the mirror list
- someone from your team is subscribed to mirror@lists.opensuse.org - see Staying informed section below.
- HTTP URL on your mirror (e.g. http://yourmirror.example.com/pub/opensuse/)
- read-only rsync access for our scanner -- for scanning which we perform to keep our download redirector database up-to-date. It is done from any IP inside 195.135.220.0/22 and 2001:067c:2178::/48. (Test via rsync -v yourmirror.example.com::)
- Limits for your server. We can either limit the requests to clients only from your region, your country, your ASN or your prefix. It's also possible to limit the access to your mirror just to clients with IPv6. If your server can not deliver big files (normally above 2GB), this can also be added as limit. And - last but not least - the amount of requests routed to your server is calculated by a score (which is normally 100) - this can also be changed if you want not so much traffic. Please request any of these limitations in your Email.
- DNS entries (with reverse mappings) of your mirror to be added to our staging server. Our staging server limits access to the current modules just to a small list of allowed hosts, so your server will not get hit by limitations that are in place for the public rsync.opensuse.org server.
Please send an Email with the following information to admin@opensuse.org:
- Admin Name:
- Admin Email:
- Subscribed to mirror@lists.opensuse.org: YES/NO
- Sponsor Name:
- Sponsor URL:
- RSYNC URL:
- RSYNC allowed only for scans: YES/NO
- HTTP URL:
- Limitations:
- IPs and DNS for the whitelist on stage.opensuse.org:
If you provide this data, we will add your mirror to our mirror database. The mirror database is used by our download server to actively redirect clients to your server. We attempt to distribute requests on a geographical basis per client IP address. The amount of redirects issued also depends on a score which we will determine together with you, in order to match your capacities.
Furthermore, we actively monitor content on mirrors, so that we redirect only to files which actually exists on them. rsync is the most efficient way to do this; scanning through 300.000 files might take only a few minutes with it. Otherwise we can fall back to HTTP, but it crawls. Thus, please do consider adding an rsync module for opensuse content, which allows for much faster scanning of your server.
Staying informed
The mirror@lists.opensuse.org mailing list (previously called suse-mirror@suse.de) is low-traffic and used mainly for announcements. It is also a suitable place for discussions around mirroring openSUSE content, should the need arise. To subscribe, simply write to mirror-subscribe@lists.opensuse.org.
The general contact address is: admin@opensuse.org
There is an IRC channel named #opensuse-mirrors at irc.libera.chat.
How to set up a mirror
See here for a howto: openSUSE:Mirror_howto
rsync modules
The rsync modules on stage.opensuse.org and rsync.opensuse.org are mostly identical. The former has additional content which is yet to be released, and since the latter syncs from it, there is a short sync time lag between them.
Sizes of the rsync modules are triangulated nightly:
An example of a commandline syncing from a module could look like this:
rsync -rlptyH rsync.opensuse.org::opensuse-full /srv/pub/opensuse/ --delay-updates --delete-delay -hi --stats
Note that this example clones the opensuse-full module, consuming a lot of disk space. The --exclude and --exclude-from arguments might be of interest, if you intend to mirror only parts of a module.
To perform a dry-run before actually downloading content and filling your disk, use rsync with -n.
modules of main interest:
NOTE: Parts of this section are outdated. The hotstuff modules are currently only available on staging, and are not dynamically updated.
- opensuse-hotstuff-160gb: The most requested files, which fit into 160 GB. This currently includes the install repo and CD/DVD media of the latest product, its updates, and the most popular other repositories. This is the most suitable module for mirrors with limited disk space. The majority of requests goes on exactly these files.
- opensuse-hotstuff-80gb: An even more restricted selection of most popular files, restricted to 80 GB of space. Use this if your mirror has very limited disk space. Still, the majority of requests goes on the files included in this module, so it is highly useful to mirror "only" these files.
- opensuse-updates: This rsync module provides the /update tree, with official updates for released openSUSE distributions, starting with openSUSE 10.3.
- opensuse-full: This rsync module provides the complete content of download.opensuse.org, except the tumbleweed directory. The reason to exclude this directory is the high frequency of updates inside.
- opensuse-full-with-factory: The same as the previous one including the tumbleweed directory containing the Tumbleweed Distribution.
- opensuse-source: This rsync module provides the /source tree, which contains source packages of openSUSE 11.1 onwards. Only available on stage.opensuse.org, but without access restrictions.
- opensuse-debug: This rsync module provides the /debug tree, which contains source packages of openSUSE 11.1 onwards, and includes released updates. Only available on stage.opensuse.org, but without access restrictions.
- opensuse-history: This rsync module provides historical copies of openSUSE releases. For the foreseeable future this only includes snapshots of Tumbleweed primarily for use with tumbleweed-cli. When rsync'ing this module, make sure you use the '-H' option, otherwise it will take up far too much space. See mirror announcement thread for more details.
modules for mirroring the Build Service repositories:
- buildservice-repos: The complete content
- buildservice-repos-main: Everything, but not the home: projects of the users
Updates do happen all the time, whenever a repository from the Build Service got rebuilt and updated. It is also possible to get the updates pushed.
Pushing support for Build Service updates
stage.opensuse.org does also host all content from the Build Service. Since the updates do happen all the time, whenever a new package set got built it is also possible to get the content pushed, instead of polling for it. The obviously requires rsync write access for stage.opensuse.org on your server. The advantages of that method are that
- the mirror is almost always up to date,
- no need to run rsync calls via all repositories. The pushing will only update the repositories which have changed. This does reduce the IO load of the mirror a lot.
- the redirector running at http://download.opensuse.org/ is aware that the packages got updated and can immediately redirect to the mirror.
How to become a pushed mirror?
The usual way (but we can also support a different way) is to open a rsync module on your server, where stage.opensuse.org gets write access. A login and password is optionally possible, but not really needed. Please write a mail to mailto:admin@opensuse.org afterwards where you tell us:
- the server name where to push
- the rsync module to be used
- optional account and password
- What the public download url will be.
Then, we need rsync read access to scan your mirror for our download redirector. The download redirector database needs to be updated periodically so it reflects the actual files on your mirror. The scanning happens from 195.135.220.0/22.
Be aware that the build service repository is forever growing, currently (Nov 22 2020) it takes up 16Tb.