openSUSE:Infrastructure policy
This policy should help everyone involved having the right expectations and assumptions to successfully provide the infrastructure behind openSUSE.
openSUSE infrastructure policy
Servers are either hosted in Prague (Czech Republic), Nuremberg (Germany), or Provo/UT (USA). New deployments will predominantly take place in Prague. All servers must comply with the requirements outlined below.
General
All servers are enrolled in Salt, our configuration management solution. This takes care of managing a baseline configuration, to have servers comply with basic requirements (see "Base requirements"). Other requirements however, albeit partially being managed by Salt as well, require active caution by the operator (see "Further requirements"). It is not permitted to circumvent or overwrite the configuration applied by Salt, and the Salt Minion service must be operational at all times. Operators are encouraged to additionally manage all configuration for the services hosted on their machine using Salt as well. All changes to configuration managed by Salt, must be requested using a merge request in https://gitlab.infra.opensuse.org/infra/salt/ (public, read-only, mirrors can be found on https://code.opensuse.org/heroes/salt and https://github.com/openSUSE/heroes-salt). The rules defined in the README file of this repository apply. All further references to the "Salt Git repository" refer to this.
Base requirements
- Passphrase based shell login is disabled, only SSH-key login is allowed.
- Elevation to root is possible using `sudo` for users in the respective, service-related, admin group.
- Servers are configured with a static network configuration (IP address, DNS and NTP servers).
- Servers are configured to use the appropriate update repositories. For distribution updates, the internal download mirrors should be used.
- Automatic updates are enabled:
- For distributions releasing updates with patch information, this implies at least automatic security updates.
- For distributions releasing updates as snapshots, this implies automatic distribution upgrades.
- All machine respond to ICMP echo requests (ping).
- All machines run an operational monitoring client.
- All machines are configured with remote syslogging.
Further requirements
- Only the minimum required connectivity should be permitted in the perimeter and host firewalls.
- All public services which should be reachable from the internet and technically can be, should be, hosted behind the designated reverse proxy servers.
- Root and service passphrases must be stored PGP encrypted in the passphrase store (https://gitlab.infra.opensuse.org/infra/pass).
- Anyone maintaining a server in the openSUSE infrastructure must be subscribed to the heroes@lists.opensuse.org mailing list.
- Anyone maintaining a server must monitor at least the tickets directly assigned to them on https://progress.opensuse.org/projects/opensuse-admin/issues.
- Persons listed as responsible for a server are expected to respond to e-mails within 48 hours.
- It is desirable that the responsible persons are additionally reachable on IRC in ircs://irc.opensuse.org/#opensuse-admin.
- The list of responsible persons (listed in the Salt repository under pillar/id/<machine domain>.sls) must be kept up to date.
- This includes delisting oneself, if one decides to no longer maintain a service.
- Machines may not require manual intervention after a reboot. All services must correctly start automatically.
- Servers must run a supported version of openSUSE as their operating system.
- Software should preferably be installed from the official distribution repositories, provided by the internal download mirrors. Software which cannot be found in said repositories, should be packaged and hosted in OBS, primarily under openSUSE:infrastructure or a service-specific subproject. Software only found in Devel projects should be linked to such a project. Direct use of Devel projects is only acceptable for interpreted software requiring large dependency chains. Use of Home projects is not permitted.
- Services reachable over the network should be protected with a distribution provided Mandatory Access Control System (preferably AppArmor, alternatively SELinux).
- The machine must be enrolled with Salt and the Salt provided configuration must be applied. It is recommended to store and manage all further configuration in the Salt Git repository as well and to use Salt for deployment.
- Circumvention of any policies (including, but not exclusive to, introducing backdoors) is prohibited.
- Exceptions must be discussed through any one of the communication channels, and agreed upon in a ticket in https://progress.opensuse.org/projects/opensuse-admin/issues, giving other team members a veto time of at least two weeks. Accepted exceptions must be linked in /Exceptions.
- Outages (whether planned or unexpected) must be tracked under https://status.opensuse.org.
- All outages lasting more than 10 minutes must be announced. Planned outages must be announced at least 24 hours in advance. Unexpected or emergency outages should be announced as soon as possible, but no later than when the outage is resolved. All outage announcements must be sent to the Heroes mailing list (heroes@lists.opensuse.org). For outages related to public-facing community services, outage announcements must additionally be sent to the Project mailing list (project@lists.opensuse.org).
Conclusion
Adding new services requires agreement to this policy and a request to the openSUSE infrastructure team by opening a ticket in https://progress.opensuse.org/projects/opensuse-admin/issues.
Violations of the policy will be documented and escalated to the openSUSE board. In emergency or security cases the openSUSE infrastructure team as well as SUSE reserve the right to shut down a server. People listed as contacts will be informed accordingly.
All running servers might be evaluated for compliance at any time.
If a service is deemed unmaintained or the server hosts content/services which may no longer be needed, the maintainer (responsible person) on record will be contacted to work out a solution. At least one person listed as responsible for the respective server(s) must respond within two weeks. If no response is received within a 2 week period, or a response is received but compliance issues are not mitigated within 4 weeks, the server will be disconnected or shut down.
Communication
IRC channel
- #opensuse-admin on the Libera.Chat network.
Issue tracker
- https://progress.opensuse.org/projects/opensuse-admin/issues (alternatively admin@opensuse.org) - Issue reports about the openSUSE infrastructure.
Mailing List
- heroes@lists.opensuse.org - Discussion list for discussion of and announcements related to the openSUSE infrastructure.
Subscribe - Unsubscribe - Help - Archives
Other communications
- List of responsible people to contact if you experience problems with specific services.
- A list of services provided for the openSUSE community along with their known status can be found here.
- A blog with generic infrastructure news can be found here.
- For ways to communicate with us individually check our list of members.