Libzypp/Metadata/YaST/content syntax

From openSUSE

Contents

YaST/Zypp media description

This page covers mainly the syntactical aspects of content, etc. See here for the semantic definition.

Motivation

UnitedLinux defines a set of base components for a Linux server distribution. These come on a single and clearly defined media, which is identical for every UnitedLinux vendor.
Applications are installed separately and the install media contents are set up by the vendor.
In order to make application installation and esp. media (content) descr iptions commonly defined, this document proposes a such a description format.

Requirements

  1. Multiple products per media
    A media may contain several products, i.e. a combined patch media.
  2. Multiple architectures per product
    A media may contain products for multiple architectures.
  3. Bi-Arch support
    An architecture may be bi-arch, where mixing of 'compatible' architectures is allowed.
  4. Multiple languages per product
    A media may contain products for multiple languages.
  5. Multiple media
    a) The content may be splitted onto multiple media.
    b) It must be able to merge splitted media into a single directory.
    c) Media can be double-

sided (e.g. DVD-ROM).

  1. A media must be easily recognizable by the system
    This is needed when re-inserting a cd-rom media for further package installation. The system must verify that the correc

t media is inserted and can be used for package installation.

  1. product name
  2. product version
  3. base distribution
    specify the distribution the product is based on.
  4. base distribution version
  5. product vendor
  6. user-readable content description
    This is needed to present media information to the user prior to installing packages. The user must be able to decide on this informatio

n to proceed or cancel the installation process.

  1. package path
    toplevel path to packages (i.e "suse/" on SuSE media, "unitedlinux/" on UL media, etc.)
  2. default language
    default language if no language is determinable.
  3. inital server for online update
  4. no wildcard search for files
    doing a wildcard search is cheap on a local media but almost impossible via HTTP. So it should be avoided if possible.

Implementation

The media and product description is splitted to a set of files, separating the media preparation data from the contents (product) and making it possible to combine several produ cts on a single media.
At toplevel of a media, a directory


       /media.1/


should exists.


If it does not exist, the presence of the correct media can not be verified. This may be convenient for an on-line media which just provides a set of packages.
In this case , no verfication takes place.


The numeric extension depends on the media number in case of splitted medias. The first media has .1, the second .2, and so on.

This allows merging all medias of a splitted set into a single directory. YaST always knows which media to expect and can thereby directly access the correct media directory.

Files inside the media.N/ directory

Inside the media.N/ directory the files media and products must exist.
Optionally, a file named patches may exist to denote a media with patches (e.g. for YaST online update).

The media.N/ file contains the media identifier needed to determine correctness when using removable media.
In case of split media, it must be present on every media of the set.

The products file contains directory specifications for each product.
In case of split media, it must be present of the first media of the set.

The patches file contains a single directory specification for the root of the patches directory tree and an optional list of exclusive products the patches apply to.
In case of split media, it must be present on the first media of the set.

The /media.N/media file

This should contain a key to quickly identify correctness of the inserted media. It is not shown to the user but should contain human-readable data for debugging purposes.

Proposal: Encode Vendor and Timestamp information in a file

File: media
Location /media.N/ directory on media
Content two or more lines of ASCII as follows
<vendor>
<YYYYMMDDHHMMSS>
[<media count>]
[

<media flags>]
[<media names>]


The identification data is stored inside the file on purpose. It could also be encoded in the file name (older SuSE media did it). But this requires a directory scan for f ile access and should be avoided.

By storing the vendor name in the file, the timestamp can be kept vendor specific. (It's the name of the vendor who created the medi a which might be different from a product vendor! It's just used as an additional key for media identification.)

All media of a media-set must provide identical vendo r and timestamp information in the media file.

The /media.1/media file

The /media.1/media file may contain additional information.
The remaining media files of a media-set may repeat this information, but it is not evaluated again.

The <media count> is needed if the installation data is split up on multiple (identical) medias.
The media count is a numerical value (all

characters are digits), and defaults to 1.

<media flags> may further describe the media.
In order to distinguish the flags from the coun

t line, the flags values start with a letter.

Currently defined flags:

flag meaning
doublesided media is double sided, YaST will ask for 'front side' for odd-numbered media and 'back side' for even-numbered media.
The default is single-sided media.

<media names> may define alternate strings to use when asking to insert a certain media.
They are defined as <key><whitespace><value> pairs, separated by \n.

key value
MEDIA<N> UTF-8 encoded string to use when asking for media <N>.
MEDIA<N>.<lang> UTF-8 encoded translation of MEDIA<N> may be defined by appending the ISO <language> or <language>_<country> code

.

Example

linux# mount /dev/cdrom /mnt
linux# ls /mnt/media.1
media
products
linux# cat /mnt/media.1/media
SuSE Linux AG
20020921153042
3
MEDIA1 SUSE Linux 9.1 DVD 1
MEDIA2 SUSE Linux 9.1 DVD 2
MEDIA3 SUSE Linux 9.1 DVD 3
linux#

The products file

This file contains the directory name for each product contained on the media.


If this file is not present, a single product on the media's root directory is assumed.



It is an ASCII file with one line per product.

Each line starts with the directory name (relative to the media's root dir !), followed by whitespace (space or tab) followed by the product name and version.
A leading slash in the directory name is only needed to specify the root directory of the media.

File: products
Location /media.1/ directory
Content one or more lines of ASCII as follows
<directory> <whitespace> <product> <whitespace> <version>'

'


Example


linux# mount /dev/cdrom /mnt
linux# cat /mnt/media.1/products
/      SuSE Linux Personal 8.1-0
sles-8 SuSE Linux Enterprise Server 8-0
openlinux      Caldera OpenLinux 4.0-0
unitedlinux-1.0        UnitedLinux 1.0-1
linux# ls -a /mnt
...
media.1/
...
sles-8/
openlinux/
unitedlinux-1.0/
...

In the above example, "SuSE Linux Personal" uses the toplevel directory, whereas the other products have their own sub-directories.

The content file

This file resides inside the product directory as specified by the products file on the media. It contains all product-specfic data to describe and identify the contents of the product.


If there was no products file, the product directory defaults to the root directory on the media.



Proposal:

File: content
Location product-dir as specified in /media.1/products file
Content UTF-8 encoded <key><whitespace><value> pairs, separated by \n
For allowed keys see next table.

Mandatory key strings

prio key allowed value
M PRODUCT product name
M VERSION product version and release as in RPM
<major>.<minor>-<release>
M DISTPRODUCT distribution id (vendor specific)
M DISTVERSION distribution version (vendor specific)
O BASEPRODUCT name of base product if different from PRODUCT
O BASEVERSION version of BASEPRODUCT, mandatory if BASEPRODUCT is defined
<major>.<minor>-<release>
M VENDOR vendor name (free form)
M ARCH.<base> space separated list of allowed architectures for <base>
(see below for further explanation)
M DEFAULTBASE Minimal architecture base supported by this product.
Default <base> architecture if no matching ARCH.<base> is found.
O TYPE The type of the product. The only value is base when a product is a base product. Add-on product has this key missing.
M REQUIRES resolvables which must be installed on the system in order to 'fulfill' this product.
This is a space separated list of names or kind:name pairs optionally followed by version constraints.
Just a name denotes a dependency to a package, i.e. sles-release or suse-release == 10.1
kind can be 'package', 'pattern', or 'product', i.e. pattern:sles-base == 10.0
O LINGUAS ISO <language> or <language>_<country> code
O PATTERNS List of patterns pre-selected by the product.
O LABEL UTF-8 encoded label.
Default label if LINGUAS is omitted or no default language can be determined.
O LABEL.<lang> UTF-8 encoded label.
<lang> has the same syntax as the LINGUAS values.
For each language in LINGUAS a matching LABEL.<l

ang> is expected.

M DESCRDIR package description directory
(relative to product directory)
M DATADIR package data directory
(relative to product directory)
O LANGUAGE default language code
O TIMEZONE default timezone (currently does not have any effect on how YaST selects the time zone).
O RELNOTESURL url of release notes, see below
O FLAGS product specific capabilities, see below
O YOUURL URL of text file containing list of online update servers
O YOUTYPE comma-separated list of YOU flags. business for SuSE business products, noyou for products to be ignored by YOU
O YOUPATH path to patches on the server
(triggers YOU to look at the maintenance web server for updates.
M META metadata checksum, for a repository metadata file
whitespace-separated list of chksum-type chksum-value file
the file path is relative to the root-dir of the media (resp. repository)
O HASH file checksum, for a linuxrc file
whitespace-separated list of chksum-type chksum-value file
the file path is relative to the root-dir of the media (resp. repository)
O KEY repository key checksum
whitespace-separated list of chksum-type chksum-value file
the file path is relative to the root-dir of the media (resp. repository)

prio 'M' == mandatory, 'O' == optional
Either LABEL or LABEL<lang> must be present.

The ARCH.<base> line needs a bit more explanation:
Supporting bi-arch architectures and optimized packages requires multiple (architecture optimized) occuran ces of a package (under the same name and version).
In order to determine which the 'right' occurance is, YaST needs a fallback strategy (a search order).
The <base> tag is used as a starting point and determines 'the' architecture of the system. This is determined by running uname -m (and looking at /proc/cpuinfo).
For this <base>, a matching ARCH.<base> must exist. If it isn't found, <base> gets the value of DEFAULTBASE for which a matching ARCH.<base> is guaranteed.
The value of the ARCH.<base> line is the search order for packages. The first package matching the archite cture is taken as the preferred candidate.
Packages with architectures not in this list are discarded and do not appear for selection. Packages with 'lower' architectures a re still available for selection, so the user can manually choose such a package.

The RELNOTESURL line is used to define the location of the release notes rpm. Currently only http is supported. The rpm is installed with rpm -U --replacepkgs 'basen ame of url'.

The FLAGS line is used to describe the (installation) capabilities of the product. It's value is a list of space separated keywords.
The following keywords are cur rently defined:

update Offer update button in YaST
usercwd Add '.' (current working dir) to PATH environment.
Normally, this is a risk but some users can't live without it.
no_you Do not offer online update during installation
SP This source is a service pack
cache Try to cache packages on disk (in order to reduce CD shuffling)
Only the first source (ordered by priority) will really be cached.

META is marked as mandatory since the content files main purpose is repository information. And whats a repository worth without metadata.
HASH is optional since its only needed for bootable (resp. self-installable) media.
KEY is optional since packages on the media might be signed with an already known key.


The location of the rpm file for a package on the installation media is determined by the =Loc: line in the [package-descr.html packages ] file.

Example


linux# mount /dev/cdrom /mnt
linux# cat /mnt/sles/content
PRODUCT SuSE Linux Enterprise Server
VERSION 10.0-0
BASEPRODUCT Code10
BASEVERSION 10.0-0
VENDOR Novell Inc
ARCH.x86_64 x86-64 i686 i586 i486 i386 noarch
ARCH.i686 i686 i586 i486 i386 noarch
ARCH.i586 i586 i486 i386 noarch
DEFAULTBASE i386
LINGUAS en de_DE
LABEL.en SuSE Linux Enterprise Sever 
LABEL.de_DE SuSE Linux Enterprise Sever
DESCRDIR suse/setup/descr
DATADIR suse
REQUIRES foo pattern:bar
LANGUAGE de_DE
TIMEZONE Europe/Berlin
RELNOTESURL http://www.suse.com/relnotes/sl_82/release-notes.rpm
FLAGS update
YOUURL http://www.suse.de/cgi-bin/suseservers.cgi
YOUTYPE business
YOUPATH i386/update/SuSE-SLES/8
META SHA1 1c3e58329ad53ab5b4fb864fec163e9d4027eb07  base-10.3-113.i586.pat
HASH SHA1 172b3cf77268f46e783e78a653902e563cb91e9a  license.tar.gz
KEY SHA1 c0354069c10819674da8706822e1d4bd0c1797e9  gpg-pubkey-0dfb3188-41ed929b.asc
linux#



The patches file

This file contains a single directory name for the root of the patches directory contained on the media and an optional list of exclusive products the patches apply to.
If this file is not present, the media's root directory is assumed and no restrictions about the products are made.

The first line of the file starts with the directory name (relative to the media's root dir!), followed by whitespace (space or tab) followed by an optional comment with the description/patchlevel of the Patch CD.
A leading s lash in the directory name is only needed to specify the root directory of the media.

Additional lines are used to specify exclusive products. If no additional lines are present, no exclusive products are assumed. If there are exclusive products the online or patc h CD update ignores all products on the installed system which are not contained in the list of exclusive products. If there are no exclusive products all products on the install ed system are used for the update.

The exclusive products are listed as one product per line in the form of the product name and the product version separated by a dash.

Proposal:

File: patches
Location /media.1/ directory
Content one line of ASCII as follows
<directory> <whitespace> <optional comment>
zero or more lines specifying exclusive prod

ucts: <productname>-<productversion> '


Example


linux# cat /mnt/media.1/patches
/ Patch-CD SuSE-SLES 7 (i386) Patchlevel 4
linux# cat /mnt2/media.1/patches
patches Patch-CD SuSE-SLOX 4 (i386) Patchlevel 1
SUSE CORE-9


The package description directory

Each such directory contains a file listing the selections (selections) and a file listing the packages (packages).

Example


linux# grep Enterprise /mnt/media.1/products
sles   SuSE Linux Enterprise Server
linux# grep DESCRDIR /mnt/sles/content
DESCRDIR suse/setup/descr
linux# ls -R /mnt/sles/suse/setup/descr
...
packages
selections
...
linux#

The packages file

This is basically a cache file for package data needed for package selection and dependency resolution.
See current [package-descr.html here] for the syntax.


The packages file must exist in order to enable a media.



The package file contains pure package / rpm data and no user readable string (i.e. translations).

The packages.LANG file

For each language defined in content:LINGUAS a packages.LANG file exists in the content:DESCRDIR directory.
This file contains translated str ings (i.e. package summary, description, etc.) to be viewed by the user.


The packages.LANG file might be ommitted. The package summary defaults to name-version-release.architecture in this case.



See [lang-descr.html here] for a format definition of such a file.

The packages.DU file

This file contains disk usage information for each package and directory used by the package.
It it used to approximate file system requirements esp. when multiple partitio ns (i.e. for /usr, /var, /opt, ...) are used.
Exact usage information can't be computed by YaST since it heavily depends on hard or symbolic link s (either in the package or in the file system) and the filesystem in use (i.e. ext2 vs. reiserfs).

See [du-descr.html here] for a format definition of this file.

(N.B. The .DU suffix does not interfere with the .LANG suffix since the language suffix starts with lower case letters.)


The packages.DU file might be ommitted. No size estimations can be given in this case.


Example


linux# grep Enterprise /mnt/media.1/products
sles   SuSE Linux Enterprise Server
linux# grep DATADIR /mnt/sles/content
DATADIR suse
linux# grep ^ARCH /mnt/sles/content
ARCH.x86_64 x86-64 i686 i586 i486 noarch
ARCH.i586 i586 i486 noarch
linux# ls -R /mnt/sles/suse
...
x86-64/
i686/
i586/
i486/
noarch/
...
linux#

The selections file

This file contains a list of selections file names (one per line, possibly with directory pathes, relative to the DESCRDIR directory).
See current *.se l for the syntax. (To be defined).


The selections file might be ommitted. YaST tries to access *.sel files in the DESCRDIR directory in this case.


Example


linux# cat /mnt/suse/setup/descr/selections
Games.sel
Gnome.sel
Kde-Desktop.sel
Kde.sel
LAMP.sel
Linux-Devel.sel
Minimal.sel
Multimedia.sel
Network.sel
Office.sel
Standard.sel
SuSE-Documentation.sel
X11.sel
default.sel
linux#

Reading directories

When YaST reads a directory by first listing its content, it looks for a file directory.yast and takes its content as directory listing, if it exists. If it doesn't exis t, it tries to list the directory by using the media backend. Listing directories isn't supported by all backends, e.g. the HTTP backend doesn't support directory listings. So di rectories which are required to be listable by YaST for all media have to include the directory.yast file.

The file can be created by ls >directory.yast. YaST ignores the directory.yast entry itself and the . and .. entries.