openSUSE:Libzypp native package installation

Jump to: navigation, search

Goal

Allow for the installation of non-rpm packages, with the following requirements

  • Transparent: for the user it should be the same user experience
  • Compatible: information recorded in the rpm database
  • Extensible: possible to add more "handlers"

Usecases

  • People installing jars from Maven central or gems from rubygems.org
  • No need to package 300 gems to get something running, even if it is 90% automated with gem2rpm

Food for thought

While we could use the native tools for downloading the package, but we want to give the same user experience:

  • Download features, proxy settings, etc.
  • Solver speed, GUI apps showing native packages together with rpm repositories
    • We need to convert the metadata to the solv format the solver operates on

Therefore an option could be:

  • Implement zypp::Downloader for rubygems.org
    • It would download http://rubygems.org/Marshal.4.8.Z and call gem2solv (as we have rpmmd2solv and rpmdb2solv)
      • It would be great to contact https://twitter.com/qrush/ and ask if we can get the same index in a more readable format. In any case we need a full index (all gems and its dependencies, we operate on local cache only)
    • gem2solv would apply the conventions: gem foo -> rubygem-foo as a zypp::Package, same with dependencies.
  • Installing would end downloading the gem files
  • ZYpp would detect that .gem is not .rpm, run an external convertor, like gem2rpm we have today
    • Native packages requiring -devel packages will be a problem. We can't inspect the extconf.rb in the gem before downloading it, too late for the solver
  • ZYpp would run the convertor and install the rpm, which in turns calls gem install, like we do today with gem2rpm, but in real time.

Technical information & Research

rubygems

Full-Full index, all versions and dependencies: 

   % irb -rubygems -ropen-uri
   irb(main):003:0> gems = Marshal.load(Gem.inflate(open("http://rubygems.org/Marshal.4.8.Z").read)); nil
   => nil
   irb(main):004:0> gems.size
   => 55891
   irb(main):005:0> gems.first
   => ["cafepress-search-1.0.0", #<Gem::Specification:0x11fd0dc @licenses=[], @summary="A client library for the Cafepress search API that allows you to search for designs and products on Cafepress.com", @post_install_message=nil, @description="A client library for the Cafepress search API that allows you to search for\ndesigns and products on Cafepress.com", @dependencies=[<Gem::Dependency type=:runtime name="0.3.0" requirements=">= 0">, <Gem::Dependency type=:runtime name="happymapper" requirements=">= 0">, <Gem::Dependency type=:runtime name="hoe" requirements=">= 2.3.1">], @requirements=[], @bindir="bin", @require_paths=["lib"], @date=Thu Dec 17 00:00:00 -0500 2009, @name="cafepress-search", @new_platform="ruby", @loaded=false, @extensions=[], @authors=["Britt Crawford", "Dimple Joseph"], @cert_chain=[], @required_rubygems_version=#<Gem::Requirement:0x1200534 @requirements=[[">=", #<Gem::Version "0">]]>, @loaded_from=nil, @has_rdoc=true, @specification_version=3, @original_platform=nil, @version=#<Gem::Version "1.0.0">, @license=[], @extra_rdoc_files=[], @files=[], @signing_key=nil, @default_executable=nil, @email=nil, @test_files=[], @required_ruby_version=#<Gem::Requirement:0x1200624 @requirements=[[">=", #<Gem::Version "0">]]>, @rdoc_options=[], @rubygems_version="1.3.5", @homepage=nil, @executables=[], @platform="ruby", @autorequire=nil, @rubyforge_project=nil>]

Full API here: http://guides.rubygems.org/rubygems-org-api/

  • Full index seems to be (unknown format)
   http://rubygems.org:80/latest_specs.4.8.gz
  • API to query for the metadata of _one_ gem (in JSON format)
   https://rubygems.org/api/v1/gems/sfcc.json

returns: 

   {"name":"sfcc","downloads":1794,"version":"0.5.0","version_downloads":97,"authors":"Duncan Mac-Vicar","info":"ruby-sfcc allows to access a CIMOM either with the WBEM protocol or by using the SfcbLocal interface provided by the sblim-sfcb CIMOM implementation from the sblim project.","project_uri":"http://rubygems.org/gems/sfcc","gem_uri":"http://rubygems.org/gems/sfcc-0.5.0.gem","homepage_uri":"http://www.github.com/dmacvicar/ruby-sfcc","wiki_uri":"","documentation_uri":"http://rdoc.info/projects/dmacvicar/ruby-sfcc","mailing_list_uri":"","source_code_uri":"http://www.github.com/dmacvicar/ruby-sfcc","bug_tracker_uri":"","dependencies":{"development":[{"name":"mocha","requirements":">= 0.9"},{"name":"rake-compiler","requirements":">= 0"},{"name":"shoulda","requirements":">= 0"},{"name":"yard","requirements":">= 0.5"}],"runtime":[]}}
  • API to query for dependencies (returned in ruby marshall format)
   http://rubygems.org/api/v1/dependencies?gems=sfcc,enhancerepo

returns 

����[ { : nameI" sfcc�:�ET:�numberI" 0.5.0�;�T: platformI" ruby�;�T:�dependencies[{ ;I" sfcc�;�T;�I" 0.1.2�;�T;�I" ruby�;�T; [{ ;I" sfcc�;�T;�I" 0.1.1�;�T;�I" ruby�;�T; [�[�I" yard�;�TI" >= 0�;�T[�I"�shoulda�;�TI" >= 0�;�T{ ;I" sfcc�;�T;�I" 0.1.0�;�T;�I" ruby�;�T; [�[�I" yard�;�TI" >= 0�;�T[�I"�shoulda�;�TI" >= 0�;�T{ ;I"�enhancerepo�;�T;�I" 0.4.1�;�T;�I" ruby�;�T; [ [�I"�activesupport�;�TI"�>= 2.3�;�T[�I" nokogiri�;�TI"�>= 1.4�;�T[�I" log4r�;�TI" >= 1.0.5�;�T[�I"�trollop�;�TI" >= 1.0.5�;�T

or in hex-dump format: 

00000000  04 08 5b 0a 7b 09 3a 09  6e 61 6d 65 49 22 09 73  |..[.{.:.nameI".s|
00000010  66 63 63 06 3a 06 45 54  3a 0b 6e 75 6d 62 65 72  |fcc.:.ET:.number| 
00000020  49 22 0a 30 2e 35 2e 30  06 3b 06 54 3a 0d 70 6c  |I".0.5.0.;.T:.pl| 
00000030  61 74 66 6f 72 6d 49 22  09 72 75 62 79 06 3b 06  |atformI".ruby.;.| 
00000040  54 3a 11 64 65 70 65 6e  64 65 6e 63 69 65 73 5b  |T:.dependencies[| 
00000050  00 7b 09 3b 00 49 22 09  73 66 63 63 06 3b 06 54  |.{.;.I".sfcc.;.T| 
00000060  3b 07 49 22 0a 30 2e 31  2e 32 06 3b 06 54 3b 08  |;.I".0.1.2.;.T;.| 
00000070  49 22 09 72 75 62 79 06  3b 06 54 3b 09 5b 00 7b  |I".ruby.;.T;.[.{| 
00000080  09 3b 00 49 22 09 73 66  63 63 06 3b 06 54 3b 07  |.;.I".sfcc.;.T;.| 
00000090  49 22 0a 30 2e 31 2e 31  06 3b 06 54 3b 08 49 22  |I".0.1.1.;.T;.I"| 
000000a0  09 72 75 62 79 06 3b 06  54 3b 09 5b 07 5b 07 49  |.ruby.;.T;.[.[.I| 
000000b0  22 09 79 61 72 64 06 3b  06 54 49 22 09 3e 3d 20  |".yard.;.TI".>= | 
000000c0  30 06 3b 06 54 5b 07 49  22 0c 73 68 6f 75 6c 64  |0.;.T[.I".should| 
000000d0  61 06 3b 06 54 49 22 09  3e 3d 20 30 06 3b 06 54  |a.;.TI".>= 0.;.T| 
000000e0  7b 09 3b 00 49 22 09 73  66 63 63 06 3b 06 54 3b  |{.;.I".sfcc.;.T;| 
000000f0  07 49 22 0a 30 2e 31 2e  30 06 3b 06 54 3b 08 49  |.I".0.1.0.;.T;.I| 
00000100  22 09 72 75 62 79 06 3b  06 54 3b 09 5b 07 5b 07  |".ruby.;.T;.[.[.| 
00000110  49 22 09 79 61 72 64 06  3b 06 54 49 22 09 3e 3d  |I".yard.;.TI".>=| 
00000120  20 30 06 3b 06 54 5b 07  49 22 0c 73 68 6f 75 6c  | 0.;.T[.I".shoul| 
00000130  64 61 06 3b 06 54 49 22  09 3e 3d 20 30 06 3b 06  |da.;.TI".>= 0.;.| 
00000140  54 7b 09 3b 00 49 22 10  65 6e 68 61 6e 63 65 72  |T{.;.I".enhancer| 
00000150  65 70 6f 06 3b 06 54 3b  07 49 22 0a 30 2e 34 2e  |epo.;.T;.I".0.4.| 
00000160  31 06 3b 06 54 3b 08 49  22 09 72 75 62 79 06 3b  |1.;.T;.I".ruby.;| 
00000170  06 54 3b 09 5b 09 5b 07  49 22 12 61 63 74 69 76  |.T;.[.[.I".activ|
00000180  65 73 75 70 70 6f 72 74  06 3b 06 54 49 22 0b 3e  |esupport.;.TI".>|
00000190  3d 20 32 2e 33 06 3b 06  54 5b 07 49 22 0d 6e 6f  |= 2.3.;.T[.I".no|
000001a0  6b 6f 67 69 72 69 06 3b  06 54 49 22 0b 3e 3d 20  |kogiri.;.TI".>= |
000001b0  31 2e 34 06 3b 06 54 5b  07 49 22 0a 6c 6f 67 34  |1.4.;.T[.I".log4|
000001c0  72 06 3b 06 54 49 22 0d  3e 3d 20 31 2e 30 2e 35  |r.;.TI".>= 1.0.5|
000001d0  06 3b 06 54 5b 07 49 22  0c 74 72 6f 6c 6c 6f 70  |.;.T[.I".trollop|
000001e0  06 3b 06 54 49 22 0d 3e  3d 20 31 2e 30 2e 35 06  |.;.TI".>= 1.0.5.|
000001f0  3b 06 54                                          |;.T|
Retrieved from "https://en.opensuse.org/index.php?title=openSUSE:Libzypp_native_package_installation&oldid=52468"