Libzypp/Profiling

From openSUSE

RPM Database Reading

The code analyzed was:

ZYpp::Ptr z = getZYpp();    
z->initializeTarget("/");
ResStore res = z->target()->resolvables();
MIL << res.size() << " resolvables" << endl;

RAM usage

The procedure in this article was written and tested with version revision 5464 Whilst there is no guarantee, it should be applicable to later versions. If you find this to be incorrect, please help to update this article.

Current code

This involves creating all resolvables from the target. In the current implementation, all the data is kept in the rpm object, including changelogs, descriptions and summaries.

functions accounted for 85.0% of measured spacetime:

• 17.5% : 0x51F5C9E: __os_malloc_rpmdb (in /usr/lib/librpmdb-4.4.so)
• 14.2% : 0x4CF3FEA: std::string::_Rep::_S_create(unsigned, unsigned, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.8)
• 12.2% : 0x43925FA: __gnu_cxx::new_allocator<std::_List_node<zypp::ChangelogEntry> >::allocate(unsigned, void const*) (new_allocator.h:88)
• 10.5% : 0x431C1C3: __gnu_cxx::new_allocator<std::_List_node<std::string> >::allocate(unsigned, void const*) (new_allocator.h:88)
• 9.4% : 0x42F8681: __gnu_cxx::new_allocator<std::_Rb_tree_node<zypp::Capability> >::allocate(unsigned, void const*) (new_allocator.h:88)
• 6.8% : 0x51F5D5B: __os_realloc_rpmdb (in /usr/lib/librpmdb-4.4.so)
• 2.9% : 0x514C1DC: (within /usr/lib/librpmdb-4.4.so)
• 2.1% : 0x42FF910: zypp::capability::buildFile(zypp::KindOf<zypp::Resolvable> const&, std::string const&) (CapabilityImpl.cc:133)
• 1.5% : 0x42F85FF: __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<zypp::Dep const, std::set<zypp::Capability, zypp::CapOrder, std::allocator<zypp::Capability> > > > >::allocate(unsigned, void const*) (new_allocator.h:88)
• 1.3% : 0x45C312F: __gnu_cxx::new_allocator<__gnu_cxx::_Hashtable_node<boost::intrusive_ptr<zypp::capability::CapabilityImpl> > >::allocate(unsigned, void const*) (new_allocator.h:88)
• 0.9% : 0x5148B09: (within /usr/lib/librpmdb-4.4.so)
• 0.8% : 0x463ECDA: __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<zypp::Locale const, std::string> > >::allocate(unsigned, void const*) (new_allocator.h:88)
• 0.7% : 0x45C2FAD: __gnu_cxx::new_allocator<__gnu_cxx::_Hashtable_node<boost::intrusive_ptr<zypp::capability::CapabilityImpl> >*>::allocate(unsigned, void const*) (new_allocator.h:88)
• and 373 other insignificant places

Modified code

The code was modified to not keep the rpm header in memory. Only the data needed to build the resolvables (NVRD) was keep in ram. The rest should be requested to disk on demand.

YaST shows summaries of packages while installing them, this could break when on-demand attribute reading is implemented, as you cannot access the rpm db to read while installing, unless you do it first.

The new memory usage is like this:

• 21.8% : 0x520DC9E: __os_malloc_rpmdb (in /usr/lib/librpmdb-4.4.so)
• 16.5% : 0x4D0BFEA: std::string::_Rep::_S_create(unsigned, unsigned, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.8)
• 13.2% : 0x4303681: __gnu_cxx::new_allocator<std::_Rb_tree_node<zypp::Capability> >::allocate(unsigned, void const*) (new_allocator.h:88)
• 9.0% : 0x520DD5B: __os_realloc_rpmdb (in /usr/lib/librpmdb-4.4.so)
• 3.9% : 0x430A910: zypp::capability::buildFile(zypp::KindOf<zypp::Resolvable> const&, std::string const&) (CapabilityImpl.cc:133)
• 3.8% : 0x51641DC: (within /usr/lib/librpmdb-4.4.so)
• 2.7% : 0x43035FF: __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<zypp::Dep const, std::set<zypp::Capability, zypp::CapOrder, std::allocator<zypp::Capability> > > > >::allocate(unsigned, void const*) (new_allocator.h:88)
• 2.3% : 0x45D622B: __gnu_cxx::new_allocator<__gnu_cxx::_Hashtable_node<boost::intrusive_ptr<zypp::capability::CapabilityImpl> > >::allocate(unsigned, void const*) (new_allocator.h:88)
• 1.4% : 0x45D60A9: __gnu_cxx::new_allocator<__gnu_cxx::_Hashtable_node<boost::intrusive_ptr<zypp::capability::CapabilityImpl> >*>::allocate(unsigned, void const*) (new_allocator.h:88)
• 1.4% : 0x5160B09: (within /usr/lib/librpmdb-4.4.so)
• 0.9% : 0x4652922: __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<zypp::Locale const, std::string> > >::allocate(unsigned, void const*) (new_allocator.h:88)
• and 373 other insignificant places