Interactive Crash Analysis

From openSUSE

Keywords: crash analysis, Apport, gdb, debuginfo, DWARF, Summer of Code

Summary

The main idea of this project is to make a debugging tool for openSUSE which handles program crashes and processes backtraces to provide developers with more detailed information about occurred errors. On the other hand it'll significantly improve user experience by automatically retrieving debug information.

GSoC Student: Nikolay Derkach

Mentors: Jan Blunck, Dirk Mueller

Rational

openSUSE provides debuginfo packages which help providing usable backtraces for application crashes. Currently however, the user is required to manually figure out which debuginfo packages are necessary, then install them and afterwards try to reproduce the crash. Part of this proposal would be to write a tool to capture crashes as they occur, to analyze the coredump, figure out which debug information is needed, to offer the user to download it, and then afterwards generate a backtrace and clean up the debuginfo/core dump.

Design

Basically my proposal consists of several parts:

Initial crash processing

To handle incoming core dumps the existing kernel infrastructure will be used. A core is piped to a crash handling application by setting a corresponding core_pattern. Before processing a core dump the handler will ask a user if it should download required debuginfo packages by means of zypper (PackageKit probably).

Core dump processing

After that, gdb does some basic core analyse (bt full, info threads, probably exposes the nearest frames), for that purpose its batch interface will be used. After all of that the crash handler generates a report and stores it in some directory. If needed, a core dump itself could be saved to disk or uploaded to an external server.

User interface

A daemon started on the system startup monitors with inotify() a specified directory for new reports. I want to keep the service as small as possible, so that it does its only task to monitor for reports and run an external frontend program. As soon as it finds new ones, it starts a GUI tool to notify a user about crashed applications. Besides a report (and probably a core dump), the user frontend will also retrieve information about a running system (like installed packages). Except a user frontend in Qt, if time permits, I will additionally write a simple CLI for further processing core dumps and reports. All these applications (GUI interface, CLI, crash handler) will use a common library. Moreover I want to get rid of default KDE and GNOME userspace crash handlers to make an equal look-and-feel for a better user experience. CLI interface among all it features has to have a notification via email (for servers).

Bugzilla

A GUI tool will have an option to submit a report to Bugzilla. Since Bugzilla requires authentication, an intermediary server is required for users to upload their dumps/reports. After that these reports could be scanned, checked for duplicates and uploaded with some default Bugzilla entry. But this is something to do after all.

Implementation

openSUSE crash analysis tool will be based on Apport. The first part of the implementation will be to adapt Apport to a local openSUSE 11.0 environment. After this starts an optimization, followed by feature adding. Feature tracking is done internally via Agile project management tool Acunote.

Details

Comments

• Q: What are the differences between this project and Apport?

• A: The project tends to be pretty similar to Apport. The core handler also uses existing kernel functionality (core_pattern). Also we want to write everything performance critical in C, the user interface will be most likely left in Python.

 Nikolay Derkach
> As for the GUI, I would like to have the core part of this project used in the
> already existing applications (bug-buddy, drkonqi) rather than having a new GUI.
> It could be done specific for openSUSE as part of GSoC, but if well thought, it
> should allow other distros to use it also.

I have looked at drkonqi and frankly I'm not happy with using it at all. First of
all my project will be far more general (not just a userspace wrapper for KDE/GNOME
apps). Secondly, it has a bulk of legacy code and patching it means rewriting.
Moreover, using existing desktop crash handlers will introduce additional complexity 
to my system. After all, I want a common user interface. All in all, I believe it's 
not worth to.

 Dirk Mueller
I think it would be a good idea to make both possible (provide a unique gui, but also 
design it in a way so that bugbuddy or drkonqi can interface with it easily).

• Q: Does that mean that the project will make a fork of Apport?

• A: Not really. We are generally against forking. We will adapt Apport to openSUSE and rewrite some parts when it's needed. If possible, we will commit our changes to the upstream. Also, a part of the project is to create a openSUSE-specific server infrastructure.

Get It!

Note: you need openSUSE 11.0 (Factory) to try Apport, since older kernels are not supported.

openSUSE adaptation of Apport is developed with using Bazaar RCS, the project is hosted on Launchpad.

To get the most recent version you need to install bzr client (Install with one click) and the run the following command:

bzr co http://bazaar.launchpad.net/~nderkach/apport/opensuse

RPM packages for openSUSE Factory will follow soon...

Contact

If you are interested are in the project, have any questions, want to provide feedback, feel free to write your comments in this page discussion or contact Nikolay Derkach directly (please use 'Interactive Crash Analysis' in a email subject).