openSUSE:Testing meeting 20101115

Jump to: navigation, search
Testing Core Team Meeting
Testing Core Team - Testing

Agenda

  1. OpenQA
  2. 11.4-MS3 experience
  3. Look back to OSC2010
  4. Bugzilla statuses

Transcript

(19:01:15) bmwiedemann: so, it is 18:00 UTC (19:01:36) Holgi: yes (19:01:46) bmwiedemann: welcome everyone to todays testing-core-team meeting :) (19:02:02) bmwiedemann: I am amazed to see quite a filled IRC channel today (19:02:56) bmwiedemann: so, today agenda... (19:03:07) bmwiedemann: so, todays agenda... (19:03:34) bmwiedemann: is empty on http://en.opensuse.org/openSUSE:Testing_meeting (19:03:37) bmwiedemann: hmmm (19:03:59) Holgi: we have opensuse 11.4 m3 (19:03:59) ReferenceSeete: Hehehe, oops. (19:04:09) bmwiedemann: of course, I have something that can be looked at, too. which is http://openqa.opensuse.org/cgi-bin/currentresults (19:04:32) Holgi: look back to OSC2010 (19:04:37) Holgi: and open qa (19:05:13) Holgi: if jürgen would be there: junior jobs (19:05:19) tigerfoot: bmwiedemann: it's really becoming more & more usefull ... (19:05:29) ReferenceSeete: I also have a question about triage. So we've got a good improptu agenda (19:05:56) ReferenceSeete: *impromtu* (19:06:06) bmwiedemann: very well then. (19:06:19) bmwiedemann: shall we start with the openqa topic? (19:06:21) ReferenceSeete: Good grief - error with a correction (19:06:49) Holgi: bmwiedemann: you are the boss ;-) (19:07:42) bmwiedemann: hehe. last summer I had asked the openSUSE board for sponsorship of a server for automated testing and while it had taken some time, this machine finally went into operation 3 weeks ago (19:09:04) bmwiedemann: it now also tests LXDE and XFCE fully automated. and I added some nice CGIs to the webserver, so that you can now get a good idea of a test-result in mere seconds (19:09:38) bmwiedemann: (also got some help with icon and CSS/styling from people in #opensuse-marketing : -) (19:09:51) Holgi: bmwiedemann: very good :-) (19:09:59) LWFinger: Agreed. (19:10:02) tigerfoot: thx goes to yaloki & gnokii (19:10:12) bmwiedemann: and nmarques (19:10:25) ReferenceSeete: Fanastic, bmwiedemann (19:10:45) Holgi: bmwiedemann: looks like you still use your 'old' domain (19:10:48) ReferenceSeete: Sorry, can't spell today apparently (19:11:28) bmwiedemann: the other news is that I started working for SUSE two weeks ago. doing other automated testing there, too (19:11:33) karsten_ heißt jetzt remur_030 (19:11:45) Holgi: bmwiedemann: even better :-) (19:11:52) LWFinger: bmwiedemann: Is that openqa site you quoted above where the results come out? (19:12:07) tigerfoot: bmwiedemann: are you founding the baracus project ? or it's just coincidencial (19:12:31) bmwiedemann: LWFinger: yes. openqa.opensuse.org does the actual testing and hosts the results, too (19:13:01) LWFinger: I presume it is OK to list that in the Weekly News. (19:13:39) bmwiedemann: LWFinger: AJ proposed a separate announcement in news.o.o (19:14:03) bmwiedemann: but I wanted this thing polished before. so feedback for improvements are welcome. (19:14:17) Holgi: nevertheless I think larry should also mentioning it in the weekly news (19:14:24) Holgi: bmwiedemann: when do you plan the extra news? (19:15:17) bmwiedemann: Holgi: how long does it take on LWN to have general public access to subscriber-only content? 7 days? (19:15:44) Holgi: bmwiedemann: I don't know (19:15:45) bmwiedemann: because there is a nice article about the testing dated 2010-11-10 (19:16:24) CzP: then two more days (19:16:29) remur_030: bmwiedemann: 7 days yes (19:16:46) remur_030: the article is very nice and you can send subscriberlinks if you want to show it around =) (19:17:30) remur_030: hm it was in the weekly news so it's public on thursday (19:17:40) bmwiedemann: for reference: the article is https://lwn.net/Articles/414413/ (19:17:40) Holgi: bmwiedemann: that part of the "look back to OSC2010" ;-) (19:18:11) bmwiedemann: thanks for the info remur_030 :) (19:18:37) tigerfoot: 3 days to wait ... :-) (19:18:57) Holgi: bmwiedemann: can't we widespread the subscriber link? (19:19:01) bmwiedemann: 3 days to polish the tool up even more. e.g. I am still unsatisfied with the front-page (19:19:14) bmwiedemann: not sure. it was AJ's link. (19:19:37) remur_030: Holgi: I don't think lwn is very happy about that (19:19:52) remur_030: anybody here hasn't seen the article yet? (19:20:03) LWFinger: I have not. (19:20:18) Holgi: ok, let's do it that way: http://www.olli-ries.net/?p=256 (19:21:20) Holgi: bmwiedemann: what's next? (19:21:28) LWFinger: Holgi: Thanks. It is now in my browser. (19:21:59) ReferenceSeete: Holgi: Thank you (19:22:23) bmwiedemann: if someone wants to help me with openqa, please tell me after the meeting. we should move on to the next topic (19:22:28) bmwiedemann: opensuse 11.4 m3  ; look back to OSC2010 ; ReferenceSeete's question about triage (19:22:52) bmwiedemann: so M3 came out a week late because of M2's most-annoying bug, but M4 should be on time (19:23:40) bmwiedemann: my testing showed some problem with libreOffice, but it turned out to be rather small (19:24:06) bmwiedemann: who else tried M3? (19:24:20) Holgi: I just did basic installation tests (19:24:27) Holgi: on i386 (19:24:41) LWFinger: I have two i686 installations, both from KDE Live CD, one VM, one real machine. (19:24:43) tigerfoot: bmwiedemann: I'm actually working on a day to day factory (19:25:10) LWFinger: I have no problems yet, but limited testing. (19:25:27) tigerfoot: and sometime (but need more time) launch the factory installer in netinstall & remote install . (actually the two works) (19:25:28) bmwiedemann: tigerfoot: I also have factory on my laptop (because sound+wifi did not work with 11.3), but I only update it sometimes... last was a week before M3 (19:25:30) ReferenceSeete: I installed it last night, I was on Factory/GNOME before that. (19:26:00) ReferenceSeete: Small issues to report, will get on them after the meeting. (19:26:05) tigerfoot: bmwiedemann: I do it every day :-) but only work with kde (19:26:17) LWFinger: bmwiedemann: Any improvement on that wifi problem? (19:26:37) Holgi: 7 bug reports so far (19:26:45) bmwiedemann: LWFinger: dont know. dont have wifi in Nuremberg. (19:27:26) bmwiedemann: so some bugs are in there. anything big that really hurts? (19:27:27) Holgi: bug #653497 probably the worst (19:28:17) LWFinger: There has been discussion on the opensuse kernel ML about using kernel 2.6.37 in 11.4. Any thoughts? (19:28:35) LWFinger: I think it is a good idea. (19:28:47) ReferenceSeete: It's a great idea for users...if it lands well. (19:29:14) bmwiedemann: LWFinger: I heard that this might really happen. (19:29:15) ReferenceSeete: But we're always up against this sort of thing. (19:29:40) LWFinger: So far, I have had only minimal problems. It should be fine by RC2 or RC3. (19:30:57) bmwiedemann: so it seems like M3 finally was a good milestone release :) (19:31:42) bmwiedemann: (ya know: I want all future milestones without major/obvious problems) (19:31:53) LWFinger: As do we all! (19:31:57) bmwiedemann: any more bugs to discuss? or shall we move on? (19:32:02) remur_030: kde 4.6beta1 is scheduled soon ;-) (19:32:29) bmwiedemann: remur_030: I read that kdepim-4.5 was already canceled because of that (19:32:57) ReferenceSeete: What is our GNOME footprint right now? (19:33:05) remur_030: bmwiedemann: yeah, looks somewhat tight even for 4.6 now =/ (19:33:35) bmwiedemann: ReferenceSeete: I had automated installs of GNOME-LiveCD and DVD with firefox+openoffice+yast tests (19:33:40) ReferenceSeete: And do we want to target the 3.0 release to help the Gnome Team? (19:34:04) ReferenceSeete: I know they are thinking of doing an unofficial respin when it is released. (19:34:07) bmwiedemann: ReferenceSeete: 3.0 will probably not be default for 11.4 - maybe as preview (19:34:11) ReferenceSeete: Right. (19:34:24) ReferenceSeete: But we pride ourselves on good desktops all around... (19:34:48) bmwiedemann: current desktops are good, too. (19:34:53) ReferenceSeete: I'm going to do it, I was just curious about general interest. (19:35:49) bmwiedemann: I had been discussing with yaloki to include KDE-Unstable in automated tests. if there are liveCDs from GNOME3, that could be included as well (19:36:32) ReferenceSeete: bmwiedemann: I understand, that would be helpful. I'll bring up that possibility with vuntz/Gnome Team (19:37:33) bmwiedemann: Gerald Pfeiffer was also interested in test statistics from Fedora RawHide (which I dont have yet) :) (19:38:00) ReferenceSeete: bmwiedemann: other than a liveCD, what else do you need to make the process as smooth as possible for you? (19:39:07) bmwiedemann: ReferenceSeete: liveCD might already be enough to detect problems. however, backtracing was often bad, because no matching debuginfo rpms were available. (19:40:32) bmwiedemann: lets discuss this later and move on to next topic (19:40:43) bmwiedemann: "look back to OSC2010" (19:41:19) bmwiedemann: tigerfoot, Holgi, javier_, jradzuweit and me have been there at least (19:41:21) Holgi: there have been 4 people attending OSC2010 from this audience (19:41:50) Holgi: sorry, 5 then (19:41:54) bmwiedemann: was really nice to see a face after all this time working together (19:42:35) Holgi: both bmwiedemann and me had a talk, bmwiedemann also a dev session (19:42:51) Holgi: I just want to mention: I talked about testopia (19:42:59) Holgi: http://conference.opensuse.org/indico//contributionDisplay.py?contribId=13&confId=0 (19:43:07) Holgi: there slides are available there (19:43:38) Holgi: unfortunately tigerfoot found an issue with testopia (19:43:45) Holgi: that is still not solved (19:44:09) Holgi: accessing testplans is still just possible for users with granted access (19:44:49) bmwiedemann: and there are also plenty pictures online - e.g. me giving talk http://news.opensuse.org/2010/10/20/opensuse-ass-kickin-keynote/ajs_6081/ (19:45:56) LWFinger: Nice picture. (19:47:30) Holgi: the link to the lwn report is already posted (19:48:19) tigerfoot: as feedback, it was a great & nice thing to discover, all that stuff. thx Bernard & Holgi ... and the report is also really great. ( now I just need time to setup it in a more industrial way ) ... (19:50:57) bmwiedemann: there was also another proposal from OSC2010 (19:51:29) bmwiedemann: the idea is to have links that lead to bugzilla searches (19:51:59) bmwiedemann: and we could scan this for issues worth being on most-annoying bugs list (19:52:45) bmwiedemann: (possibly with workarounds so that people would not have to wait for a fix just to continue testing) (19:54:47) ***tigerfoot sorry need to go now, but will follow discussion here, and on the ml too... thx javier_ to drive me here :-) (19:54:51) bmwiedemann: this reminds me that for MS3, I intervened in the marketing team to get the link to most-annoying list back in (they did forget it again) (19:55:21) bmwiedemann: tigerfoot: was nice having you with us. (19:56:03) ReferenceSeete: tigerfoot: have a good day/evening (19:56:18) Holgi: cu tigerfoot (19:57:17) LWFinger: I will need to lerave soon also. I will leave the link up to see what is said. (19:57:32) LWFinger: *leave* (19:58:01) bmwiedemann: have fun. (19:58:20) bmwiedemann: would anyone here like to do the bugzilla-scanning? (19:59:31) ReferenceSeete: Scanning for most-annoying bugs? (19:59:35) bmwiedemann: yes (19:59:49) bmwiedemann: just for factory and latest milestone (20:00:21) ReferenceSeete: Sure, what criteria are you looking for? Typical blockers type things? Or most duplicates? (20:01:44) bmwiedemann: either. (20:02:10) bmwiedemann: this searches for open issues on MS3: https://bugzilla.novell.com/buglist.cgi?classification=openSUSE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=NEEDINFO&bug_status=REOPENED&version=Milestone%203%20of%206&product=openSUSE%2011.4 (20:02:15) ReferenceSeete: Sure thing. I'll add it to my triage tasks. (20:03:35) bmwiedemann: very good. this brings us right to the last point on todays agenda... about triaging (20:04:59) ReferenceSeete: Yes, two things. First, about our role in triaging Bugzilla. (20:05:11) ReferenceSeete: I understand that Novell has its own internal QA (20:05:38) ReferenceSeete: I've asked about us closing/resolving bugs, and was told that it's on a team-by-team basis. (20:06:01) ReferenceSeete: The KDE team's perspective seems to be 'go ahead'. (20:06:32) ReferenceSeete: Given that we're an OSS project I assume we can just go ahead and triage since that helps everyone. (20:06:45) bmwiedemann: closing as in "worksforme/wontfix" or with real fixes? (20:06:54) ReferenceSeete: Things that we know are fixed. (20:07:28) remur_030: we have some guidelines for kde bug triages, let me see if i can find it again (20:07:46) ReferenceSeete: Yes, I've seen that wiki article. But openSUSE is more than KDE. (20:08:11) bmwiedemann: well. there was one problem with bugzilla. when people enter new bugs, they are shown the list of "open" issues. so if you mark an issue as resolved or closed, chances are, someone will waste his time, opening a dup of it (20:08:19) ReferenceSeete: http://en.opensuse.org/openSUSE:Bug_Screening_KDE (20:08:28) remur_030: ReferenceSeete: yes, just as reference (20:08:54) bmwiedemann: so I tended to leave issues open, until a fix was available (20:09:01) ReferenceSeete: Well a bug I had in mind was this one: https://bugzilla.novell.com/show_bug.cgi?id=640584 (20:09:06) bugbot: openSUSE bug 640584 in openSUSE 11.4 (Banshee) "banshee: Got a SIGSEGV while executing native code." [Major,New] (20:09:20) ReferenceSeete: The problem the user had (and I verified) existed on an older version of Banshee. (20:09:37) ReferenceSeete: Since updating the package in Factory to 1.8, the problem no longer exists. (20:09:41) ReferenceSeete: Therefore: Resolved. (20:10:32) bmwiedemann: OK. the issue was for MS1, so after 1-2 months, one could ask the user, if the problem persists with current version (20:11:16) bmwiedemann: (not here, but in other cases) (20:11:28) remur_030: bmwiedemann: could there be a seperate 'status' that will be considered open for the open issues list? (20:12:42) bmwiedemann: https://bugzilla.novell.com/enter_bug.cgi?product=openSUSE%2011.4&format=guided is linked from the wiki's bugreporting page (20:13:11) bmwiedemann: not sure if/how that can be tuned. Holgi, do you know more there? (20:13:39) remur_030: because there is just a ton of bugs that will not see any work but still be left open then (20:13:43) Holgi: bmwiedemann: I did not get the question - status for what? (20:13:44) ReferenceSeete: remur_030: yes that wiki article is a good reference! (20:14:47) remur_030: ReferenceSeete: thruth be told in case something isn't severe enough we close it often as upstream because the -kde team lacks the man power (20:14:59) bmwiedemann: ah. it appears to show "RESOLVED" issues (20:15:18) remur_030: there we track if there will be a patch added that can be backported (20:15:28) remur_030: but mostly nothing happens their either ;-) (20:16:48) bmwiedemann: so issues with a bugfix should not be marked "CLOSED", but "RESOLVED" until the next milestone is out (20:17:29) ReferenceSeete: Fair. (20:17:39) ReferenceSeete: Which leads me to the next question. (20:18:04) bmwiedemann: some other bugtrackers also have a "UNCONFIRMED" state to distinguish between "maybe-bugs" and "known-to-exit-bugs" (20:18:34) bmwiedemann: which can help in triaging (20:18:46) ReferenceSeete: "UNCONFIRMED" and "EXPIRED" would both be helpful. (20:19:50) ReferenceSeete: I cannot mark a bug as "RESOLVED" without "specifying a comment". Which I do not know how to do. (20:20:11) bmwiedemann: ReferenceSeete: just write some text in the big text-area (20:20:34) ReferenceSeete: So I just need to add some extra talk? (20:20:35) bmwiedemann: e.g. "This seems to be fixed/working with the current version" (20:21:12) bmwiedemann: some other projects have elaborate prepared texts for this (20:21:40) ReferenceSeete: Ok, great. (20:22:10) remur_030: EXPIRED seems like a nice substate for the outdated stuff (20:22:27) remur_030: as in closed-> expired or resolved-> expired (20:22:27) ReferenceSeete: I was hoping I could link back to an existing comment that detailed the fix. But this is fine. (20:22:34) remur_030: resolved sounds wrong here though (20:22:58) ReferenceSeete: How so? (20:23:01) remur_030: ReferenceSeete: you can, the comment #x is a link, just copy&paste this (20:23:23) remur_030: ah it's 'comment x' =) (20:23:53) ReferenceSeete: True, lack of creativity on my part there. (20:27:07) bmwiedemann: Holgi: any chance, we can get the "UNCONFIRMED" state enabled in bugzilla? (20:27:23) Holgi: bmwiedemann: my guess is: no (20:27:24) ReferenceSeete: Apparently we have an "UNCONFIRMED" status in bugzilla. ReferenceSeete remur_030 (20:27:34) ReferenceSeete: We just don't use it? (20:27:48) bmwiedemann: ReferenceSeete: it is there, but disabled. (20:28:40) ReferenceSeete: So, your question to Holgi stands... (20:28:47) Holgi: bmwiedemann: but we could work using "tags" (20:29:00) bmwiedemann: are tags public? (20:29:21) Holgi: bmwiedemann: I think so, but not sure (20:30:23) bmwiedemann: e.g. I have a tag "watchlist114dev" https://bugzilla.novell.com/buglist.cgi?cmdtype=runnamed&namedcmd=watchlist114dev (20:31:26) bmwiedemann: but I think it is non-public (20:31:35) bmwiedemann: we could use the new/assigned bit (20:31:58) Holgi: bmwiedemann: that is the usual way at suse (new/assigned) (20:32:29) bmwiedemann: only that you often do not know about possible assignees (20:32:45) Holgi: bmwiedemann: true (20:33:28) Holgi: bmwiedemann: good guess to assign to teamleads of component (20:33:33) bmwiedemann: or assigning to bugzilla-screening? (20:33:54) Holgi: bmwiedemann: that's also possible (20:35:41) bmwiedemann: OK. so we use the new/assigned bit instead of UNCONFIRMED (20:35:58) remur_030: afaik assigned usually removes the current assignee, in the case of kde this is the kde maint team, so if you are watching this you'll get no updates after 'assigned' (20:36:34) bmwiedemann: you sure? (20:37:18) Holgi: remur_030: if kde maint team is still in CC list then you should see it ReferenceSeete remur_030 (20:37:40) Holgi: remur_030: usual prior assignees are in CC automatically (20:38:15) bmwiedemann: ReferenceSeete: instead of expired there is resolved/closed "noresponse" or "wontfix" or "worksforme" (20:39:32) remur_030: https://bugzilla.novell.com/show_activity.cgi?id=652519 (20:39:36) ReferenceSeete: bmwiedemann: Ok. I don't suspect I'll be expiring too many bugs, since I'm currently more concerned with 11.4 than 10.3. Still, it's too bad because all 3 miss the intent of expired. (20:40:05) remur_030: the bug gets reassigned, and the last assignee isn't in the cc list anymore (20:40:45) bmwiedemann: ReferenceSeete: say, someone reported a minor issue and after 2 years with no fix, you just close it as "wontfix" (20:41:46) bmwiedemann: ReferenceSeete: say, someone reported a major issue but did not give enough information to reproduce... and after some time (and asking again), you just close it as "noresponse" or "worksforme" (20:41:57) Holgi: remur_030: maybe because of the address is marked as mailing list (20:42:22) ReferenceSeete: bmwiedemann: I understand. (20:42:57) bmwiedemann: or maybe you had something different in mind with "expired"? (20:44:43) remur_030 hat den Raum verlassen (quit: Remote host closed the connection). (20:45:52) bmwiedemann: we should end todays meeting then. if anyone has any proposals/help to improve openqa.opensuse.org webpages, please email me or the opensuse-testing list (20:46:05) ReferenceSeete: bmwiedemann: It's really about cleaning up bugzilla with a good reason that fits the purposes of the triager. The TCT isn't deciding "wontfix", nor are we declaring "worksforme" - it very well could still be broken. "noresponse" after 2 years just indicates that we haven't been triaging well enough. (20:47:01) bmwiedemann: 2 years was just an example. during factory 1-2 months could be more appropriate (20:47:54) ReferenceSeete: It's really about having clear 'sight lines' on why the bug is being closed by a particular person. But it's not a big deal, we just indicate that "expiry" is the true reason in the comments. (20:48:42) ReferenceSeete: Just transparency really. (20:48:52) bmwiedemann: OK (20:49:23) bmwiedemann: having well-written, polite longer texts is also good (20:49:43) bmwiedemann: such as "Dear user, I am closing this bugreport today because... (20:50:25) ReferenceSeete: Makes perfect sense. Bug reporters appreciate feeling like they are listened to. FOSS projects lose too many testers to feelings of being ignored or slighted. (20:51:31) bmwiedemann: maybe we should have a en.opensuse.org wiki page with those texts (20:51:45) bmwiedemann: and explainations when+why to use which (20:52:57) ReferenceSeete: Perhaps I should bring this issue back to the devs? Because it's still not clear that all the teams are on board with "external" triage. (20:53:45) ReferenceSeete: I don't think that there will be a deluge of people triaging with an agenda, but I don't want to break anyone's workflow either. (20:54:56) bmwiedemann: having the texts + workflow for reviving or expiring old bugs could still be useful. (20:55:04) bmwiedemann: sometimes you can just do things (20:56:50) ReferenceSeete: Ok. (20:58:26) Holgi: bmwiedemann: is there anythin more on the agenda? (20:58:51) bmwiedemann: no (20:58:53) Holgi: bmwiedemann: it's quite late (20:59:14) bmwiedemann: will go to sleep then. was nice to have you in the meeting today (20:59:15) Holgi: bmwiedemann: and we also covered 18:00 utc ;-) (20:59:55) bmwiedemann: when shall the next meeting be? (21:00:19) Holgi: Thu, Nov 25, openSUSE 11.4 Milestone 4 release (21:00:21) bmwiedemann: Monday 2010-12-06 18:00 UTC? (21:00:43) Holgi: not sure if nikolaus is a good choice (21:01:02) Holgi: parents may be busy on that evening (21:01:03) bmwiedemann: hm. nikolaus comes 12h before (21:01:28) Holgi: bmwiedemann: you are the expert here ;-) (21:02:42) Holgi: 2010-12-06 then? (21:02:56) bmwiedemann: should be good. (21:03:12) Holgi: objections or all felt asleep? (21:03:27) bmwiedemann: who will upload the log? (21:03:37) ReferenceSeete: Well it's midday in Canada, so...no sleepiness here. (21:04:05) Holgi: ReferenceSeete: I see (21:04:58) Holgi: ok, no objections (21:05:37) bmwiedemann: meeting is then closed.

Retrieved from "https://en.opensuse.org/index.php?title=openSUSE:Testing_meeting_20101115&oldid=30068"