- Note: The meeting will take place on IRC, because it's the holidays and we're going to be very relaxed....
|Meeting of the QA Team
previous - next
|Date and Time (UTC)
||2013-12-30 19:30 UTC
|[ This Meeting] IRC Meeting today!
Joren, Joel, Robinson (aka 'colonelqubit'), Arnaud, Sophie, Cloph (briefly)
|Talkyoo Room Number
|537138 IRC Meeting today!
Agenda + Minutes
Content from the meeting itself ("the minutes") will be displayed in blue boxes like this.
Before we Start
A few minutes before we officially start the call, we'll try to have a few people on hand to help assist newbies set up their teleconferencing gear, get things muted, and log on to all of our communication channels:
Put start time and attendees list in the table
The State of QA
Including any Opening Discussion
- Bigger hat for Joel Madero: He's been elected to the Board
- T-shirts update:
- More T-shirts delivered to:
- Still en-route to:
- Hackert, Mandal, Viehöver, Münch
- BugHunting Session 4.2.0
- Bjoern: update re: Kiva gift cards?
- Updater news?
- colonelqubit: T-shirts: I'm still hoping to hear back from 4 of our helpful volunteers.
- The tracking #s the post office gave me all stop in New York
- jmadero: we know they at least got that far
- colonelqubit: yeah, a couple of them were tracked into Europe, but some were delivered but still show as "in transit"
- I'll check w/people and see who is going to be at FOSDEM, and try to hand-deliver what I can
UNCONFIRMED bug count:
- December 16th: 956
- Today (30th): 781
- Difference: -175
- colonelqubit: Let's see -- the UNCONFIRMED bug count is down an amazing 175 bugs to 781!
- This is certainly the lowest # of UNCONFIRMED bugs I've seen in my days here in QA
- jmadero: 781 impressive!
- colonelqubit: What makes me happy about our team right now is that there's often someone in the QA channel when I stop by
- an active channel means that people are more likely to stop and stay, which leads to more bug-chomping. so...a big thanks to all of you!
- Too bad Steve Bell and Reisi and Rob aren't here!
- congrats to them as well
- jmadero: next year I think our goal should be no bug more than 3 months old in UNCONFIRMED
- about 2/3rds currently are more than 90 days
- Focusing on new UNCONFIRMED bugs is good policy. If someone reports a bug and doesn't get it in some resolved or NEW state in 3-6 months, irritation follows
- colonelqubit: Here's a good metric "What are the stats (mean, median, worst/best case) for how long before a bug gets touched by someone other than the reporter?
- This is doable, but we'd need to find time to code it up.
Topics for ESC?
Does anyone have any topics/concerns that I should bring to the Engineering Steering Committee (ESC)? (Robinson)
I'm going to bring up the following bugs:
- (Perhaps more a11y bugs?)
- jorendc: jmadero: colonelqubit: what if it's a confirmedRegression (whiteboard)? ...isn't it then just a 'regression' in keywords?
- jmadero: I'm still confused about that
- colonelqubit: Yes, that's one of those niggling implementation details in our bug tracker. For consistency and sanity, I'd suggest PossibleRegression and ConfirmedRegression, both in the whiteboard, but FDO offers 'regression' in the keywords, and we've used that for some time, and I think we have some tooling that knows about that, so....something to fix in the future
- jmadero: does BSA put PossibleRegression in keywords?
- colonelqubit: jmadero: no, in whiteboard. keywords can only be from a set list
- jmadero: that's what I meant
- colonelqubit: Anything for me to bring to the ESC? There are a few bugs I know that people wanted me to address
- jmadero: um - nothing for this week for ESC - I am concerned about people committing things and not having time to fix their regressions
- colonelqubit: I think that's an ongoing concern for us in QA
- jmadero: but, let's not address it again to ESC...like hitting a dead cow
- well I guess one thing - if we find a relatively nasty regression and person does not have time to fix it, should we reverse the commits... or basically, how does QA measure benefits vs. costs of a commit vs. the regression is causes?
- colonelqubit: as we reduce the # of UNCONFIRMED bugs, and as we bisect more regressions, it will become more obvious if the build breaks or regressions are introduced and not promptly fixed
- So, the sooner we identify a regression, the more likely a commit will get reversed/re-written
- jmadero: I think QA needs to be the driving force behind finding more devs, as we have successfully grown our team; we seem to have some skill at it
- ACTION: Sophie will plan another (4.2) meeting -- with Native-Lang teams -- for January
- colonelqubit: sophi: any update about the 4.2 meeting for january?
- sophi: colonelqubit: I took some days off last week, so no, not now
- colonelqubit: good, good. everyone should be rested-up!
- sophi: colonelqubit: will begin to organize it by the end of the week
- ACTION: Robinson/Cloph will talk to Tollef about next steps in Bugzilla migration
- Waiting to hear back from Tollef
Topics to review:
- Bugzilla abbreviation
- Top contenders: lobz (7 votes) and lo (6 votes)
- Suggested reconciliation on 'lbz'
- Dec 18 - Will have run-off between lobz and lo if someone on the QA list asks for it
- Dec 28 - No run-off requested, we'll use "lobz#1234".
- ACTION: Get Tollef to activate voting on FDO (merged item)
- ACTION: Get Tollef to add BOLD statement to each FDO mail sent out to not reply via mail (merged item)
- ACTION: Get Tollef to clarify 'version' label in Bugzilla (merged item)
- ACTION: Will track the filed bug and wait to hear updates (re: Bold statement on FDO mail)
- colonelqubit: * Bugzilla Migration...the usual update.....
- colonelqubit sighs
- jmadero: I have risen it to the appropriate channels, so after holidays hopefully
- I also told him that you (Robinson) right now have perhaps more free time then you will in the near future, so the sooner the better, since you are the one taking on a ton of the work
- colonelqubit: In other bugzilla news, people preferred "lobz" as the Bugzilla abbreviation. And no, I haven't made up a lobster mascot yet...
- jmadero: so LOBZ# it is
- colonelqubit: but I'll make an Action item for myself
- jmadero: are we going to move back to 1, or are we keeping current #'s and we'll have skips?
- colonelqubit: jmadero: keep the current #s
- jmadero: ah, probably makes sense, but a little shitty
- colonelqubit: IIRC, we're at least 50% of the bugs on FDO, if not 2/3rds, so the gaps aren't too huge, and we'll blast forward into new swaths of bug #s in no time
- ACTION: Robinson will work on a new lobster mascot for our Bugzilla
- ACTION: Make combined bibisect repo available for testing (Robinson)
- IDEA: Add bibisect into regular Developer workflow
- IDEA: Create graph/tool to show which builds contain a given commit
- colonelqubit: I'm still working on bibisect docs. I'll ping the list when I'm done (perhaps today, if I can steal some time)
- colonelqubit: jmadero: did you say that the canonical bibisect repo is down again?
- arnaud_versini: colonelqubit: bibisect will be only for ubuntu or for other OS, like OpenSuse or Windows ?
- jmadero: arnaud_versini: goal is every OS
- colonelqubit: arnaud_versini: Right now we have a couple of different bibisect repos
- jmadero: long term goal
- colonelqubit: I think we'll always have a stronger concentration on GNU/Linux systems, but shm_get has been working on bibisect repos for OSX, and Reisi007 is working on a GUI for windows bibisect
- arnaud_versini: colonelqubit: Ok good thing to have all OS for that
- colonelqubit nods
- shm_get: colonelqubit I do not know though if anyone every used the mac bibisect....
- jmadero: shm_get: I think bibisect is going to grow with a GUI
- colonelqubit: shm_get: fair enough -- I'll sic SteveBell and joren on it soon
- jmadero: I'm almost sure we'll use it
- ultimately we hope that every bug is triaged with bibisect, so that we can locate the first version, and have a bibisect ready for every bug
- jmadero: indeed - I think a couple years from now it's the only thing QA will use, especially if we get the GUI functions we want, ie. automatic uploading to our bug tracker
- ACTION: Punt on implementation until after we have bibisect repos up and running (All)
- Florian is working on a Windows-only version
- colonelqubit: No Reisi007, so no update from him
- [DONE] ACTION: Sophie will create page in silverstripe
- [DONE] ACTION: Robinson will talk to cloph about getting page up (theme support?)
- Dec 18 - Cloph "can have a look later today or tomorrow - surely this week"
- Dec 23 - Page up on the Silverstripe draft site
- colonelqubit: http://qa.libreoffice.org/feedback/
- jmadero: +1
- colonelqubit: Big thanks to cloph for getting the code in place.
- jmadero: is the goal 4.2 release points to this?
- colonelqubit: jmadero: So that's a good question
- colonelqubit: Re: Feedback page, any reason we shouldn't try to get it into 4.2?
- jmadero: not that I can think of
- jmadero: the walrus - that's an awesome addition
- seems like you created a mascot
- another lesson of "just do" instead of ask
- sophi: colonelqubit: do not forget to connect with the site redesign, so it's not overlooked in the migration
- colonelqubit: sophi: exactly
- colonelqubit: So, regarding the Feedback page, two ACTION items
- colonelqubit: ACTION: Robinson will ping chs/website list to make sure the feedback page is styled per the site redesign
- colonelqubit: I don't have LO building on this machine right now -- who wants to make the commit changing the URL and push it to gerrit?
- colonelqubit: jmadero?
- jmadero: colonelqubit: I can dig and see, I'm sure it's not as easy as it sounds
- colonelqubit: jmadero: IIRC it's just in the code somewhere
- jmadero: lol hahaha
- jmadero: 19,000,000 lines
- jmadero: I'll find it
- colonelqubit: ACTION: Joel will make the commit changing the url in LibreOffice (and ask for it to target 4.2.0)
- colonelqubit: hooray!
- colonelqubit: We're getting things done!
- sophi: colonelqubit: just on the feedback page, some NL team will have it translated
- so the url will need to be adapted to NL pages too
- colonelqubit: sophi: I can't remember -- did I give you a bug about that?
- the url passes-in some variables like locale
- sophi: colonelqubit: and we are already in string freeze, so we have to make sure that the translation of the page is done and url in Pootle for 4.2.1
- colonelqubit: sophi: could you file a bug and cc me on it? Something like "Implement localization of Feedback page"
- sophi: colonelqubit: ok, and I'll take care of coordination with l10n/NL teams
- colonelqubit: sophi: you and I can chat w/cloph about the best way to select a translation based on the current locale
- (later, cloph showed up and pointed out that the url in the program is a hub url, so we can change what it points to at any time)
- jmadero: indeed - so my action item is done
- [DONE] ACTION: Robinson will ask ESC about the issue of offering the last version in a release series as 'stable' -> filing bugs, etc..
- e.g. Should we even give end users the last "bugfix" release? It's a dead-end version that won't have updates -- why not go from
- 4.0.5 - August 22nd, to
- 4.1.2 - October 4th ?
- colonelqubit: So as some of you may remember, we had a discussion about how we offer versions in the updater
- I talked w/ESC about the situation, and they said that it's a trade-off between giving users a newer version, and providing stability
- Joren and I came up with an interesting proposal: Offer users a choice of stability
- for example: "jorendc: "libreoffice is highly developed, would you like to follow "1: early adopt branch" "2: stable release" "3: bussiness stable release"
- jmadero: I like it
- colonelqubit: we could put that into the updater GUI in LibreOffice
- Our current updater path would turn into "Business stable release"
- jmadero: or have a (gasp) option...a drop down you select, just like Ubuntu has
- you select early adopter - it'll give you updates to the most recent major release
- colonelqubit: Yes, kind of like debian 'stable', 'testing', 'unstable', 'something we found in a lab'
- Of course, any change to our existing system may require dev time, so that can be an impediment to getting things done
- jmadero: but it's an idea
- colonelqubit: The nice thing here is that a handful of us on the QA team have read-over the updater code (Rob, me, ...maybe Joren?) and are interested in helping to push that side of things forward
- I'll take this plan to the next ESC meeting
- ACTION: Robinson will plead our case re: user-choice in updater to the Supreme Court of Steering Engineers
- cloph_away is in vacation mode - so just cross-read over the last lines: I don't think a "pick your level of stability" is the way to go.
- [DONE] ACTION: Joel will email out to the website mailing list asking about updating the Download page
Website Redesign Mockups:
- colonelqubit: sophi already mentioned that we want the Feedback page to have the new look-and-feel. Anything else relevant to QA?
- sophi: not from my side
- jmadero: not here
- jorendc: nops
- colonelqubit: sophi: let's try to keep QA appraised of the latest drafts of the new QA page, so people can give feedback
- sophi: colonelqubit: ok
- colonelqubit: ACTION: Sophie/Robinson - Keep QA in the loop re: updates to QA pages
- ACTION: Robinson will get an update on the latest progress here from Stuart/a11y Team
- ACTION: If you try to use LibreOffice without a mouse (and using a screen reader), how is the experience? (Everyone)
Relevant tracker bugs:
- colonelqubit: I haven't heard back from Stuart; I'll ping him again
- Please let me know about any a11y issues; in general, those can become MABs
- sophi: there has been some discussion on the design list, I'm not sure the design is still final
- ACTION: Rob/Joren will work to finish our MIME-type-fixing script!
- colonelqubit: jorendc: any update here?
- jorendc: think we fixed most of them
- colonelqubit: hooray!
- jorendc: correction, most of the interesting _documents_ (we can open using LibreOffice)
- colonelqubit: I know that it can be hard to do cleanup work when there's a permanent fix, but we can't implement it (yet)
- jorendc: probably still some work to do, but I think we did most of the work
- colonelqubit: so big kudos to joren for this work. who else was doing that work?
- jorendc: with help from Joel, Michael S, ...
- colonelqubit: Michael Stahl, that's right. Good on him
- sophi: yes, kudos to the team
- jorendc: btw, not that active lately related to our Twitter account, but when I did start ther were 95 followers. checked yesterday: 102
- so I think that's gerat
- the more followers, the more people see bugs passing by, so might get noticed and confirmed
- sophi: jorendc: yes, right
- ACTION: Mock something up/have something to show for next QA Meeting (Robinson, Joren)
- [SUPERSEDED] ACTION: Joel will come up with a timeframe for future Bug Hunting Sessions/Triage Contests
- Let's try to have an event every 6 months
- colonelqubit: Joel currently has an action item to come up with a timeframe for our events...any update?
- jmadero: colonelqubit: I sent out the mailing
- sophi had a suggestion but I think we couldn't do it, so NL will have to do their own thing
- someone had said they would convert the email to wiki. I haven't seen that
- jmadero: sophi correct me if I'm wrong. it's unfortunate, but it seems like it's too late in the game if we hold off until translation is done
- unless the goal of the bug hunting sessions were to find bugs to correct by x.x.1, not x.x.0
- sophi: jmadero: no problem, it was only a suggestion
- jmadero: I liked the suggestion, so I'm happy saying that we keep it after translation and we focus on getting bugs reported so they are fixed by x.x.1 not x.x.0
- other opinions?
- jmadero: we have to weigh NL vs. getting things timely found -- it's challenging
- if we do earlier bug hunt, then some NL teams will be left out, and translations are not complete
- the later we do it, less likely we find nasty bugs and that devs have enough time to fix them
- colonelqubit: true and true. I see two questions here
- colonelqubit: 1) When can we get the biggest bang for our buck?
- what's the best time for us to inject the most volunteer effort into improving the state of a new release
- colonelqubit: 2) When's the best time to have an event in terms of 2ndary factors like recruitment?
- jmadero: sophi you have a better idea of NL teams
- how many people do you think would join that wouldn't join if we did it earlier?
- sophi: it's difficult because we have so much changes in translation that we need serious checks, but the overall quality of the version is either important
- jmadero: sophi but would the NL teams still bug hunt for non translation issues (can most people get around with English version of LibreOffice), search for crashes, and the like?
- sophi: jmadero: less but they test with half translated versions too
- jmadero: it does suck to have a translation go bad, but not as bad as finding a crasher bug after release. The more people we have searching for those the better
- colonelqubit assumes that the string churn is too big to translate as we go
- jmadero: and how far after .0 does .1 come out...like 2-3 weeks?
- sophi: jmadero: yes
- colonelqubit: ReleasePlan
- jmadero: okay, I vote to keep it later in the .0 cycle then. RC1ish
- I'll do another go at a timetable, but I like more people involved, especially if we can get some of them in the chat
- worst case, we have a nasty bug that gets fixed 2-3 weeks after release, but we get more people involved -- that will help us going forward
- jmadero: and we still have our "core team" that will do checks throughout, so hopefully we 10-15 find the nasties
- jmadero: okay action item for me
- push another one out for timetable
- + explain rationale I suppose
- ACTION: Joel will come up with a timeframe for future Bug Hunting Sessions/Triage Contests, with rationale.
Long-term/Frozen Pending Items
These are long-term or frozen items (i.e. on ice until something else happens) that we don't want to forget about...
- ACTION: Joel will turn this into an EasyHack and work on it (slowly)
- ACTION: Joel will propose stricter guidelines for what qualifies as a MAB
- colonelqubit: jmadero: where are we with our big-picture plans?
- we've mentioned wanting to consolidate into 2 MAB lists. Are we targeting 4.3 with that?
- jmadero: colonelqubit: I think that is best - there are still some kinks that I need to think baout and talk through
- colonelqubit: would a separate meeting help?
- jmadero: I don't think it's necessary, I just need to come up with a real plan
- but currently, do we have a 4.3 MAB list? we do right?
- colonelqubit: I didn't create one
- colonelqubit: okay -- I was hoping that we'd have our alternate system ready by then [4.3]!
- jmadero: well we can change its name
- (jmadero: there is no 4.3 most annoying bug)
- jmadero: colonelqubit: can you verify one last time with ESC that they are happy with 2 lists: one for current master (pre stable release) and then one for everything else
- I know michael keeps a ton of stats, and this will screw that up for sure
- colonelqubit: what would they be named?
- jmadero: Pre-Release Current MAB, and then "MAB for everything else" obviously not that just need to come up with a useful nae
- colonelqubit: ok. Sure, come up with names and an overview, and I'll bring it to ESC
- jmadero: I might as well say it -- another dev said they aren't following any tracker that we do from MAB to priorities
- colonelqubit: If devs aren't going to follow those fields, we should just hide them, so we don't waste time on them
- jmadero: well hoepfully they carry over to the new tracker, that we can lock. then they can follow it..maybe
- colonelqubit: again, that requires buy-in, but yes, I'm hopeful
- jmadero: and the other thing, for new devs I think it's good to offer guidance, so with ESC as long as they write off to merging to 2 lists yet to be named, we'll just do it
- the only issue is that the list continues to grow and grow (with fixed bugs) so at some point we have 150 bugs, 140 of them closed, 10 open
- colonelqubit: sure. As I said, please get me a short plan/names/etc... that I can present to them
- colonelqubit: personally, I'm okay with anything *reasonable* in the short-term, as I hope us to mostly move to bugzilla fields in the future
- jmadero: sure
- colonelqubit: Anyhow, we should wrap up this topic for now (if anyone else is still awake..)
- ACTION: Joel will continue to monitor the situation
- ACTION: Put up a message on the message board at his University and see who is interested (Joel)
- ACTION: USA involvement in hackfests -- (Keep on the agenda, discuss on QA list) (All)
New Action Items
All items proposed between meetings go here
NEW ITEM: Title (Author)
New Items (Proposed/Discussed at the Meeting)
NEW ITEM: Daily builds for all platforms (Joren)
- jorendc: ah yes, topic to add to our agenda ... Availability of daily builds for all supported versions and platforms (Windows, Mac, Linux)
- colonelqubit: jorendc: you're taking that one?
- ACTION: Joren will take point on making sure we get daily builds for win/mac/linux
- jorendc: 1) not sure how to "take" that, because my knowledge of tinderboxes is 0,0%
- 2) studying/exams coming
- so not willing to take any action items right now (which I'm sorry about)
- colonelqubit: jorendc: not the technical side of it, just making sure it gets done
- okay, I'll take it -- no worries!
- colonelqubit: ACTION: Robinson will take point on making sure we get daily builds for win/mac/linux
- jmadero: good luck on that
- colonelqubit: jorendc: just because I bring something up to the ESC doesn't mean that it happens
- jorendc: yeah, true
- colonelqubit: Anyone have anything else?
- jmadero: nothing here
- jorendc: nopes
- colonelqubit: Okay! Our next QA meeting will be in two weeks on Jan 13th. Until then.... Happy New Year!
- jorendc: Happy NEWYEAR
- jmadero: happy new year all
- sophi: happy new year all
- jorendc: o/
- colonelqubit: jorendc: like santa, I'm always watching
- jorendc: revived o/
- sophi: bye all, good day/night!
Put meeting-adjourned time in the table
(Add topics below and reference them as #Example Topic in the Agenda/Minutes above)