QA/Meetings/2013/May 17

=Prep=



= Agenda + Minutes =

 Minutes:

Content from the meeting itself ("the minutes") will be displayed in blue boxes like this.

Start

 * Meeting started at: 13:08
 * Participants: Petr, Joel, Florian R., Bjoern, SteveBell (only IRC/pad), John Lehnov (new member?)

Notes:
 * Didn't have a special meeting on May 10; punted items from May 3 will be dealt with at this meeting.
 * Joel had to leave around 13:12; Robinson lead for the remainder of the meeting

Opening Discussion
 (none)

PENDING ITEM: Proposals regarding our workflow

 * PROPOSAL: Joel: Don't create proposal and AGREE to it on the same day. Instead we should discuss, add a PROPOSAL (keyword in bold), and allow those who do not attend call to reply via email. If no negative feedback by next call, we can mark as AGREED and remove the Item from our PENDING ITEMS list.
 * PROPOSAL: Joel: Slow down on some of our proposals/work. E.g. on the creation of the qa.libreoffice.org
 * PROPOSAL: Joel: If we make a decision that takes a lot of time for other people, wait a long time; 3+ weeks to hold on for people
 * PROPOSAL: Joel: Don't make a rule about how to triage things; Just lead by example


 * Punt to later in the meeting
 * PUNTED: Until next meeting (ran out of time).

PENDING ITEM: NEEDINFO Stagnant Bugs

 * [DONE] ACTION: Joel will write a wiki page about the process for closing bugs that have been in stagnant NEEDINFO, including links to the discussion, rationale for doing it, etc.

 bug will go to INVALID status after 30 days."
 * Made wiki - QA/Bugzilla/Fields/Status/NEEDINFO
 * Text will be something like "Your bugs have been in NEEDINFO status for 180 days...click this link to see our policy. Please attach information within 30 days or your
 * ACTION: Joel will do a test run today, then will email QA List (and core people) and ask for feedback.
 * If all is good, Joel will try to ramp up to 50+ a day

PENDING ITEM: Documentation for Localized French BSA

 * [DONE] ACTION: Joel will talk to Rob and will look for a 2nd developer for BSA code
 * [DONE] ACTION: Joel will hand over developers to Rob (as the BSA is Rob's baby)


 * Floeff says that the French BSA is operational (as far as he knows); we still need to ping Rob and wrap this up.
 * Bjoern - Rob asked about Hamburg Hackfest on IRC yesterday -- maybe find a contributor there (and also do some knowledge transfer)

PENDING ITEM: Get French BSA Operational

 * ACTION: Joel will continue to shepherd the project; Will coordinate with Rob


 * [Ed. TODO -> Merge with previous PENDING ITEM]
 * Continue to work on BSA

PENDING ITEM: What to do with FDO bugs filed against Extensions, Templates

 * ACTION: Joel will (try to) contact the maintainer of the LO Extension Site again
 * [DONE] ACTION: Joel will bring this up to the ESC -- we need to tell users up front that Extensions are not supported by us (especially when upgrading)


 * Thorsten contacted Andreas...
 * We need a way for users to contact extension/template developers
 * Joel - For now, re: Extension bugs, he's been cc'ing Mmeeks
 * In 1 of 2 cases, the Extension was pre-bundled, so it was definitely our bug.
 * Petr - Andreas is active changes on the conference site right now -- so might be punted for now
 * Joel - There might be some concerns (privacy of those dev emails, etc...)

PENDING ITEM: Certified QA Team
PUNTED: we want Florian R. on the call.
 * ACTION: Joren will create macOS-specific pages
 * ACTION: Joel will create Linux-specific pages
 * [DONE] ACTION: Florian R. will create Windows-specific pages
 * [DONE] ACTION: Joel will ping marketing about new site


 * [Ed. Change ITEM -> "QA Website"]
 * Florian R. - Needs pics for Robinson, Petr;
 * Also needs bios
 * Bjoern: just copy paste from https://www.documentfoundation.org/foundation/board/ for starters
 * Joel - Thinks we should just go live with what we have (placeholders if necessary)
 * ACTION: For everyone - if you want a pic and/or bio on the site, please send that info to Florian R.
 * Joel mentions that he'll get his Action Item (Linux pages) done by next meeting
 * Joel - Re: Marketing, Italo said that he'd write something up for us
 * ACTION: Florian R. will update Windows-specific pages

PENDING ITEM: Special Meeting

 * [DONE] ACTION: Joel will ask on the mailing list to see if we want to have a special meeting (perhaps in 1 week) to address the rest of our action items
 * May 13 - Given that we have a meeting at week's-end, we're probably not going to have a special meeting anymore...


 * Didn't have a special meeting; we'll probably get through every item today anyhow
 * Joel - If we don't have 3+ people available for a special meeting, probably should wait

PENDING ITEM: Should we activate voting on FDO?
PUNTED: To keep meeting to 1hr
 * ACTION: Bjoern will keep shepherding the implementation of this feature


 * Bjoern - We're waiting on Tollef here

PENDING ITEM: Add BOLD statement to each FDO mail sent out to not reply via mail
PUNTED: To keep meeting to 1hr
 * ACTION: Bjoern will keep working on this feature


 * Joel: Waiting for Tollef here

PENDING ITEM: Update the whiteboard/keywords page
PUNTED: To keep meeting to 1hr
 * ACTION: Joel will update the whiteboard/keywords page


 * Joel - hasn't done it yet; working on reorganzing info from QA/BSA/BugReport Details
 * Pages under QA/Bugzilla/Fields
 * Added some OS-specific whiteboard statuses (mostly Windows for now; KDE, etc.. might be useful in the future)
 * Also added Need Advice status
 * Joel wants to use that really sparingly
 * Only use if you have all of the information on the bug, and the Users' list and IRC aren't able to help, then use this status

=== PENDING ITEM: How should the BugReport and Bug Documentation wiki pages be organized? === PUNTED: To keep meeting to 1hr
 * [ABANDONED] ACTION: Robinson will organize a separate call when Rainer, Rob, and other interested parties can figure out a plan here.
 * May 1 - Robinson refactored some of QA/FAQ, BugReport Details, and other pages under QA/Bugzilla/. Work is progressing well; more updates to come.
 * May 13 - Robinson and Joel are making and/or planning more changes (e.g. Whiteboard statuses, Keywords, etc..)


 * Robinson - We're doing well; don't think we need a call right now. I'll continue working with Joel

=== PENDING ITEM: Use an online collaborative editor for minutes so that they can be viewed as the call progresses === PUNTED: To keep meeting to 1hr
 * [DONE] ACTION: Joel will update the topic in the IRC channel for the next meeting to say "In meeting -- join us in Pad XXX" (etc..)
 * ACTION: Robinson will ask Floeff to install Etherpad on a TDF server somewhere.
 * April 19 - Emailed the website list and asked if installation would be possible (in the future).


 * Etherpad working well
 * Might ask floeff again if we can install it (will test on Jonathan A's virtual server first)

PENDING ITEM: SI-GUI
PUNTED: To keep meeting to 1hr
 * [DONE] ACTION: Florian R. -- Update Wikipage


 * Florian R. - It's working well on the TDF server

PENDING ITEM: Windows QA guys
PUNTED: To keep meeting to 1hr
 * [DONE] ACTION: Joel will go to the user's list and recruit some more Windows users for our team (esp. Win8 users)

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * Joel installed win7
 * We also have Florian R. on windows 7 (Could install XP, 8 @ school laptop)
 * For now, send Windows8-related issues to Florian R.

PENDING ITEM: Q&A for QA
PUNTED: To keep meeting to 1hr
 * [DONE] ACTION: Florian R. will lead discussion on proposed call section: Q&A for new triagers

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * Florian R. - We have a "getting started" section, but ... need more?

PENDING ITEM: What to do with releases after EOL
PUNTED: To keep meeting to 1hr
 * ACTION: Robinson will ping the QA list and specifically ask if Petr or Rainer have comments.
 * Here's what we roughly agreed to at the meeting:
 * AGREED: For users inquiring about tech support after the EOL date of a release, we politely indicate that the release is EOL and ask them to upgrade to a new version (or go talk to their vendor/distro/paid tech support)
 * AGREED: We de-list versions in Bugzilla 6 months after the release has been EOLed.
 * April 19 - Robinson emailed QA list soliciting feedback.
 * Some resistance to de-listing old versions (per request from anonymous devs)
 * Less controversial: de-list EOL builds from the BSA, and de-list alpha/beta builds from FDO once a release has shipped

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * PUNT: It's a big one to discuss

PENDING ITEM: OS Specific Whiteboard Status
PUNTED: To keep meeting to 1hr
 * [DONE] ACTION: Joel will update the whiteboard page with new status for Win8

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * Joel - Added to the whiteboard page

PENDING ITEM: How to deal with Unconfirmed bugs that QA team cannot triage
PUNTED: To keep meeting to 1hr
 * [DONE] ACTION: Joel will add a whiteboard status 'NEED EXPERT ADVICE'
 * [DONE?] ACTION: Bjoern will talk to michael about MSDN-related info

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * We have the Need_Advice whiteboard status

PENDING ITEM: Locally-hosted Bugzilla
PUNTED: To keep meeting to 1hr
 * [SUPERSEDED] ACTION: Robinson will start to investigate what steps would need to happen to migrate each integrated piece for Bugzilla
 * Per Joel, Tollef is interested in helping us with this process.

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * ACTION: Robinson - Keep this on our agenda so we keep track of things

PENDING ITEM: Regressions within a release
PUNTED: To keep meeting to 1hr
 * [DONE] ACTION: Robinson will talk w/Joel about a game plan to tackle these regressions
 * April 19 - Robinson emailed QA list to discuss strategy for regressions
 * April 21 - Bjoern provided Bugzilla searches and other info:

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace"> ACTION: Joel will email a spreadsheet to Michael/Bjoern so they can take a look at the results from our testing.
 * Only about half of the bugs were regressions within a release
 * However, this was very time-consuming (Joel spent a few hours testing bugs that were, for the most part, already fixed)

PENDING ITEM: FDO/Twitter Feed(s)
This is an old Action Item that is now done...
 * [DONE] ACTION: Get Twitter Feed(s) running showing FDO activity (not sure of exact Action Item language)

<div  style="background-color:#DFFFFF; border-style: dashed; border-width:   1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida   Console,Liberation Mono,Courier New, monospace"> Bjoern reports that there are now two Twitter channels thanks to Zekis: Zekis' original mail: http://nabble.documentfoundation.org/Ask-LibreOffice-org-and-Bugzilla-Twitter-accounts-td4056631.html
 * LibreOffice bugzilla activity on Bugzilla: https://twitter.com/LibreOfficeBugs and
 * Ask LibreOffice activity: https://twitter.com/AskLibreOffice

New Action Items
All items proposed between meetings go here

NEW ITEM: Clarify 'version' in Bugzilla (Robinson)
PUNTED: To keep meeting to 1hr

We use the 'Version' field in Bugzilla to indicate the earliest build in which we can reproduce a given bug. The field is currently labeled 'Version,' causing confusion to some of our users -- e.g. some users will update the version if they can reproduce the bug in the latest build.

I suggest that we change the label on this field to "Earliest version..." (with the ellipsis), and add a tooltip "Earliest version in which this bug appears." We might want to make a change in the BSA as well?


 * April 22 - Joel emailed Tollef and asked if we could implement the change in FDO
 * The BSA currently says: "Version the bug appeared:". Is that good enough?

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * Might not be possible? (shared bugzilla with other projects)
 * ACTION: Keep track of where Tollef is with this enhancement

NEW ITEM: Clarify Bibisect + Version # (Joel)
PUNTED: To keep meeting to 1hr

Currently there are different ways in which we are using bibisct - some are using it and changing version # if the bug exists in 3.6alpha (earliest bibisect), others are using it only sporadically when a bug says regression but not updating version.
 * We should get on the same page, if a bug exists in earliest bibisect... do we update version? If so to what?
 * If we are going to, should every bug essentially be triaged with bibisect if possible?
 * What happens if a bug was around in earliest version but then fixed at some point and then rebroken?

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace"> PUNT - Joel had to leave, so we'll punt until next time.

NEW ITEM: Provide table of (no)repro tests on bugs (Robinson)
PUNTED: To keep meeting to 1hr

On a related note to Joel's Item above regarding Bibisect and version #, I propose that we consider the creation of a table of repro results for testing the bug on different versions of LO on different OSes.
 * April 23 - Robinson, Joel chat about some ideas
 * April 24 - Robinson & Joren chat about new ideas and plan for a table for visualization
 * Robinson's (ugly) mockup:

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * Robinson - Maybe if we can just change the label of the version field (might need locally-hosted Bugzilla?), then that will go a long way towards helping
 * Bjoern - we can kind of keep track of information via the 'Target' field (to keep track of when it is fixed...or ostensibly fixed)
 * ACTION: Keep this on our radar until the Version label is changed or we have another solution.
 * [Ed. TODO - Combine this action item with the "version-in-Bugzilla" Item]

New Items (Proposed/Discussed at the Meeting)
<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">

NEW ITEM: IRC IZBot (Joel)

 * Isn't linking bugs pasted in the channel...??
 * ACTION: Joel will investigate why our IRC bot isn't linking to FDO properly anymore

NEW ITEM: Google+ Events for meetings (Florian R.)

 * Florian R. proposes that we link to a Google+ event for the next meeting in the minutes
 * ACTION: Florian R. will do this --> Google+ Event: May 31st QA Meeting

NEW ITEM: Bibisect Repositories (Florian R.)

 * Florian R. - Can we download the current bibisect repo from Canonical and then keep it up to date using git (git fetch, git pull, etc..)?
 * Bjoern - You can use a download tool [Ed. --> rsync?] to update the repo, but can't use git, currently
 * The ability to use git to update the repository would be very helpful
 * ACTION: Bjoern/Robinson will investigate the possibility of being able to update the bibisect repository via git

NEW ITEM: HardHacks and Storing encrypted documents in plaintext (Joel, Florian R.)
''[Ed. We got off-topic while discussing ; these are the notes from our discussion]''


 * Joel - HardHacks is being discussed in ESC to determine if it's useful enough to continue or if other paths are more efficient
 * Petr - This may not be a good time to talk about HardHacks, as we're working on feature freeze for 4.1 and are very busy


 * Florian R. - Wanted to discuss what to do when developers & QA disagree about priority
 * In general (some) disagreement about the severity of a few bugs
 * In the end we have to accept that developers have ultimate say on their time, they are all very busy and need to think about what they think will benefit users the most.
 * If a particular bug looks nasty but isn't getting the attention we think it should, we should bring it to ESC and let them decide, if they disagree, private conversations can be held to try to reach some agreement.
 * QA is to guide development, but not to control any actions by developers, if developers don't want to fix a bug, or disagree with priority, we should cautiously accept this but respectful conversation (privately) is welcome.
 * [Ed. .....lots of discussion... PLEASE push this to a NEW ITEM next time! ]
 * ACTION: If there's something actionable we need to discuss re: Hard Hacks or this bug, please bring it up on the QA/Mailing List or put it on the Agenda for QA/Meetings/2013/May 31.

Announcements

 * ANNOUNCEMENT: Our next meeting will take place... QA/Meetings/2013/May 31st (Friday) at 13:00 UTC, unless otherwise noted on the QA/Mailing List.

<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">

End
<div style="background-color:#DFFFFF; border-style: dashed; border-width: 1px; padding: 10px; margin: 10px; font-family:Monaco,Lucida Console,Liberation Mono,Courier New, monospace">
 * Meeting adjourned at: 14:27

= Topics =

(Add topics below and reference them as in the Agenda/Minutes above)

Release Regressions
Notes from Bjoern:
 * this is a table showing the regressions in question (with version 3.6.x with x >= 1), probably a good idea to ignore all which are not unresolved or fixed => currently ~55 bugs to check
 * if these are already present in 3.6.0, mark them as version:3.6.0.4 release (they should then disappear from the table)
 * if these are indeed NOT in 3.6.0, it would be nice to hint devs at the issue
 * these seem to be fixed, but NOT fixed in 3.6 -- worth checking if they are all false positives => currently ~8 bugs to check
 * if such a bug is fixed in 3.6, mark the issue as target:3.6.X (they should then disappear from the table)
 * if such a bug is NOT fixed in 3.6, it would be nice to hint devs at the issue

Repro Table
Here's a quick mock-up of how the "Repro Table" could look:

Some ideas:
 * Builds will be listed chronologically from left to right
 * We can include results from bibisect runs as well
 * Individual runs will link to comments
 * More than 1 person can report a repro against a particular OS/Version combination

Questions:
 * What to do about 'mistakes' ?
 * What if 1 person can repro and another can't?

Here's a mockup of the table in Bugzilla:



Implementation
One way of implementing such a table would be to parse specially-formatted comments on bug reports. For instance, a user leaves a comment on a bug like this:

I tested this bug doing X and Y and Z and found interesting results (etc..)
 * 1) REPRO
 * 2) Ubuntu 12.04.2
 * 3) LO 3.4.5

We'd take that information, add it to the existing data about previous REPRO/NOREPRO instances, and re-generate the table for the bug report.

The syntax would be pretty flexible, so that all of these would be parsed to the same values:


 * 1) NOREPRO
 * 2) macOS 10.6.2
 * 3) LO 4.0.0.1


 * 1) LibreOffice 4.0.0.1
 * 2) Mac 10.6.2
 * 3) No Repro


 * 1) No Repro, LibreOffice 4.0.0 RC1, Mac 10.6.2