|Meeting of the QA Team|
|Date and Time (UTC)||2013-05-03 13:00 UTC|
|Participants||Joel, Robinson, Florian R., Petr|
- 1 Prep
- 2 Agenda + Minutes
- 2.1 Start
- 2.2 Opening Discussion
- 2.3 Pending Action Items
- 2.3.1 PENDING ITEM: NEEDINFO Stagnant Bugs
- 2.3.2 PENDING ITEM: Documentation for Localized French BSA
- 2.3.3 PENDING ITEM: Get French BSA Operational
- 2.3.4 PENDING ITEM: Contest Swag
- 2.3.5 PENDING ITEM: What to do with FDO bugs filed against Extensions, Templates
- 2.3.6 PENDING ITEM: Certified QA Team
- 2.3.7 Ending Meeting
- 2.3.8 PENDING ITEM: Should we activate voting on FDO?
- 2.3.9 PENDING ITEM: Add BOLD statement to each FDO mail sent out to not reply via mail
- 2.3.10 PENDING ITEM: Update the whiteboard/keywords page
- 2.3.11 PENDING ITEM: How should the BugReport and Bug Documentation wiki pages be organized?
- 2.3.12 PENDING ITEM: Use an online collaborative editor for minutes so that they can be viewed as the call progresses
- 2.3.13 PENDING ITEM: SI-GUI
- 2.3.14 PENDING ITEM: Windows QA guys
- 2.3.15 PENDING ITEM: Q&A for QA
- 2.3.16 PENDING ITEM: What to do with releases after EOL
- 2.3.17 PENDING ITEM: OS Specific Whiteboard Status
- 2.3.18 PENDING ITEM: How to deal with Unconfirmed bugs that QA team cannot triage
- 2.3.19 PENDING ITEM: Locally-hosted Bugzilla [was How to deal with Unconfirmed bugs...]
- 2.3.20 PENDING ITEM: Regressions within a release
- 2.4 New Action Items
- 2.5 New Action Items (Proposed at the Meeting)
- 2.6 Announcements
- 2.7 End
- 3 Topics
Agenda + Minutes
Due to recent events and some feedback from multiple people throughout the project, we held an initial discussion about how we hold meetings and operate as a team. The discussion focused on how to encourage participation and minimize fraction while ensuring that projects still move forward in a timely manner.
Pending Action Items
PENDING ITEM: NEEDINFO Stagnant Bugs
- [DONE] ACTION: Joel will do 1st test of 50 bugs this week
PENDING ITEM: Documentation for Localized French BSA
- [SUPERSEDED] ACTION: Robinson will add Charles Schultz as 2nd contact for FR BSA
- April 19 - Sophie asks on QA List to hold off on adding Charles; Wants FR QA Mailing List to be in charge of BSA, not individuals
- ACTION: Joel will talk to Rob and will look for a 2nd developer for BSA code
PENDING ITEM: Get French BSA Operational
- [DONE] ACTION: Joel will coordinate next steps of getting FR BSA running
- April 20 - Per Bjoern, Jean-Baptiste Faure volunteered as a second contact (along with Sophie) to the French community.
PENDING ITEM: Contest Swag
- [DONE] ACTION: Joel will try to find a way to have a variety of prizes; will let the Board know if we go with our Backup plan.
- [DONE] ACTION: Joel will try to get conversation started again
- ACTION: Joel will (try to) contact the maintainer of the LO Extension Site again
PENDING ITEM: Certified QA Team
- ACTION: Joren will create macOS-specific pages
- ACTION: Joel will create Linux-specific pages
- ACTION: Florian R. will create Windows-specific pages
- ACTION: Joel will ping marketing about new site
PENDING ITEM: Should we activate voting on FDO?
- ACTION: Bjoern will keep shepherding the implementation of this feature
- ACTION: Bjoern will keep working on this feature
PENDING ITEM: Update the whiteboard/keywords page
- ACTION: Joel will update the whiteboard/keywords page
- ACTION: Robinson will organize a separate call when Rainer, Rob, and other interested parties can figure out a plan here.
- May 1 - Robinson refactored some of QA/FAQ, BugReport Details, and other pages under QA/Bugzilla/. Work is progressing well; more updates to come.
PENDING ITEM: Use an online collaborative editor for minutes so that they can be viewed as the call progresses
- ACTION: Joel will update the topic in the IRC channel for the next meeting to say "In meeting -- join us in Pad XXX" (etc..)
- ACTION: Robinson will ask Floeff to install Etherpad on a TDF server somewhere.
- April 19 - Emailed the website list and asked if installation would be possible (in the future).
PENDING ITEM: SI-GUI
- ACTION: Florian R. -- Update Wikipage
PENDING ITEM: Windows QA guys
- ACTION: Joel will go to the user's list and recruit some more Windows users for our team (esp. Win8 users)
PENDING ITEM: Q&A for QA
- ACTION: Florian R. will lead discussion on proposed call section: Q&A for new triagers
PENDING ITEM: What to do with releases after EOL
- ACTION: Robinson will ping the QA list and specifically ask if Petr or Rainer have comments.
- Here's what we roughly agreed to at the meeting:
- AGREED: For users inquiring about tech support after the EOL date of a release, we politely indicate that the release is EOL and ask them to upgrade to a new version (or go talk to their vendor/distro/paid tech support)
- AGREED: We de-list versions in Bugzilla 6 months after the release has been EOLed.
- April 19 - Robinson emailed QA list soliciting feedback.
- Some resistance to de-listing old versions (per request from anonymous devs)
- Less controversial: de-list EOL builds from the BSA, and de-list alpha/beta builds from FDO once a release has shipped
- Here's what we roughly agreed to at the meeting:
PENDING ITEM: OS Specific Whiteboard Status
- ACTION: Joel will update the whiteboard page with new status for Win8
- ACTION: Joel will add a whiteboard status 'NEED EXPERT ADVICE'
- ACTION: Bjoern will talk to michael about MSDN-related info
PENDING ITEM: Locally-hosted Bugzilla [was How to deal with Unconfirmed bugs...]
- ACTION: Robinson will start to investigate what steps would need to happen to migrate each integrated piece for Bugzilla
- Per Joel, Tollef is interested in helping us with this process.
PENDING ITEM: Regressions within a release
- ACTION: Robinson will talk w/Joel about a game plan to tackle these regressions
- April 19 - Robinson emailed QA list to discuss strategy for regressions
- April 21 - Bjoern provided Bugzilla searches and other info: #Release Regressions
New Action Items
All items proposed between meetings go here
NEW ITEM: Clarify 'version' in Bugzilla (Robinson)
We use the 'Version' field in Bugzilla to indicate the earliest build in which we can reproduce a given bug. The field is currently labeled 'Version,' causing confusion to some of our users -- e.g. some users will update the version if they can reproduce the bug in the latest build.
I suggest that we change the label on this field to "Earliest version..." (with the ellipsis), and add a tooltip "Earliest version in which this bug appears." We might want to make a change in the BSA as well?
- April 22 - Joel emailed Tollef and asked if we could implement the change in FDO
- The BSA currently says: "Version the bug appeared:". Is that good enough?
NEW ITEM: Clarify Bibisect + Version # (Joel)
Currently there are different ways in which we are using bibisct - some are using it and changing version # if the bug exists in 3.6alpha (earliest bibisect), others are using it only sporadically when a bug says regression but not updating version.
- We should get on the same page, if a bug exists in earliest bibisect... do we update version? If so to what?
- If we are going to, should every bug essentially be triaged with bibisect if possible?
- What happens if a bug was around in earliest version but then fixed at some point and then rebroken?
NEW ITEM: Provide table of (no)repro tests on bugs (Robinson)
On a related note to Joel's Item above regarding Bibisect and version #, I propose that we consider the creation of a table of repro results for testing the bug on different versions of LO on different OSes.
- April 23 - Robinson, Joel chat about some ideas
- April 24 - Robinson & Joren chat about new ideas and plan for a table for visualization
- Robinson's (ugly) mockup: #Repro Table
New Action Items (Proposed at the Meeting)
- ANNOUNCEMENT: Our next meeting will take place... QA/Meetings/2013/May 17th (Friday) at 13:00 UTC, unless otherwise noted on the QA/Mailing List.
(Add topics below and reference them as #Example Topic in the Agenda/Minutes above)
Notes from Bjoern:
- this is a table showing the regressions in question (with version 3.6.x with x >= 1), probably a good idea to ignore all which are not unresolved or fixed => currently ~55 bugs to check
- if these are already present in 3.6.0, mark them as version:220.127.116.11 release (they should then disappear from the table)
- if these are indeed NOT in 3.6.0, it would be nice to hint devs at the issue
- these seem to be fixed, but NOT fixed in 3.6 -- worth checking if they are all false positives => currently ~8 bugs to check
- if such a bug is fixed in 3.6, mark the issue as target:3.6.X (they should then disappear from the table)
- if such a bug is NOT fixed in 3.6, it would be nice to hint devs at the issue
Here's a quick mock-up of how the "Repro Table" could look:
Tested releases in chronological order
- Builds will be listed chronologically from left to right
- We can include results from bibisect runs as well
- Individual runs will link to comments
- More than 1 person can report a repro against a particular OS/Version combination
- What to do about 'mistakes' ?
- What if 1 person can repro and another can't?
Here's a mockup of the table in Bugzilla:
One way of implementing such a table would be to parse specially-formatted comments on bug reports. For instance, a user leaves a comment on a bug like this:
I tested this bug doing X and Y and Z and found interesting results (etc..) ## REPRO ## Ubuntu 12.04.2 ## LO 3.4.5
We'd take that information, add it to the existing data about previous REPRO/NOREPRO instances, and re-generate the table for the bug report.
The syntax would be pretty flexible, so that all of these would be parsed to the same values:
## NOREPRO ## macOS 10.6.2 ## LO 18.104.22.168
## LibreOffice 22.214.171.124 ## Mac 10.6.2 ## No Repro
## No Repro, LibreOffice 4.0.0 RC1, Mac 10.6.2