TDF LibreOffice Document Liberation Project Community Blogs Weblate Nextcloud Redmine Ask LibreOffice Donate

Meeting of the QA Team previous - next
Date and Time (UTC)	2013-05-17 13:00 UTC
Chair	Joel Madero
Secretary	Robinson Tryon
Started	13:08
Adjourned	14:27
Participants	Petr, Joel, Florian R., Bjoern, SteveBell, John Lehnov
Pad	http://beta.etherpad.org/p/partyhat

Prep

Agenda + Minutes

Minutes:
Content from the meeting itself ("the minutes") will be displayed in blue boxes like this.

Start

Meeting started at: 13:08
Participants: Petr, Joel, Florian R., Bjoern, SteveBell (only IRC/pad), John Lehnov (new member?)
Notes:
Didn't have a special meeting on May 10; punted items from May 3 will be dealt with at this meeting.
Joel had to leave around 13:12; Robinson lead for the remainder of the meeting

Opening Discussion

(none)

Pending Items

PENDING ITEM: Proposals regarding our workflow

PROPOSAL: Joel: Don't create proposal and AGREE to it on the same day. Instead we should discuss, add a PROPOSAL (keyword in bold), and allow those who do not attend call to reply via email. If no negative feedback by next call, we can mark as AGREED and remove the Item from our PENDING ITEMS list.
PROPOSAL: Joel: Slow down on some of our proposals/work. E.g. on the creation of the qa.libreoffice.org
PROPOSAL: Joel: If we make a decision that takes a lot of time for other people, wait a long time; 3+ weeks to hold on for people
PROPOSAL: Joel: Don't make a rule about how to triage things; Just lead by example

Punt to later in the meeting
PUNTED: Until next meeting (ran out of time).

PENDING ITEM: NEEDINFO Stagnant Bugs

[DONE] ACTION: Joel will write a wiki page about the process for closing bugs that have been in stagnant NEEDINFO, including links to the discussion, rationale for doing it, etc.

Made wiki - QA/Bugzilla/Fields/Status/NEEDINFO
Text will be something like "Your bugs have been in NEEDINFO status for 180 days...click this link to see our policy. Please attach information within 30 days or your
bug will go to INVALID status after 30 days."
ACTION: Joel will do a test run today, then will email QA List (and core people) and ask for feedback.
If all is good, Joel will try to ramp up to 50+ a day

PENDING ITEM: Documentation for Localized French BSA

[DONE] ACTION: Joel will talk to Rob and will look for a 2nd developer for BSA code
[DONE] ACTION: Joel will hand over developers to Rob (as the BSA is Rob's baby)

Floeff says that the French BSA is operational (as far as he knows); we still need to ping Rob and wrap this up.
Bjoern - Rob asked about Hamburg Hackfest on IRC yesterday -- maybe find a contributor there (and also do some knowledge transfer)

PENDING ITEM: Get French BSA Operational

ACTION: Joel will continue to shepherd the project; Will coordinate with Rob

[Ed. TODO -> Merge with previous PENDING ITEM]
Continue to work on BSA

PENDING ITEM: What to do with FDO bugs filed against Extensions, Templates

ACTION: Joel will (try to) contact the maintainer of the LO Extension Site again
[DONE] ACTION: Joel will bring this up to the ESC -- we need to tell users up front that Extensions are not supported by us (especially when upgrading)

Thorsten contacted Andreas...
We need a way for users to contact extension/template developers
Joel - For now, re: Extension bugs, he's been cc'ing Mmeeks
In 1 of 2 cases, the Extension was pre-bundled, so it was definitely our bug.
Petr - Andreas is active changes on the conference site right now -- so might  be punted for now
Joel - There might be some concerns (privacy of those dev emails, etc...)

PENDING ITEM: Certified QA Team

PUNTED: we want Florian R. on the call.

ACTION: Joren will create macOS-specific pages
ACTION: Joel will create Linux-specific pages
[DONE] ACTION: Florian R. will create Windows-specific pages
[DONE] ACTION: Joel will ping marketing about new site

[Ed. Change ITEM -> "QA Website"]
Florian R. - Needs pics for Robinson, Petr;
Also needs bios
Bjoern: just copy paste from https://www.documentfoundation.org/foundation/board/ for starters
Joel - Thinks we should just go live with what we have (placeholders if necessary)
ACTION: For everyone - if you want a pic and/or bio on the site, please send that info to Florian R.
Joel mentions that he'll get his Action Item (Linux pages) done by next meeting
Joel - Re: Marketing, Italo said that he'd write something up for us
ACTION: Florian R. will update Windows-specific pages

PENDING ITEM: Special Meeting

[DONE] ACTION: Joel will ask on the mailing list to see if we want to have a special meeting (perhaps in 1 week) to address the rest of our action items
- May 13 - Given that we have a meeting at week's-end, we're probably not going to have a special meeting anymore...

Didn't have a special meeting; we'll probably get through every item today anyhow
Joel - If we don't have 3+ people available for a special meeting, probably should wait

PENDING ITEM: Should we activate voting on FDO?

PUNTED: To keep meeting to 1hr

ACTION: Bjoern will keep shepherding the implementation of this feature

Bjoern - We're waiting on Tollef here

PENDING ITEM: Add BOLD statement to each FDO mail sent out to not reply via mail

PUNTED: To keep meeting to 1hr

ACTION: Bjoern will keep working on this feature

Joel: Waiting for Tollef here

PENDING ITEM: Update the whiteboard/keywords page

PUNTED: To keep meeting to 1hr

ACTION: Joel will update the whiteboard/keywords page

Joel - hasn't done it yet; working on reorganzing info from QA/BSA/BugReport Details
Pages under QA/Bugzilla/Fields
Added some OS-specific whiteboard statuses (mostly Windows for now; KDE, etc.. might be useful in the future)
Also added Need Advice status
Joel wants to use that really sparingly
Only use if you have all of the information on the bug, and the Users' list and IRC aren't able to help, then use this status

PENDING ITEM: How should the BugReport and Bug Documentation wiki pages be organized?

PUNTED: To keep meeting to 1hr

[ABANDONED] ACTION: Robinson will organize a separate call when Rainer, Rob, and other interested parties can figure out a plan here.
May 1 - Robinson refactored some of QA/FAQ, BugReport Details, and other pages under QA/Bugzilla/. Work is progressing well; more updates to come.
May 13 - Robinson and Joel are making and/or planning more changes (e.g. Whiteboard statuses, Keywords, etc..)

Robinson - We're doing well; don't think we need a call right now. I'll continue working with Joel

PENDING ITEM: Use an online collaborative editor for minutes so that they can be viewed as the call progresses

PUNTED: To keep meeting to 1hr

[DONE] ACTION: Joel will update the topic in the IRC channel for the next meeting to say "In meeting -- join us in Pad XXX" (etc..)
ACTION: Robinson will ask Floeff to install Etherpad on a TDF server somewhere.
- April 19 - Emailed the website list and asked if installation would be possible (in the future).

Etherpad working well
Might ask floeff again if we can install it (will test on Jonathan A's virtual server first)

PENDING ITEM: SI-GUI

PUNTED: To keep meeting to 1hr

[DONE] ACTION: Florian R. -- Update Wikipage

Florian R. - It's working well on the TDF server

PENDING ITEM: Windows QA guys

PUNTED: To keep meeting to 1hr

[DONE] ACTION: Joel will go to the user's list and recruit some more Windows users for our team (esp. Win8 users)

Joel installed win7
We also have Florian R. on windows 7 (Could install XP, 8 @ school laptop)
For now, send Windows8-related issues to Florian R.

PENDING ITEM: Q&A for QA

PUNTED: To keep meeting to 1hr

[DONE] ACTION: Florian R. will lead discussion on proposed call section: Q&A for new triagers

Florian R. - We have a "getting started" section, but ... need more?

PENDING ITEM: What to do with releases after EOL

PUNTED: To keep meeting to 1hr

ACTION: Robinson will ping the QA list and specifically ask if Petr or Rainer have comments.
- Here's what we roughly agreed to at the meeting:
  - AGREED: For users inquiring about tech support after the EOL date of a release, we politely indicate that the release is EOL and ask them to upgrade to a new version (or go talk to their vendor/distro/paid tech support)
  - AGREED: We de-list versions in Bugzilla 6 months after the release has been EOLed.
- April 19 - Robinson emailed QA list soliciting feedback.
- Some resistance to de-listing old versions (per request from anonymous devs)
  - Less controversial: de-list EOL builds from the BSA, and de-list alpha/beta builds from FDO once a release has shipped

PUNT: It's a big one to discuss

PENDING ITEM: OS Specific Whiteboard Status

PUNTED: To keep meeting to 1hr

[DONE] ACTION: Joel will update the whiteboard page with new status for Win8

Joel - Added to the whiteboard page

PENDING ITEM: How to deal with Unconfirmed bugs that QA team cannot triage

PUNTED: To keep meeting to 1hr

[DONE] ACTION: Joel will add a whiteboard status 'NEED EXPERT ADVICE'
[DONE?] ACTION: Bjoern will talk to michael about MSDN-related info

We have the Need_Advice whiteboard status

PENDING ITEM: Locally-hosted Bugzilla

PUNTED: To keep meeting to 1hr

[SUPERSEDED] ACTION: Robinson will start to investigate what steps would need to happen to migrate each integrated piece for Bugzilla
Per Joel, Tollef is interested in helping us with this process.

ACTION: Robinson - Keep this on our agenda so we keep track of things

PENDING ITEM: Regressions within a release

PUNTED: To keep meeting to 1hr

[DONE] ACTION: Robinson will talk w/Joel about a game plan to tackle these regressions
- April 19 - Robinson emailed QA list to discuss strategy for regressions
- April 21 - Bjoern provided Bugzilla searches and other info: #Release Regressions

ACTION: Joel will email a spreadsheet to Michael/Bjoern so they can take a look at the results from our testing.
Only about half of the bugs were regressions within a release
However, this was very time-consuming (Joel spent a few hours testing bugs that were, for the most part, already fixed)

PENDING ITEM: FDO/Twitter Feed(s)

This is an old Action Item that is now done...

[DONE] ACTION: Get Twitter Feed(s) running showing FDO activity (not sure of exact Action Item language)

Bjoern reports that there are now two Twitter channels thanks to Zekis:
LibreOffice bugzilla activity on Bugzilla: https://twitter.com/LibreOfficeBugs and
Ask LibreOffice activity: https://twitter.com/AskLibreOffice
Zekis' original mail: https://lists.freedesktop.org/archives/libreoffice-qa/2013-May/004587.html

New Action Items

All items proposed between meetings go here

NEW ITEM: Clarify 'version' in Bugzilla (Robinson)

PUNTED: To keep meeting to 1hr

We use the 'Version' field in Bugzilla to indicate the earliest build in which we can reproduce a given bug. The field is currently labeled 'Version,' causing confusion to some of our users -- e.g. some users will update the version if they can reproduce the bug in the latest build.

I suggest that we change the label on this field to "Earliest version..." (with the ellipsis), and add a tooltip "Earliest version in which this bug appears." We might want to make a change in the BSA as well?

April 22 - Joel emailed Tollef and asked if we could implement the change in FDO
The BSA currently says: "Version the bug appeared:". Is that good enough?

Might not be possible?  (shared bugzilla with other projects)
ACTION: Keep track of where Tollef is with this enhancement

NEW ITEM: Clarify Bibisect + Version # (Joel)

PUNTED: To keep meeting to 1hr

Currently there are different ways in which we are using bibisct - some are using it and changing version # if the bug exists in 3.6alpha (earliest bibisect), others are using it only sporadically when a bug says regression but not updating version.

We should get on the same page, if a bug exists in earliest bibisect... do we update version? If so to what?
If we are going to, should every bug essentially be triaged with bibisect if possible?
What happens if a bug was around in earliest version but then fixed at some point and then rebroken?

PUNT - Joel had to leave, so we'll punt until next time.

NEW ITEM: Provide table of (no)repro tests on bugs (Robinson)

PUNTED: To keep meeting to 1hr

On a related note to Joel's Item above regarding Bibisect and version #, I propose that we consider the creation of a table of repro results for testing the bug on different versions of LO on different OSes.

April 23 - Robinson, Joel chat about some ideas
April 24 - Robinson & Joren chat about new ideas and plan for a table for visualization
- Robinson's (ugly) mockup: #Repro Table

Robinson - Maybe if we can just change the label of the version field (might need locally-hosted Bugzilla?), then that will go a long way towards helping
Bjoern - we can kind of keep track of information via the 'Target' field (to keep track of when it is fixed...or ostensibly fixed)
ACTION: Keep this on our radar until the Version label is changed or we have another solution.
[Ed. TODO - Combine this action item with the "version-in-Bugzilla" Item]

New Items (Proposed/Discussed at the Meeting)

NEW ITEM: IRC IZBot  (Joel)Isn't linking bugs pasted in the channel...??
ACTION: Joel will investigate why our IRC bot isn't linking to FDO properly anymore
NEW ITEM: Google+ Events for meetings (Florian R.)Florian R. proposes that we link to a Google+ event for the next meeting in the minutes
ACTION: Florian R. will do this --> Google+ Event: May 31st QA Meeting
NEW ITEM: Bibisect Repositories (Florian R.)Florian R. - Can we download the current bibisect repo from Canonical and then  keep it up to date using git (git fetch, git pull, etc..)?
Bjoern - You can use a download tool [Ed. --> rsync?] to update the repo, but can't use git, currently
The ability to use git to update the repository would be very helpful
ACTION: Bjoern/Robinson will investigate the possibility of being able to update the bibisect repository via git
NEW ITEM: HardHacks and Storing encrypted documents in plaintext (Joel, Florian R.)[Ed. We got off-topic while discussing #PENDING ITEM: Update the whiteboard/keywords page; these are the notes from our discussion]
Joel - HardHacks is being discussed in ESC to determine if it's useful enough to continue or if other paths are more efficient
Petr - This may not be a good time to talk about HardHacks, as we're working on feature freeze for 4.1 and are very busy
Florian R. - Wanted to discuss what to do when developers & QA disagree about priority
In general (some) disagreement about the severity of a few bugs
In the end we have to accept that developers have ultimate say on their time, they are all very busy and need to think about what they think will benefit users the most.
If a particular bug looks nasty but isn't getting the attention we think it should, we should bring it to ESC and let them decide, if they disagree, private conversations can be held to try to reach some agreement.
QA is to guide development, but not to control any actions by developers, if developers don't want to fix a bug, or disagree with priority, we should cautiously accept this but respectful conversation (privately) is welcome.
[Ed. .....lots of discussion... PLEASE push this to a NEW ITEM next time! ]
ACTION: If there's something actionable we need to discuss re: Hard Hacks or this bug, please bring it up on the QA/Mailing List or put it on the Agenda for QA/Meetings/2013/May 31.

Announcements

ANNOUNCEMENT: Our next meeting will take place... QA/Meetings/2013/May 31st (Friday) at 13:00 UTC, unless otherwise noted on the QA/Mailing List.

End

Meeting adjourned at: 14:27

Topics

(Add topics below and reference them as #Example Topic in the Agenda/Minutes above)

Release Regressions

Notes from Bjoern:

this is a table showing the regressions in question (with version 3.6.x with x >= 1), probably a good idea to ignore all which are not unresolved or fixed => currently ~55 bugs to check
- if these are already present in 3.6.0, mark them as version:3.6.0.4 release (they should then disappear from the table)
- if these are indeed NOT in 3.6.0, it would be nice to hint devs at the issue
these seem to be fixed, but NOT fixed in 3.6 -- worth checking if they are all false positives => currently ~8 bugs to check
- if such a bug is fixed in 3.6, mark the issue as target:3.6.X (they should then disappear from the table)
- if such a bug is NOT fixed in 3.6, it would be nice to hint devs at the issue

Repro Table

Here's a quick mock-up of how the "Repro Table" could look:

	Tested releases in chronological order
	3.4.2 2011-08-07	Bibisected (b78226) 2012-07-09	4.0.0.1 2013-01-13	3.6.6 2013-04-14	Master (3591f5) 2013-04-24
macOS 10.4	Joren			😻
Ubuntu 12.04.2		Qubit			Qubit
Win8	Joren		Joel, Joren	Joren

Some ideas:

Builds will be listed chronologically from left to right
We can include results from bibisect runs as well
Individual runs will link to comments
More than 1 person can report a repro against a particular OS/Version combination

Questions:

What to do about 'mistakes' ?
What if 1 person can repro and another can't?

Here's a mockup of the table in Bugzilla:

Implementation

One way of implementing such a table would be to parse specially-formatted comments on bug reports. For instance, a user leaves a comment on a bug like this:

I tested this bug doing X and Y and Z and found interesting results (etc..)

## REPRO
## Ubuntu 12.04.2
## LO 3.4.5

We'd take that information, add it to the existing data about previous REPRO/NOREPRO instances, and re-generate the table for the bug report.

The syntax would be pretty flexible, so that all of these would be parsed to the same values:

## NOREPRO
## macOS 10.6.2
## LO 4.0.0.1

## LibreOffice 4.0.0.1
## Mac 10.6.2
## No Repro

## No Repro, LibreOffice 4.0.0 RC1, Mac 10.6.2