QA/Bugzilla/Gardening

From The Document Foundation Wiki
< QA‎ | Bugzilla
Jump to: navigation, search

This page is about the Gardening of Bugzilla, including periodic maintenance, small cleanups, and automated tools. See QA/Easy_Hacks for QA tasks with a broader scope.

Introduction

Note: Gardening will be increasingly relevant after we migrate Bugzilla to TDF infrastructure.

Like thatching a lawn or pruning an orchard, it's important to do some periodic maintenance on our Bugzilla install lest it become overgrown, tangled, and unusable by QA or developers. Gardening includes a number of different areas:

  • Periodic maintenance
  • Cleaning up Fields
  • (and more)

Gardening and Bug Triage complement each other. The more that our QA/Team learns about triage and gardening of Bugzilla, the more effectively we can stay on top of all of the reports in the bug tracker.

Tasks

See the Table of Contents for a list of Tasks. If we have any Tasks that live on other wiki pages, we'll list them here:

  • nothing yet

Automation

It's hoped that in the future some tasks may be able to be automated (e.g. fixing commas in the whiteboard). Any suggestions on this front are welcomed!

Bugzilla Administration

There may be some task overlap with Bugzilla Administration, but many tasks are outside the scope of Gardening:

  • Dealing with user account creation/deletion
  • SPAM
  • Harassment
  • Deleting Attachments (only happens in rare cases)

Task: Remove commas from the whiteboard

Frequency Ideally, at least once a month
Purpose Tags in the Whiteboard are space-delimited. Commas don't belong in this field.
Last completed Buovjaga (talk) 2017-02-11T19:00:00 (UTC)
  1. Find bugs with a comma in the Whiteboard
  2. Remove the commas and leave a comment with this message:
Removing comma from Whiteboard (please use a space to delimit values in this field)
https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Whiteboard#Getting_Started

Task: Use proper markup in the Whiteboard

Frequency Whenever
Purpose Tags in the Whiteboard should follow our Markup Guidelines. We can update tags to harmonize our usage.
Last run

(we're never going to 'complete' this one)

2014

Generally speaking, we want people to follow our Markup Guidelines, including

  • Separating tags with spaces
  • Using "wimpyCaps"
  • Not writing sentences in the Whiteboard!

Running

  • Query for LibreOffice bugs
  • Bugs usually need to be updated one at a time
  • Some tags are ALL CAPS (e.g. the BSA)
  • Note that many searches are case-insensitive, so searching for "bsa" in the Whiteboard (even in a regexp) seems to return hits for "BSA" and "BsA" as well.

Task: Monitor REOPENED bugs

Frequency ~ Every 3 months
Purpose The REOPENED status is reserved for bugs that were thought to be FIXED. It is sometimes used mistakenly in other contexts.
Last completed Buovjaga (talk) and Yousuf Philipz (talk) 2016-10-21T15:43:27 (UTC)

For more details, see QA/Bugzilla/Fields/Status/REOPENED

In Bugzilla, Status: REOPENED has the technical meaning that a bug is:

  1. Assigned
  2. The assigned developer said they fixed it
  3. Someone has reproduced the problem on a version which, according to the developer, the problem should not appear.

Running

  1. Query for all bugs with REOPENED status
    • (It's a short list -- only 27 as of 2014-12-30)
    • TODO: Each time you run the query, edit it and update the status-last-changed-date (at the bottom of the page) to be 1 day before the Last Completed date. That way you won't review any REOPENED bugs that the last reviewer already verified as legitimate.
  2. Review each bug manually and determine if the bug meets the criteria specified above.
    • If you have time, try to reproduce the bug to see if it can be RESOLVED WORKSFORME
  3. If a bug doesn't meet the criteria for REOPENED, change the status appropriately. Feel free to leave a comment such as:
Bug does not meet the criteria for Status 'REOPENED'
https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Status/REOPENED#Criteria
Status -> (whatever status is appropriate)

Task: Cleanup Backport Requests

Frequency ~ Every 1 month
Purpose Bugs with Whiteboard tag backportRequest:X.y need some TLC to make sure they have the right Status, Whiteboard and Keyword tags, etc..
Last completed X1sc0 (talk) 2016-11-30T09:42:13 (UTC)

For more information, refer to QA/Bugzilla/Fields/Whiteboard/Advanced and QA/Bugzilla/Fields/Status/RESOLVED#Fixed_in_a_newer_version

Running

QA/Bugzilla/Useful_Queries#Backport_Requests gives two lists:

  • One for bugs that need to be reverse-bibisected
  • One for bugs that are ready for devs to address (reverse-(bi)bisected or otherwise have a commit identified)

On both lists, all bugs should have status RESOLVED or CLOSED

  • For each bug, confirm (via testing or reading comments) that the bug is fixed in the latest version or master branch
  • Follow guidelines here to set the Status and Whiteboard tags
  • If a bug has been fixed in branch X.y (in a release, beta, or daily build), then any backportRequest:X.y tag should be removed from said bug

Task: bibisected, bisected or bibisectRequest keywords not tagged with regression

Frequency ~ Every 3 months
Purpose Bugs with keyword bibisected, bisected or bibisectRequest need to be evaluated for regression.
Last completed X1sc0 (talk) 2016-11-12T13:40:24 (UTC)

For more information, refer to QA/Bugzilla/Fields/Keywords.

Running

  1. Find all bugs with keyword bibisected, bisected or bibisectRequest but without regression Keywords: Query
  2. Check each bug report manually to determine if it is regression ( important to check the history to see if someone remove the keyword in the past )
    • If it is, add Keyword regression

Task: Regressions not tagged with (bi)bisect

Frequency ~ Every 3 months
Purpose Bugs with keyword regression need to be evaluated for (bi)bisecting. Even if they can't be bibisected, they need to be marked with Keyword preBibisect.
Last completed X1sc0 (talk) 2016-09-20T16:13:09 (UTC)

For more information, refer to QA/Bugzilla/Fields/Keywords and QA/HowToBibisect.

Running

  1. Find all bugs with keyword regression but without any 'bibisect*' Keywords ( Exclude bugs that have been bisected too):
  2. Check each bug report manually to determine if it is bibisectable
    • If it is, add Keyword bibisectRequest
    • Otherwise, if the bug is outside the bibisect range, add Keyword preBibisect

Task: Bugs untouched for a year

Frequency At least every couple of months

(more frequent runs aren't bad, as any given bug will only get a new update every 12 months)

Purpose Ask reporters/testers for help confirming that old bugs still affect current builds.
Last completed X1sc0 (talk) 2017-05-22T13:38:52 (UTC)

Running the task

  1. Search for bug reports with no update in 1 year: Query
    • Check through the results and make sure everything looks kosher
    • General consensus is to limit to:
      • 1+ years no activity
      • Exclude: Any [Meta], [Task] and [Super] bugs, EasyHacks, ux-advise bugs (with keyword needsUXEval) and enhancement requests
  2. Then perform the bulk-update, leaving a comment

It would be a good idea to use the QA Administrators account for this update.

Notes

Bugs untouched for a year: Stock Message

** Please read this message in its entirety before responding **

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present on a currently supported version of LibreOffice 
(5.2.7 or 5.3.3  https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the version of LibreOffice and 
your operating system, and any changes you see in the bug behavior
 
If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave 
a short comment that includes your version of LibreOffice and Operating System

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not 
appropriate in this case)


If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3)

http://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to "inherited from OOo";
4b. If the bug was not present in 3.3 - add "regression" to keyword


Feel free to come ask questions or to say hello in our QA chat: http://webchat.freenode.net/?channels=libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug-20170522

Task: Cleaning-out NEEDINFO

Frequency Every month
Purpose Bugs shouldn't sit in NEEDINFO for ever. If they are untouched for 6 months, we will plan to close them (and then close them, a month later) as INSUFFICIENTDATA.
Last completed - Stock Message X1sc0 (talk) 2017-07-27T12:07:02 (UTC)
Last completed - Stock Follow-up Message X1sc0 (talk) 2017-07-27T12:07:02 (UTC)

Running

More background information: QA/Bugzilla/Fields/Status/NEEDINFO

  • NOTE: Running this task takes time, and the memory to return 30 days later to do a follow-up bulk update. Ask for help before undertaking this task!
  • We often use the QA Administrators Bugzilla account for performing these operations.

Steps:

  1. Find NEEDINFO bugs that haven't changed in 180 days
    • Pick a few bugs and take a quick look to make sure that everything looks kosher
    • Exclude: EasyHacks
  2. Update the ID to match the day the action is done at the end of the message
  3. Perform the bulk-update:
    • Point users to the NEEDINFO page
    • Request information again
    • Let user know that if requested information is already there, to appropriately set the status of the bug so we move the bug forward

Cleaning-out NEEDINFO: Stock Message

Dear Bug Submitter,

This bug has been in NEEDINFO status with no change for at least
6 months. Please provide the requested information as soon as
possible and mark the bug as UNCONFIRMED. Due to regular bug
tracker maintenance, if the bug is still in NEEDINFO status with
no change in 30 days the QA team will close the bug as INSUFFICIENTDATA
due to lack of needed information.

For more information about our NEEDINFO policy please read the
wiki located here:
https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Status/NEEDINFO

If you have already provided the requested information, please
mark the bug as UNCONFIRMED so that the QA team knows that the
bug is ready to be confirmed.
 
Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-NeedInfo-Ping-20170727

Cleaning-out NEEDINFO: 30 days later

30 Days after the first notification:

  1. Query the same list of bugs by searching for bugs touched by qa-admin account on the day that it was run.
    • This might not be perfect, but it's close enough for government work :-)
  2. Update the ID to match the day the action is done at the end of the message
  3. Bulk-update:
    • Close the bug as INSUFFICIENTDATA because of lack of information
    • Let user know that if they are still experiencing the bug they can open it back up as UNCONFIRMED if they provide requested information

Cleaning-out NEEDINFO: Stock Follow-up Message

Dear Bug Submitter,

Please read this message in its entirety before proceeding.

Your bug report is being closed as INSUFFICIENTDATA due to inactivity and
a lack of information which is needed in order to accurately
reproduce and confirm the problem. We encourage you to retest
your bug against the latest release. If the issue is still
present in the latest stable release, we need the following
information (please ignore any that you've already provided):

a) Provide details of your system including your operating
   system and the latest version of LibreOffice that you have
   confirmed the bug to be present

b) Provide easy to reproduce steps – the simpler the better

c) Provide any test case(s) which will help us confirm the problem

d) Provide screenshots of the problem if you think it might help

e) Read all comments and provide any requested information

Once all of this is done, please set the bug back to UNCONFIRMED
and we will attempt to reproduce the issue. Please do not:

a) respond via email 

b) update the version field in the bug or any of the other details
   on the top section of our bug tracker

Warm Regards,
QA Team

MassPing-NeedInfo-20170727

Task: Assigned reports with no movement

Frequency Every three months.
Purpose Abandoned assigned reports should be freed for other developers to work on.
Last completed Never

Running the task

  1. Search for open bug reports that are assigned to someone and had the assignee change more than three months ago: [1]
  2. You can immediately reset the assignee and correct the status, if
    • you know the assigned developer is retired from the project
    • it is a case of a regular user mistakenly assigning the report to themselves
  3. If the assignee is an active developer,
    • you can ping them on IRC about the report (to minimize bug spam)
    • you can leave a comment asking, if they still want to keep the report assigned to themselves


Task: Clean deletionrequest component

Frequency Every 3 months.
Purpose Delete bugs inside deletionrequest component
Last completed X1sc0 (talk) 2017-08-01T08:49:54 (UTC)

Running the task

  1. Enable allowbugdeletion in Administration -> Parameters -> Administrative Policies.
  2. Once you have added all the soon to be deleted bugs to your trash component go back to the component list in the administration page (Administration -> Products -> LibreOffice -> Edit Components). You should see an “Action” column on the far right of the table with all your components with a Delete link. Find the deletionrequest component and click the associated “Delete” link.
  3. Confirm the deletion.
  4. Disable allowbugdeletion in Administration -> Parameters -> Administrative Policies

More information here