QA/How to Improve Bugzilla

From The Document Foundation Wiki
< QA
Jump to: navigation, search
Audio-cassette.png This page (may) contain content that is old and/or out of date.

If you have experience in this area, please review this page for accuracy.

For quite a while we had a very comfortable solution with our own bugzilla instance. But our project is growing, and we have to check whether this solution still is optimum for us or whether we will have to think about a change.

Please help to complete this page. It's a Wiki, please do not suggest what should be added, simply add the contents. Only if you are not sure concerning your ideas start a discussion on the Discussion page.

Ideas

Allow logic sort additionally to alphabetical

  • Why (Example): Components currently are sorted alphabetically, It might be useful to have some Groups like "Libo-Components" A-Z, "Software Module related" A-Z, "Others" A-Z
  • Examples:
  • How:
  • Related Bugs:
  • Related Bugs:

search Version Picker (and others)

  • Why: If there are lots of items in a picker (see AOOo Version picker) it might be annoying to find the required Version. In AOOo they have the advantage that they now started "AOOo" versions, simple typing "A" brings you to the first of those items. Our current numbering system does not allow such tricks (excepted for "unspecific"). So it might be useful if normal Browser search also works for the pickers or if when picker list is open typing of some letters will show up the first item containing that typed string.
  • Examples:
    • [ ]
    • [ ]
  • How:
  • Related Bugs:

True version tracking

  • Why: Bugzilla is not aware of the various branches and starting from which commit (or version) the bug is fixed in each branch. Having a bug tracking system that tracks this information allows much more easy / cleaner querying. For example: show me all bugs still open in version 3.5.4, or in branch libreoffice-3-5. Ideally, it would also keep track of the decision to fix or not *per* *branch*, so that we can issue requests like "show me all bugs open in libreoffice-3-5 that we have not decided not to fix in 3.5. This would also allow e.g. to show a user reporting a bug the (relevant) bugs fixed since the version (s)he is reporting against, with something like "These bugs have been fixed since the release of 3.5.3; please look them over and see if your bug is in there".
  • Examples:
    • The Debian bug tracking system tracks "versions where fixed and versions where bug found". Most probably not adequate for LibreOffice for other reasons, though.
  • How:
    • This would need switching to another bug tracking system than bugzilla.
      • I am not sure - basically that should be possible using Bugzilla Tags and Flags -- Rainer Bielefeld 2012-06-04T17:22:10 (CEST)
        • Bug sightings feature dealing with branches is planned: moz#55970
    • Enable and use "Target Milestone" field in bugzilla, but this needs #Account related permissions to prevent abuse from users (see [1])

Possible description editing

  • Why: Bug Description, aka comment 0, is the first thing everyone read when open a bug. So, ideally it should be correct, complete, and shortest possible, without some angry texts or emotion. The problem is, if I understand correctly, we (bug triagers / developers) can't edit it, and the people who comes late will have to read all comments to see what exactly is the problem.
  • Examples:
    • Launchpad bug tracker has ability to edit the description. See the change on 2011-10-12 23:28:31 from [2] for example.
  • How: This is discussed in moz#540
  • Related Bugs:

Show activity in comment page

  • Why: Showing bug activity, such as changing status, priority, keywords, etc, in comment page would reduce a time to go to "history" page and comparing who did the change in which comments and why.
  • Examples:
    • Launchpad bug tracker has this feature along with separated history page. See comment 2, 5, 6 in lp#785397 and "full activity log" at the bottom of the page.
    • Google Code bug tracker has this feature without separated history page. See comment 2, 3, 4 in chromium bug 130079.
  • How:
    • b.m.o has InlineHistory extension for Bugzilla (from extension's README: the information is only available to logged in users) example bug (source)
  • Related Bugs:

Associate User with NEEDINFO State

  • Problem: If a bug is NEEDINFO state, the name of the info provider is sometimes well hidden in the many comments. Some people even forget to explain who should provide the information.
  • Possible solution: If you set NEEDINFO state, you should be able to define the name of the info provider in extra filed. It should not be the "assigned" field because the info provided would have problem to assign the bug back to the requestor.
  • Custom field displayed only when bug is in NEEDINFO status. Problem: you can't create custom username fields out-of-the-box.
  • Examples: This works well in [bugzilla]

3. reporter invitations

Currently it's not yet defined how that will be done.

4. getting useful extensions onboard

We will need more concrete wishes ("Details") here within short time!:

Detail: Additional reports and statistics

  • Why: Currently several statistics are maintained outside Bugzilla, for example daily / monthly number of bugfixes as line chart. It would save a lot of time if we could get such timeline related charts directly from Bugzilla. I would like to be able to create line charts like "Rainerbielfeld's Master Bug reports, weekly average last 6 months, may be even from the scratch.
  • Examples:
  • How: for example Bugzilla Automated Metrics; Create New Data Sets based on queries?
Ad hoc date related Charts and Tables for Actions

Currently only Bug numbers at a particular day can be shown in a chart easily. Statistics concerning actions only can be created with long preparation for the future. We need the possibility to create charts concerning questions to the past, some examples

  • Bug reports / User(s) per month user related
  • Monthly bug fixes Component related
  • Comments of a particular user per week

6. Other

Hide obsolete Version numbers for new Reports

  • Why: New reporters generally will report for their current version, so all the old Versions might be only worrying. But to define the Version with what the problem started the old Versions still will be required after the original report has been done.
  • How: Currently there will be a possibility to suppress old Versions in the guided forms of bugzilla.

Time shift reporter invitation for more contribution

  • Why: Reporters who used the Bugzilla Guided Forms (and so might be novices) should be invited to help to confirm bugs. But immediately after the report might be awkward, for example because the reporter still is angry because of the problem he had with LibreOffice. So a time shift seems appropriate, invitation should happen (for example) 1 week after the report or when the report has been confirmed of fixed.
  • Examples:
  • How: Might be done with 3 Items:
    • Automated (daily cronjob)
      • Read E-Mail Addresses / Reporter names
      • Send E-mails
      • May be create Database "No second invitation".
    • Currently not affecting Contracting?

Appropriate Bugzilla Help

  • Why: Currently Bugzilla Status help is for "New Workflow", where (for example) Status NEW does not exist. Currently there is no interest to change Workflow, and Help seems to be only available for new one.
  • How: Help of obsolete Version?

Show warning if users change version to a later one

  • Why: Unexperienced users often change Version wrongly (<BugReport Details#Version>) to a later version because they find out that a bug still exists in the latest LibO version. It costs time to change back to appropriate most early version where the but appeared
  • How: It seems that that can be done easily with some additional JavaScirpt in the pages.
  • Related Bugs: tdf#49168

stuff from the old migration page

  • Increase size limit on attachments DONE -- bumped to 10MB
      1. Add a note to the attachment page telling users to ping us at <bugzilla-attachment-assistance@bugs.documentfoundation.org> if their attachment is too big to upload.
  • Change status to ASSIGNED when someone is assigned to address the bug.
    1. Ability to open/present files from archives
      • So if someone uploads a zip of 5 docx files, have a web interface to show those and allow downloading them individually
    2. Side-by-side comparison
      • It's really helpful to have side-by-side screenshots to compare how a doc renders
      • Creating a 2-up (or 4-up, 6-up, etc..) should be done programmatically, and save human time
    3. Relationships between attachments
      • have a way to indicate that X is a screenshot of Y when opened with version Z on OS A. (where X, Y are both attachments)
      • Relationships could allow us to upload the files first, then to say "make a 2-up of X and Y".
        • Ideally, someone else could come along and say "and make it a 3-up with Z" (with a history to allow undos, of course :-)
  • A plugin to transfer bugs between bugtrackers
    • so if www bugs end up bugzilla, we can just transfer them other, rather than resolving them and asking the reporter to re-file
  • Add a field like "Latest Confirmation on:" to prevent users changing the "other version field" to be changed, AOO has this, see i#99130 for instance
  • Turn bug reporters into triagers, testers, etc..
    • Incorporate triage into bug report process:
      • Show bug reporters bugs that are similar to theirs (e.g. SVG bugs, or templating bugs, or OSX bugs)
      • Ask them if the suggestion is the same bug as theirs or different.
      • If it's different bug, see if they can reproduce it
      • This will bring them into the triaging process
    • Tommy suggests:
      • When a new user finish reporting a new bugs let's redirect him/her to a 'thanks' page
      • Message: "thanks for filing a new bug report. In the meanwhile when you are waiting your bug to be confirmed, why don't you take the time to try to triage one of the UNCONFIRMED bugs opened by other users like you?"
      • Give the user links to the "how to triage bugs wiki" and UNCONFIRMED bugs list
    • Slogans:
      • "Submit a new bug, triage another one"
      • "Submit one, triage one" ?
      • "Help weed the garden, and it's easier to see the bugs"
      • "Submit one, Test two" ( this gets people thinking about giving back a little more, plus not all people are going to actually test...)

Stuff from QA/Projects/Ideas/Self-Hosted Bugtracker + talk page

  • Master bug description, which serves as the single description the assignee can look at to get the whole picture of the reported bug. This should be editable by qualified personnel after the fact.
  • Better user management (banning, suspending, warnings)
  • "Bug Heat" - a good algorithm to judge how important a bug is based on multiple factors
  • NEEDINFO - when a user posts something to a bug in NEEDINFO status, ask if they want to move the bug back to UNCONFIRMED ("have you provided all information requested")
  • Match our bots ability to do things like tdf#9999 and have a link appear (uniformity of our systems)
  • Ability to distinguish easily between contributor (especially developer) and "regular user" - so we know if person X changes status and they are a developer (or QA member) to not mess with the bug
  • CC'ed members should flag their subject (vs. if someone adds themselves to the bug) - this is so devs can easily filter their emails
  • Versions - three distinct versions "oldest" "latest tested" and "branch" -- branch should be automated based on either oldest or latest
  • Confidential Files -- ability to automatically remove confidential information (properties and text) if someone checks a box - but we should make it difficult to do for a user to dissuade users from just submitting confidential files every time
  • Integration of ASK and our bug tracker -- some way to automatically move a comment/report on ASK to a bug on libreoffice (and visa versa)
  • Complex/Large Files - for files that are complex and demonstrate multiple bugs so aren't very useful for a particular bug but good to see how combined bugs can create problem - ability for admins to mark an attachment as such a complex bug and move to some other location
  • Automatically close NEEDINFO bugs - 1 warning then close (what is currently done manually)
  • being able to change status on the upload attachment page without assigning the bug to you https://bugzilla.mozilla.org/show_bug.cgi?id=234530
  • being able to list bugs that has only been commented on by the reporter - no other user has commented on the bug except the author https://bugzilla.mozilla.org/show_bug.cgi?id=121335
  • being able to upload multiple files at one go, or else a checkbox on the upload page that will jump you back to the upload page after uploading
  • means of downloading the bugzilla html file and all its attachment at one go - useful for offline bug triaging
  • easy access to a bug submitter's list of submitted bugs from the first entry of a bug report
  • user profile page showing stats per day, week, 2 weeks, month, 3 months, 6 months, 1 year, since beginning - bugs submitted, bugs confirmed, comments made, bugs set to needinfo, bugs assigned, bugs closed
  • when status is changed from unconfirmed to needinfo, bugzilla shouldnt set 'ever confirmed' to 1
  • sub component drop down menu on submission page similar to the BSA form containing FILEOPEN, MAILMERGE, etc.
  • it shouldn't make mistakes with bug auto referencing like in bug 80179
  • better handling of emailed comments - eliminate html email version of comment (bug 80060 comment 2 attachment, bug 80188 comment 5 attachment), and eliminate useless 0 or 1 byte attachments from email replied comments (bug 80188 comment 6, bug 80060 comment 3) and trim quoted reply text (i.e. bug 80188 comments 5, bug 79918 comment 3, bug 79574 comment 2) similar to launchpad.net
  • have tooltip for Version and Status similar to tooltips for Product, Component, etc
  • have Whiteboard link to wiki
  • have statuses that can only be particular users - bug reporter (unconfirmed), QA team (new, needinfo, resolved [invalid, wontfix, duplicate, worksforme, notabug, notourbug]), and developers (everything)
  • bug reporters shouldn't be allowed to reply to emails sent to them from the bug tracker, and the email shouldn't contain email addresses of commenters
  • "Bug Heat" - a good algorithm to judge how important a bug is based on multiple factors

Aside build-in feature of Bugzilla counting duplicates at https://bugs.documentfoundation.org/duplicates.cgi one can think about implementing something like Launchpad algorithm - https://dev.launchpad.net/Bugs/BugHeat#Algorithm

  • Ability to distinguish easily between contributor (especially developer) and "regular user" - so we know if person X changes status and they are a developer (or QA member) to not mess with the bug

This can be done using icons for groups - https://bugs.documentfoundation.org/docs/en/html/groups.html. Just create developers, qa, etc. groups, set an icon, add the people and it will be displayed along the username.

  • Automatically close NEEDINFO bugs - 1 warning then close (what is currently done manually)

This could be done using build-in Whining system - https://bugs.documentfoundation.org/docs/en/html/whining.html.

  • easy access to a bug submitter's list of submitted bugs from the first entry of a bug report
  • user profile page showing stats per day, week, 2 weeks, month, 3 months, 6 months, 1 year, since beginning - bugs submitted, bugs confirmed, comments made, bugs set to needinfo, bugs assigned, bugs closed

There is UserProfile extension used at https://bugzilla.mozilla.org with such data available from the bug's page, where you can select user's profile or activity. Source available at https://github.com/mozilla/webtools-bmo-bugzilla/extensions/UserProfile

  • when status is changed from unconfirmed to needinfo, bugzilla shouldnt set 'ever confirmed' to 1

The question is whether NEEDINFO status should be still used. I'd rather propose UNCONFIRMED status with NEEDINFO flag set (which is searchable) along with Needinfo extension used at https://bugzilla.mozilla.org. It provides Needinfo menu under the comment field, where other/reporter/assignee/qa contact/myself is easily accessible and special logic for this type of flag. More about flags https://bugs.documentfoundation.org/docs/en/html/flags.html. Source available at https://github.com/mozilla/webtools-bmo-bugzilla/extensions/Needinfo

  • have statuses that can only be particular users - bug reporter (unconfirmed), QA team (new, needinfo, resolved [invalid, wontfix, duplicate, worksforme, notabug, notourbug]), and developers (everything)

Apart from basic Group Controls this can be done customizing who can change what according to https://bugs.documentfoundation.org/docs/en/html/cust-change-permissions.html. Best would be to create Bugzilla extension, like on bugzilla.mozilla.org - see the code examples starting at line 350 https://github.com/mozilla/webtools-bmo-bugzilla/blob/master/extensions/BMO/Extension.pm#L350.