QA/Bibisect/Bibisecting tutorial

From The Document Foundation Wiki
< QA‎ | Bibisect
Jump to: navigation, search

Introduction

This tutorial will show you how to find the code change that caused a regression bug. It is a good idea to go through a known clear investigation as your first learning step, because sometimes bibisecting and its results can be messy.

Preparation

We will be investigating a bug that prevents us from opening a certain document. Testing a bug does not get any simpler: the only required action is trying to open the document.

To get an idea of which version range the bug appeared in, you would normally need to test with multiple bibisect repositories or alternatively multiple parallel-installed release versions. To make things simpler for the tutorial, we start with the knowledge that the bug appeared in the 6.1 version range (6.0 branch point to 6.1 branch point and then 6.1.x minor releases).

Setting up the bibisecting environment on Linux and macOS is easy: just install git. On Windows, you need to install Cygwin.

Next, from a command line shell, clone the 6.1 repository.

Windows:

   git clone https://git.libreoffice.org/bibisect-win32-6.1

Linux:

   git clone https://git.libreoffice.org/bibisect-linux-64-6.1

macOS:

   git clone https://git.libreoffice.org/bibisect-mac64-6.1

The downloading and processing of the repository will take a relatively long time (perhaps an hour, depends on your system).

Bibisecting

  1. Download attachment 138317 from tdf#114351. The rest of the steps will assume the document is in your typical Downloads directory.
  2. In your command line shell, enter the bibisect repository directory with the cd command.
  3. Give chained commands to checkout the oldest code commit in the repository and try to open the problematic document. Windows:
    git checkout oldest && instdir/program/soffice 'c:\users\username\downloads\BrokenCrossRef.odt'
    
    Linux:
    git checkout oldest && instdir/program/soffice ~/Downloads/BrokenCrossRef.odt
    
    macOS:
    git checkout oldest && LibreOffice.app/Contents/MacOS/soffice ~/Downloads/BrokenCrossRef.odt
    
  4. Observe, how the document opens just fine.
  5. Give the same chained commands as previously, but replace oldest with master. You can reach the previous command in the shell by using your up arrow keys. Move with your left/right arrow keys and Home/End keys and modify the command.
  6. Observe how you get a read error for the document in the newest commit of the repository.
  7. Again, bring up the chained commands, but replace the first one entirely with
    git bisect start master oldest
    
  8. Bring up the previous commands and if the document opened fine, replace the first command with
    git bisect good
    
    If you saw the read error, replace the first command with
    git bisect bad
    
  9. Continue this dance of goodness and badness until you see a block of text like this:
   4a73f868e7d2f62b1ccd480731896a36a1f94035 is the first bad commit
   commit 4a73f868e7d2f62b1ccd480731896a36a1f94035
   Author: Norbert Thiebaud <nthiebaud@gmail.com>
   Date:   Tue Mar 6 20:20:50 2018 -0800
       source sha:27008aa028cde8d270e898c5743a9fe5c7701dab
       source sha:27008aa028cde8d270e898c5743a9fe5c7701dab

Analysis

The above bad commit result is taken from the Windows 6.1 repository. The bad commit refers to the binary build in the repository itself. The actual source commit is the hash string after "source sha:". The person next to "Author:" is the one who created the bibisect repository and has nothing to do with the regression bug.

If you want to verify that the result is correct, you can do

   git checkout 4a73f868e7d2f62b1ccd480731896a36a1f94035

and try to open the document (you should see the read error). Then checkout the previous commit with

   git checkout HEAD^1

Now you should be able to open the document.

You can view the details of the code change by taking the "source sha" string and putting it in a URL that you can then visit: https://gerrit.libreoffice.org/plugins/gitiles/core/+/27008aa028cde8d270e898c5743a9fe5c7701dab%5E!/

The commit message for the change is "xmloff: convert XMLTextParagraphExport to get rid of "GraphicURL". In the code changes we can see that it is dealing with importing XML, which is in line with the read error we are facing.

Making the results public

Sometimes a regression has already been reported and analysed. This means we should always do a search in Bugzilla before declaring our results.

  1. Go to the advanced search in Bugzilla.
  2. Hold down Ctrl key and left-click the --- below "Resolution:". This will allow you to search both open and closed reports.
  3. Expand the Detailed Bug Information section.
  4. Paste the "source sha" string into the Comment field.
  5. In the Keywords field, type regression and click the Search button.

After the last step, you will see a list of bugs all blaming the same commit. One of them is essentially identical with our problem: tdf#122144 - Cannot open file in LibreOffice 6.1: Format error discovered in file.

In the case that no reports are found, do these steps in the report that requested bibisecting:

  1. Remove the keyword 'bibisectRequest' from Keywords field and add the keyword 'bibisected'.
  2. If the commit causing the regression has been identified (as in this tutorial), also add the keyword 'bisected'.
  3. Add the blamed developer to CC.
  4. In your comment, mention which bibisect repository you used, show the blamed commit and include the text 'Adding Cc: to <developer's name>'