QA/Bibisect/Bibisecting tutorial

TDF LibreOffice Document Liberation Project Community Blogs Weblate Nextcloud Redmine Ask LibreOffice Donate

• Wiki Home
• Development
• Design
• QA
• Events
• Documentation
• Website
• Localization
• Accessibility
• Marketing
• Diversity
• Wiki Help

• Bug Report
• Bug Triage
• Projects
• Regression Test
• Automated Tests
• Bibisect
• QA Meetings
• QA FAQs

Introduction

This tutorial will show you how to find the code change that caused a regression bug. It is a good idea to go through a known clear investigation as your first learning step, because sometimes bibisecting and its results can be messy.

Video

Below is a video representation of the tutorial done under Windows 10 and using PowerShell 7.

Prerequisites

In order to work on bibisecting, you need to be familiar with paths⁠, the role directories⁠ play in them and how to navigate a hierarchical file system⁠. It is absolutely critical that you understand how to be aware of your location in the file system, how to travel in it, where your downloaded files end up in and how you can move files into different directories. Otherwise you will waste many hours.

When you are comfortable with the concept of files and directories in the context of graphical applications and file managers, it is not a big step to learn the command line. Mozilla Developer Network has a command line crash course taking into account all major operating systems. MIT has more Unix/Linux centric courses for the shell and command line environment. MIT's shell course is still useful as a general reference because macOS is a type of Unix and Windows PowerShell accepts many of the standard Unix commands.

Preparation

We will be investigating a bug that prevents us from opening a certain document. Testing a bug does not get any simpler: the only required action is trying to open the document.

Unfortunately, due to backwards compatibility issues at the operating system level, the specific bibisect repository used in the tutorial does not work with recent Linux and macOS versions. Linux and macOS testers can instead do the following and adapt the steps in the tutorial to match the ones in the chosen bug:

1. go the advanced search in Bugzilla
2. in the Summary field, input: fileopen crash
3. pick --- and FIXED for the resolution
4. add keywords bisected, regression
5. in Custom Search select Version - matches regular expression and input a "starts with" pattern that matches a more recent version, for example ^7 to include all versions starting with 7 or ^7\.6 to limit to a specific major version
6. Clone the respective bibisect repository and test the issue with it (Linux, macOS)

To get an idea of which version range the bug appeared in, you would normally need to test with multiple bibisect repositories or alternatively multiple parallel-installed release versions. To make things simpler for the tutorial, we start with the knowledge that the bug appeared in the 6.1 version range (6.0 branch point to 6.1 branch point and then 6.1.x minor releases).

To set up the bibisecting environment on Windows, Linux or macOS, install git. When installing Git for Windows, it is a good idea to select the option ”Checkout as-is, commit Unix-style line endings” because this is required, if you ever get into LibreOffice development.

Next, from a command line shell, download the 6.1 repository.

Windows (note: read all of the instructions behind the link - only step 1 is done differently with PowerShell).

Download command to use with PowerShell on Windows (do not use PowerShell 5, but the latest one):

Invoke-WebRequest -Uri https://bibisect.libreoffice.org/win32-6.1.git-bundle -OutFile win32-6.1.git-bundle -Resume

Linux

macOS

The downloading and processing of the repository will take a relatively long time (perhaps an hour, depends on your system).

When you do the steps for downloading and cloning, make sure you keep working in the correct directory. So if you take a break in between and turn off your computer, you must remember to change to the right directory next time.

Note that on macOS, you might have to right click and select open in Finder to verify the app the first time you use a new bibisect repository.

Bibisecting

1. Download attachment 157140 from tdf#130093. The rest of the steps will assume the document is in your typical Downloads directory.
2. In your command line shell, enter the bibisect repository directory with the cd command.
3. Give chained commands to checkout the oldest code commit in the repository and try to open the problematic document.
   Windows:

   git checkout oldest && instdir\program\soffice 'C:\users\username\downloads\Lotus testfile.lwp'

   Linux:

   git checkout oldest && instdir/program/soffice ~/Downloads/Lotus\ testfile.lwp

   macOS:

   git checkout oldest && LibreOffice.app/Contents/MacOS/soffice ~/Downloads/Lotus\ testfile.lwp

4. Observe, how the document opens just fine.
5. Give the same chained commands as previously, but replace oldest with master. You can reach the previous command in the shell by using your up arrow keys. Move with your left/right arrow keys and Home/End keys and modify the command.
6. Observe how you get a General input/output error for the document in the newest commit of the repository.
7. If during this process you get an error saying your local changes to some Python files would be overwritten by checkout, use this command to throw away the changes:

   git checkout .

8. Again, bring up the chained commands, but replace the first one entirely with

   git bisect start master oldest

9. Bring up a previous chained command and if the document opened fine, replace the first command with

   git bisect good

   If you saw the error or a crash upon opening, replace the first command with

   git bisect bad

10. Continue this dance of goodness and badness until you see a block of text like this:

   a8220993df5ae8b0e673e536c0d9dd7b458e55db is the first bad commit
   commit a8220993df5ae8b0e673e536c0d9dd7b458e55db
   Author: Norbert Thiebaud <nthiebaudgmail.com>
   Date:   Mon Feb 12 13:20:52 2018 -0800

       source sha:22fc8c634c5f9b09d45aff0403503f4d8226328d

       source sha:22fc8c634c5f9b09d45aff0403503f4d8226328d

Analysis

The above bad commit result is taken from the Windows 6.1 repository. The bad commit refers to the binary build in the repository itself. The actual source commit is the hash string after "source sha:". The person next to "Author:" is the one who created the bibisect repository and has nothing to do with the regression bug.

If you want to verify that the result is correct, you can do a checkout for the binary commit (the exact hash depends on the repository you use):

   git checkout a8220993df5ae8b0e673e536c0d9dd7b458e55db

Try to open the document (you should see the read error). Then checkout the previous commit with the following command:

   git checkout HEAD^

Now you should be able to open the document.

You can view the details of the code change by taking the "source sha" string and putting it into a URL that you can then visit:

https://git.libreoffice.org/core/commit/22fc8c634c5f9b09d45aff0403503f4d8226328d

In the code changes we can see that it is dealing with the lotuswordpro module, which is in line with the topic.

Making the results public

Sometimes a regression has already been reported and analysed. This means we should always do a search in Bugzilla before declaring our results.

1. Go to the advanced search in Bugzilla.
2. Hold down Ctrl key and left-click the --- below "Resolution:". This will allow you to search both open and closed reports.
3. Expand the Detailed Bug Information section.
4. Paste the "source sha" string into the Comment field.
5. In the Keywords field, type regression and click the Search button.

In the case that no reports are found, do these steps in the report that requested bibisecting:

1. Remove the keyword 'bibisectRequest' from Keywords field and add the keyword 'bibisected'.
2. If the commit causing the regression has been identified (as in this tutorial), also add the keyword 'bisected'.
3. Add the blamed developer to CC (note: this is not the author of the commits of the bibisect repo - you have to view the details of the code change as described in the Analysis section).
4. Add the name of the blamed developer to the Regression By field (if you don't see this field, ask TDF staff to add you to the contributors group in Bugzilla).
5. In your comment, mention which bibisect repository you used, show the blamed commit and include the text 'Adding Cc: to <developer's name>'