QA/Bibisect

From The Document Foundation Wiki
< QA(Redirected from QA/HowToBibisect)
Jump to: navigation, search

Introduction

Bibisect stands for "binary bisect". It's a procedure that helps identify commits that have caused regressions. Regressions are a most annoying artifact that unfortunately come with software development and QA. We want to deal with regressions quickly before time passes and they get harder and harder to triage and fix.

How does Bibisecting Work?

Each bibisect package is a git repository populated with hundreds or thousands of builds of LibreOffice. Added to the repository in chronological order, the builds provide a fast and simple way for us to test a bug against many different versions without having to compile LibreOffice each time we switch to a different version.

We can leverage the tools built into git for binary search through commits (e.g. git bisect) as well as benefit from the compression and de-duplication functions included in git. For example, in a series of builds that each might take up 500MB of disk space, through de-duplication we can potentially use only 25MB per additional build.

Watch Effective Bisection and Bibisection (Matthew Francis's talk at LibOCon 2015 on YouTube) for both an introduction, and practical details.

Details

A successful bibisect will leave developers with only a few commits to search through to find the 'culprit' commit that we believe introduced a regression. Depending upon the particular bibisect repository, this range might be 60, 100, or even 2 commits. (We also have some very 'coarse' bibisect repositories that use released versions + beta + RC builds, and these may have ranges spanning multiple hundreds of commits)

QA can go the additional step of fully bisecting (determining the exact commit) or can leave this to the developers. It's often easy to find the commit that caused the issue after bibisecting the bug.

With approximately 10 gigs, you'll have hundreds of builds to work with. A bibisect can take as little as 15 minutes.

Currently, bibisecting is possible on:

General Instructions

Note: Each operating system is different, please read the full instructions that correspond to your specific operating system (GNU/Linux, OS X, or Windows) for more complete instructions, instructional videos, and other related information.

Bibisecting relies on 6 steps:

  1. Setting up your environment, including installing any extra software
  2. Downloading the bibisect package
  3. Bibisecting the bug
  4. Getting a bibisect log
  5. Attaching the bibisect log to the bug report
  6. Updating the bug report fields

Limitations

Performing a bibisect is only possible if we have a repository available for the correct OS, covering the correct range of LibreOffice commits. We have very good coverage on GNU/Linux, and are improving our coverage on Windows and OS X.

Some regressions are difficult to reproduce, and may require multiple test runs for a bug to appear. As you may imagine, trying to bibisect such regressions is very difficult, and requires one to perform many test runs after each checkout of a new version of LibreOffice to give some security / statistics that the bug is indeed present or not present with the given version.

We do not currently have bibisect support for

Versions

Versions are listed on each OS page. See #Details above.

Not Bibisectable

Some bugs are not bibisectable. These include

  • Bugs that don't show up on a supported platform (see #Limitations)
  • Bugs that predate our bibisect builds (We're working on extending the range covered)
  • Bugs that depend upon build options not enabled in the TDF builds (e.g. -kde)

If a bug with bibisectRequest Keyword is found to be not bibisectable, leave a comment on the bug explaining why we're not able to use bibisect to track down the introduction of this problem, and replace Keyword bibisectRequest with notBibisectable.

Finding bugs

Finding bugs needing bibisect

Following Bugs with Keyword bibisectRequest are waiting for a bibisect:

ID Summary Priority Status
43715 Impress: presentation minimizer not reduces resolution of images (regression after 3.3) medium NEW
43822 Ctrl-W does not close document when the focus is in the Navigator medium NEW
51119 EDITING: OLE objects from writer are impossible to manage medium NEW
54551 FILEOPEN: long time to load .DOC file medium NEW
54703 Hidden Sections are no longer hidden when printing or exporting pdf. medium NEW
56173 FILESAVE impossible for password protected shared .ods medium NEW
60428 Duplex Printing - HP Officejet Pro 8600 Plus medium NEW
61083 Underlines are not imported from EMF medium NEW
64423 Import of XML Excel file into Calc fails with General input/output error. medium NEW
65535 Adding a comment in the middle of a word makes it recognised as misspelt by spellchecker medium NEW
66054 FORMATTING: font work medium NEW
66237 Document preview of tables shows overlayed toolbar icons and misplaced column headers medium NEW
67744 FORMATTING: Adobe Courier Std font is always bold on Mac OS X medium NEW
69254 FORMATTING: Custom font handling medium NEW
69461 SLIDESHOW: Inserted Movie files show a white screen for about one frame at the start high NEW
69479 Formula is getting corrupted after input medium NEW
69631 PDF: Diagrams from DOC files not exported to PDF unless clicked in medium NEW
69932 VIEWING: LibO 4.x.x flicker low NEW
70607 LibO uses 100% CPU and crashes when quit during extension update high NEW
70710 filled areas are not filled for WMF images medium NEW
70838 FILESAVE: doc, docx: wrong size and position exporting rotated shapes medium NEW
71828 Distorted eps-exports with LO Draw medium NEW
73873 File Corruption - Issue with w:headerReference in document.xml medium NEW
75218 Letter Wizard should be improved to follow locale customs medium NEW
75260 double line border styles wrongly displayed high NEW
75471 Newly opened LibO document frames stay in background (i. e. LibO does not get focus) medium NEW
75650 Formulas with boolean result change cell format while other formulas keep it medium NEW
76644 extended tips missing for menu items medium NEW
77444 LO closed then opening any document by double-click never opens the file. mouse movement required for the file to open highest NEW
77525 PRINTING: WMF don't correctly print with 'Reduce Image Resolution' enabled on Windows (workaround: Right-click "Compress Graphic" and "Calculate") medium REOPENED
78779 PDF: Bookmarks not exported as described medium NEW
79063 cell height of spreadsheet printed different after cut/paste to Writer as object when row height set using dialog low NEW
79452 Themes with a separator line per toolbar incorrectly places separator line when toolbar is on bottom low NEW
80473 Wrong style applied to DOCX paragraph when open in CJK UI (because of wrong style hereditability) medium NEEDINFO
82091 Spelling dialog <F7> correction of Calc cells does not remove red wavy underline, but context menu spelling correction does medium NEW
82113 FILEOPEN: Text in EMF not displayed medium NEW
87165 FILEOPEN: text missing in tables in .rtf medium NEW
87218 FILEOPEN: table displayed in wrong position in .rtf low NEW
87219 FILEOPEN: line at the top displayed at the bottom in .rtf medium NEW
87912 EDITING: Image is hidden, when anchor is character and another image is inserted on the same page medium NEW
89147 Calc comment text invisible when using missing font in .xls file medium NEW
89497 EDITING, FORMATTING: Text frame gets lower, sometimes making text invisible, when switching templates medium NEW
89546 Single column Autofilter with "Not empty" shows empty cells medium NEW
89696 unknown software exception 0x40000015 at address 0x0036327c high NEW
90414 Text inside Folded Corner rendering at Draw is different when export to SVG medium NEW
90580 VBA modules break valid StarBasic code; cannot delete "VBA context" medium NEW
90621 FORMATTING: Custom character styles are not applied in bibliographies medium NEW
91237 CALC hangs on web table paste high NEW
91622 Close button in soffice.exe command line help message dialog not clickable medium NEW
91770 Crash on closing on Windows highest NEW
92052 Black line around focused area within focused widgets medium NEW
92268 Can't open document via webdav link + SSO medium NEEDINFO
92472 FILEOPEN: First checkbox in row has too large dimensions on docx import medium NEW
92617 Base is crashing frequently when EDITING data and switching Forms high REOPENED
92771 "run macros" dialog shows libraries in wrong section medium NEW
92796 [ODF] Writer does not remove unused page background image/bitmap medium NEW
92909 UI - Table creation field data types are displayed in rounded buttons that do not fill grid cell (OSX only) medium NEW
92928 Same network connection error for each call to GETPIVOTDATA medium NEW
93085 Application window is too high, status bar is not visible (1024x600 resolution), size of module launch inherited from the StartCenter which gets a fixed minimum height (comment 14) high NEW
93191 Sheet Tab Controls Right Click Not Working Correctly medium NEEDINFO
93352 RTL: Horizontal scrolling for right-to-left Sheet moves in opposite direction medium NEW
93361 Pivot table column total doesn't work high NEW
93660 FILEOPEN: Writer cannot open DOCX with large table again medium NEW
93874 FILEOPEN: DOCX import table: colors and structure lost high NEW
94259 X-Y-Graphic in Excel File are shown wrong in version 4.4, 4.5, 5.01 medium NEW
94265 bootstrap exception when running C# application with LibreOffice API medium NEW
94448 UI: Draw’s context menu lacks Text option medium NEW
94666 Narrow spacing of menu items, and truncated tooltips with lower resolution low NEW
94737 Poor subpixel rendering on Linux medium NEW
94792 Fileopen: performance regression for xlsx with chart (per Comment 4) medium NEW
94816 MAILMERGE: Writer freeze in Serial letter assistant step 6. with German UI high NEEDINFO
94942 Corrupt dialog under 5.0.2.2 (Format -> Cells -> Border-> Width) medium NEW
94950 FILEOPEN: text line inside .doc file showed in wrong position medium NEW
95150 Border of tablecontrol doesn't show color medium NEW
95191 Forms and dialogs opened via macros on ODB file open no longer work high NEW
95209 Broken font rendering of some characters in Opus Standard font (was ok in LO4.4.2) medium NEW
95431 Insert big image get black box medium NEW
95476 FILESAFE: Fileexport to xhtml shows display:condition; for section, which is defined as displayed with condition medium NEW
95539 Option + arrow does not work properly when RTL is enabled medium NEW
95658 Repagination caused by changing print settings in File > Print .. under "Range and Copies" is too slow low NEW
95680 Convert - Metafile - Save Image... No function. Must use Alt+R medium NEW
95960 UI - Displaced field value widget in User-Defined Document Properties medium NEW
96095 Performance regression when opening "Getting Started" in 5.0+ medium NEW
96176 DRAW: Slow performance opening and editing a file medium NEW
96341 Version 5.1 takes a lot more RAM than version 5.0 in Windows medium NEW
96500 EDITING: flicker during mouse-move of table control (form design mode) medium NEW
96753 KDE: big file opening and navigating is slow vs. 4.3 medium NEW
96826 Typewriter attribute given too much weight when finding font based on attributed medium NEW
96897 Function window always displayed medium NEW
96947 Check marks on menus (Windows 7 Aero theme, OpenGL enabled) miss their borders medium NEW
97129 No feedback while saving a document medium NEW
97226 Can not open Remote Files in Google Drive on Windows XP medium NEW
97231 QueryValue not working as expected medium NEW
97395 FILEOPEN: Libre Office odb crashes when try to open tables high NEW
97548 EDITING: When selecting cells for a formula, Ctrl is incorrectly behaving like Shift medium NEW
97579 Unable to link Access 2010 database causing LireOffice to freeze and requiring force kill high NEW
97768 working with onscreen-keyboard LO misunderstands AltGr+E (Euro) as Alt+E medium NEW
97947 FILEOPEN RTF layout problem with table medium NEW
98079 FILESAVE CALC crashes sporadically when saving multiple sheet workbook (Win x64) high NEW
98094 NAVIGATOR: Unable to drag header in 'Content View', if first was tried to drag a header with all Categories visible. medium NEW
98173 Regular quotation marks in italics in Times New Roman overwrite text - hebrew text medium NEW
98243 cell scroll highlighting breaks cell border display when navigating up / down at document magnification of at least 150% low NEW
98349 Handling of PUA mapped "symbol fonts" like Chess fonts, some characters not mapping back into keyboard when used in LO medium NEW
98454 Visual corruption when viewing a document in web view, help documentation, and text based controls medium UNCONFIRMED
98491 Split hsqldb Base application will not start under OSX10.11.x causes LO to freeze requiring force kill high NEW
98546 EDITING sheet links created using a macro are not updated immediately medium NEW
98598 VIEWING: Background image and web layout medium NEW
98967 Paragraph "Fixed" line spacings using Point units are incorrectly being rounded--unable to set spacing to tenths or hundredths of a Point precision medium NEW
98993 Rounding off entries distorts line spacing medium NEW
99071 Navigator heading is Truncated when mouse click medium NEW
99088 Export to PNG messes up with DPI medium NEW
99403 FILESAVE: progress bar intermittent in showing progress of save medium NEW
99537 SIDEBAR: Content panel setting not saved between context changes medium NEW
99651 PRINTING - text in an embedded EMF not printed/exported to PDF correctly medium NEW
99685 Slides are flashing buffer of prior slide at the end of "Fade" transition, or end of "Fine Dissolve" transition medium NEW
99707 Impress stalls presentation on 3-liner text block medium NEW
99720 Custom animation "Put on the Brakes" doesnt show correctly in slide show medium NEW
99784 EXTENSION MANAGER - freeze / hang when attempting to install extensions on OSX 10.11 by double-click on oxt file highest NEW
99810 FILESAVE DOCX: Lines lost when saved as DOCX medium NEW
99912 Writer creates a new hyperlink when proofing hyperlink text medium NEW
99924 Impress does not display GDI correctly in OSX high NEW
99927 Changing font in certain cases causes visual glitches in UI (eg. blank menus and dialogs) medium NEW
99946 Calendar control shows misaligned day abbreviations (OSX) high NEW
100050 UI / VIEWING: Tango icons for saving flicker while saving medium NEW
100117 LibO crashes when switching Sidebar decks in Writer/Calc if two windows visible (KDE4) medium NEW
100119 Korean Autocorrect does not work in OS X 10.11 medium NEW
100291 When using the Move/Copy Sheet to a different workbook actions are grayed out (only when 1 sheet exists in source document) medium NEW
100477 Report builder generates empty pages high NEW
100611 DATALOSS: Document with chart can be saved as chart, and can't be opened again without manual fix medium NEW
100622 Menu shortcuts don't work with right ALT key medium NEW
100750 Captions aren't editable when a frame of a picture is anchored "As character" medium NEW
100756 FileOpen: Opening documents via LibreOffice (from other applications) medium NEW
100813 Writer crashes immediately after openning this .docx or on scroll (Linux fixed, Windows remains) medium NEW
100925 When overwriting a cell, new content is displayed on top of the old content until finishing the edit highest NEW
100996 Tools -> Solver... does not display message anymore if no compatible Java installed medium NEW
101293 Bad marker icons in Galaxy and Highcontrast theme low NEW
101305 UI - dark blue selection within template browser as well as within recent documents view medium NEW
101528 Crashes on 3D chart modification high NEEDINFO
101873 Searching for text doesn't find it the first time after searching for text that is not in the document medium NEW
101895 Paragraph dialog units shown as "ch" and "lines" when "Enable char unit" is enabled by default medium NEW
101915 F11 shortcut should shift keyboard focus to control sidebar content panel of Styles and Formatting deck medium NEW
101948 Search fails to find specified indentation in imported Word document medium NEW
101952 UI bug when switching "Maximize" and "Unmaximize" when using gen/X11 backend medium NEW
102224 Google crosextra fonts missing kerning - difference to MS C fonts (and Windows to Linux) medium NEW
102320 Draw: Vectorizing text results in garbage output high NEW
102405 Repagination from web view causes TOC to appear on more pages medium UNCONFIRMED
102730 EDITING: Image in slide shifts when selected (see comment 6) medium NEW
103056 Endless scrolling upwards after reaching top of document medium NEW
103151 SVG rendered wrong in the presenter console medium NEW
103218 Imported XSLX wrong image boundaries medium NEW
103252 macOS: Time to open StartCenter high NEW
103361 Rendering bug with symbols misplacement medium NEW
103363 MAILMERGE: Outgoing mailserver (set in Tools > Options > Writer > Mail merge Email) not found medium NEW
103375 MAILMERGE: unpredictable failures: sometimes looses embedded image, or not all mails sent, when e-mailing as pdf medium NEW
103412 Base report builder: "font color" and "background color" buttons in conditional formatting are broken medium NEW
103474 FORMATTING: bracket shape not shown correctly medium NEW
103476 FORMATTING: bad paragraph and line spacing in textbox medium NEW
103543 Extremely slow deletion of rows from particular XLSX (or hangs) in Linux medium NEW
103553 FORMATTING: Writer show zero page borders in specific docx file medium NEW
103567 FILESAVE: Hyperlinks inserted in gallery objects are not persistent in ODP files medium NEW
103570 Java not selected automatically when a database is opened for the first time low UNCONFIRMED
103586 Assigning shortcuts to macro in context LibreOffice (not a module) does not work anymore medium NEW
103658 FILESAVE: Incredibly slow to save on first change after opening medium NEW
103753 create a new sheet while cell is in edit mode medium NEW
103800 FILEOPEN PPTX: Wrong font color medium NEW
103806 Slide sorter does bizarre things medium NEW
103830 Form field names not update in python macro medium NEW
104020 FILESAVE Saving business cards loses positioning of image medium NEW
104097 FILESAVE: Correct formatted sheet document is miss-formatted displayed after save + reopen medium NEW
104110 FILEOPEN RTF: The text within a textbox contained in a drawing canvas is invisible medium NEW
104111 FILEOPEN RTF: Three red dots appear if a textbox is placed in a drawing canvas medium NEW
104163 FILEOPEN: DOCX: background image not imported medium NEW
104217 FILEOPEN: PPTX: Wordart is imported with no color medium NEW
104222 FILESAVE: PPTX: performance regression at save time in Linux, write error in Windows medium NEW
104225 VIEWING: sluggishness while scrolling medium NEW
104228 rtf bad opening medium NEW
104241 Not possible to scroll past page 1 in GS51-GettingStartedLO.odt (large image on page 1) medium NEW
104261 VIEWING: Calc red comment indicator is changing shape low NEW
104270 COMMENTING: Inserting a Comment at the end of paragraph causes it's termination symbol to shift right low NEW
104296 VIEWING: Relatively slow scrolling in GDI mode if a simple GIF-file is embedded medium NEW
104334 FILEOPEN: DOC: Incorrect StyleRef medium NEW
104348 FILEOPEN DOCX: Spacing below paragraph which is not there in Word medium NEW
104381 Missing acessible text-caret-moved events in Writer paragraphs medium UNCONFIRMED
104384 FILEOPEN DOCX: Textboxes in drawing canvas are corrupted medium NEW
104385 FILEOPEN DOCX: No cloud shape medium NEW
104414 FILEOPEN DOCX: SAXParseException: unknown error' medium NEW
104423 UI: Edit > Duplicate (Shft+F3) doesn't any longer remember last used values medium UNCONFIRMED
104447 FILEOPEN: Very slow opening of specific file medium NEW
104488 LibO hangs when scrolling document highest NEW

189 Open; 0 Resolved; 189 Total (0% complete)


In addition all unresolved regressions in LibreOffice might benefit from bibisecting.

Finding bugs already bibisected

For the links to bugzilla queries, please see here.

With the above info added to the bug a developer knows that the regression was introduced between the commits fb754a0df859e30255c25af8fa19bfaa75f257e7 (good) and 2d19e9bb07ccff3134f855812dddfda5c07b1fe4 (bad) on master. A:

git log fb754a0df859e30255c25af8fa19bfaa75f257e7..2d19e9bb07ccff3134f855812dddfda5c07b1fe4

on the source repository will then show the 128 commits including the one that introduced the bug, making it a lot easier for the developer to close in on the culprit. For more details, see:

Special situations

running with a separate user profile

To create separate profile directory in /tmp/ (which will be automatically removed during the next machine boot):

./opt/program/soffice -env:UserInstallation=file:///tmp/soffice-bisect

If you want to start fresh for each bisect iteration, you can add git commit hash to directory name. For example:

./opt/program/soffice \
-env:UserInstallation=file:///tmp/soffice-bisect-$(git rev-parse HEAD)

Finding a fix

Sometimes a bug is a collateral damage of another (possibly minor) bug, in a non obvious way. In such cases that other bug might be fixed on master, but was not backported to the release branch. You can find the fix, just like finding a bug, except that you need to type

git bisect good

for

git bisect bad

and vice versa. If the fix is small and important enough, it likely can be easily backported to the release branch.

Troubleshooting

Please add your problems and how you solved them. This section covers issues that affect more than one OS. For OS-specific issues, please see the individual OS wiki pages.

Unable to extract files

If you can't extract files (you should get an error message). Try to rename the file from *.tar.lzma to *.tar.xz.

Installed LibreOffice became modified unexpectedly

Problem / Messages:

   error: Your local changes to the following files would be overwritten by checkout:
     opt/program/pythonloader.pyc
     opt/program/uno.pyc
     opt/program/unohelper.pyc
   Please, commit your changes or stash them before you can switch branches.
   Aborting

Solutions:

git checkout .

will repair the problem. Afterwards you can start the next test run as usual with

git bisect good # or git bisect bad

If that does not solve the problem, you can try

git clean -fd
git reset --hard

The ultimate repair is to delete everything except one

.git/

hidden folder, and do

git checkout oldest

.


Unable to start soffice

Problem: Upon trying to run soffice within bibisect you receive the error "unexpected operator terminate called after throwing an instance of 'com::sun::star::uno::DeploymentException'

Solution: Reset your libreoffice profile located in ~/.config/libreoffice. Make sure to backup the folder if you have any settings you'd like to preserve for use with your stable libreoffice release. After you're done with bibisect you can try to return your backed up profile to it's original location (ie. delete the new profile folder and replace it with the backed up one)


Further information

notes

For details on how the bibisect archives are generated:

For information on merging all of our bibisect repositories into one big one, see

We want to make a GUI for bibisect: