QA/Bibisect

From The Document Foundation Wiki
< QA(Redirected from QA/HowToBibisect)
Jump to: navigation, search

Introduction

Bibisect stands for "binary bisect". It's a procedure that helps identify commits that have caused regressions. Regressions are a most annoying artifact that unfortunately come with software development and QA. We want to deal with regressions quickly before time passes and they get harder and harder to triage and fix.

How does Bibisecting Work?

Each bibisect package is a git repository populated with hundreds or thousands of builds of LibreOffice. Added to the repository in chronological order, the builds provide a fast and simple way for us to test a bug against many different versions without having to compile LibreOffice each time we switch to a different version.

We can leverage the tools built into git for binary search through commits (e.g. git bisect) as well as benefit from the compression and de-duplication functions included in git. For example, in a series of builds that each might take up 500MB of disk space, through de-duplication we can potentially use only 25MB per additional build.

Watch Effective Bisection and Bibisection (Matthew Francis's talk at LibOCon 2015 on YouTube) for both an introduction, and practical details.

Details

A successful bibisect will leave developers with only a few commits to search through to find the 'culprit' commit that we believe introduced a regression. Depending upon the particular bibisect repository, this range might be 60, 100, or even 2 commits. (We also have some very 'coarse' bibisect repositories that use released versions + beta + RC builds, and these may have ranges spanning multiple hundreds of commits)

QA can go the additional step of fully bisecting (determining the exact commit) or can leave this to the developers. It's often easy to find the commit that caused the issue after bibisecting the bug.

With approximately 10 gigs, you'll have hundreds of builds to work with. A bibisect can take as little as 15 minutes.

Currently, bibisecting is possible on:

General Instructions

Note: Each operating system is different, please read the full instructions that correspond to your specific operating system (GNU/Linux, OS X, or Windows) for more complete instructions, instructional videos, and other related information.

Bibisecting relies on 6 steps:

  1. Setting up your environment, including installing any extra software
  2. Downloading the bibisect package
  3. Bibisecting the bug
  4. Getting a bibisect log
  5. Attaching the bibisect log to the bug report
  6. Updating the bug report fields

Limitations

Performing a bibisect is only possible if we have a repository available for the correct OS, covering the correct range of LibreOffice commits. We have very good coverage on GNU/Linux, and are improving our coverage on Windows and OS X.

Some regressions are difficult to reproduce, and may require multiple test runs for a bug to appear. As you may imagine, trying to bibisect such regressions is very difficult, and requires one to perform many test runs after each checkout of a new version of LibreOffice to give some security / statistics that the bug is indeed present or not present with the given version.

We do not currently have bibisect support for

Versions

Versions are listed on each OS page. See #Details above.

Not Bibisectable

Some bugs are not bibisectable. These include

  • Bugs that don't show up on a supported platform (see #Limitations)
  • Bugs that predate our bibisect builds (We're working on extending the range covered)
  • Bugs that depend upon build options not enabled in the TDF builds (e.g. -kde)

If a bug with bibisectRequest Keyword is found to be not bibisectable, leave a comment on the bug explaining why we're not able to use bibisect to track down the introduction of this problem, and replace Keyword bibisectRequest with notBibisectable.

Finding bugs

Finding bugs needing bibisect

Following Bugs with Keyword bibisectRequest are waiting for a bibisect:

ID Summary Priority Status
43715 Impress: presentation minimizer not reduces resolution of images (regression after 3.3) medium NEW
43822 Ctrl-W does not close document when the focus is in the Navigator medium NEW
51119 EDITING: OLE objects from writer are impossible to manage medium NEW
54551 FILEOPEN: long time to load .DOC file medium NEW
54703 Hidden Sections are no longer hidden when printing or exporting pdf. medium NEW
56173 FILESAVE impossible for password protected shared .ods medium NEW
60428 Duplex Printing - HP Officejet Pro 8600 Plus medium NEW
61083 Underlines are not imported from EMF medium NEW
64423 Import of XML Excel file into Calc fails with General input/output error. medium NEW
65535 Adding a comment in the middle of a word makes it recognised as misspelt by spellchecker medium NEW
66054 FORMATTING: font work medium NEW
66237 Document preview of tables shows overlayed toolbar icons and misplaced column headers medium NEW
67744 FORMATTING: Adobe Courier Std font is always bold on Mac OS X medium NEW
68784 cell comment pop-up window doesn't disappears completely when mouse moves away medium NEW
69098 FILEOPEN Imported MS-word document (.doc) does not show check boxes medium NEW
69254 FORMATTING: Custom font handling medium NEW
69461 SLIDESHOW: Inserted Movie files show a white screen for about one frame at the start high NEW
69479 Formula is getting corrupted after input medium NEW
69631 PDF: Diagrams from DOC files not exported to PDF unless clicked in medium NEW
69932 VIEWING: LibO 4.x.x flicker low NEW
70607 LibO uses 100% CPU and crashes when quit during extension update high NEW
70710 filled areas are not filled for WMF images medium NEW
70838 FILESAVE: doc, docx: wrong size and position exporting rotated shapes medium NEW
71828 Distorted eps-exports with LO Draw medium NEW
74179 .xls table linked in .ppt file is not correctly printed or exported to .pdf medium NEW
75218 Letter Wizard should be improved to follow locale customs medium NEW
75260 double line border styles wrongly displayed high NEW
75471 Newly opened LibO document frames stay in background (i. e. LibO does not get focus) medium NEW
75650 Formulas with boolean result change cell format while other formulas keep it medium NEW
76644 extended tips missing for menu items medium NEW
77444 LO closed then opening any document by double-click never opens the file. mouse movement required for the file to open highest NEW
78326 Sometimes can't delete a table in Impress/Draw medium NEW
78779 PDF: Bookmarks not exported as described medium NEW
79063 cell height of spreadsheet printed different after cut/paste to Writer as object when row height set using dialog low NEW
79452 Themes with a separator line per toolbar incorrectly places separator line when toolbar is on bottom low NEW
82091 Spelling dialog <F7> correction of Calc cells does not remove red wavy underline, but context menu spelling correction does medium NEW
82113 FILEOPEN: Text in EMF not displayed medium NEW
87165 FILEOPEN: text missing in tables in .rtf medium NEW
87218 FILEOPEN: table displayed in wrong position in .rtf medium NEW
87219 FILEOPEN: line at the top displayed at the bottom in .rtf medium NEW
87912 EDITING: Image is hidden, when anchor is character and another image is inserted on the same page medium NEW
89497 EDITING, FORMATTING: Text frame gets lower, sometimes making text invisible, when switching templates medium NEW
89546 Single column Autofilter with "Not empty" shows empty cells medium NEW
89696 unknown software exception 0x40000015 at address 0x0036327c high NEW
90414 Text inside Folded Corner rendering at Draw is different when export to SVG medium NEW
90580 VBA modules break valid StarBasic code; cannot delete "VBA context" medium NEW
90621 FORMATTING: Custom character styles are not applied in bibliographies medium NEW
91237 CALC hangs on web table paste high NEW
92052 Black line around focused area within focused widgets medium NEW
92268 Can't open document via webdav link + SSO medium NEEDINFO
92472 FILEOPEN: First checkbox in row has too large dimensions on docx import medium NEW
92617 Base is crashing frequently when EDITING data and switching Forms high REOPENED
92771 "run macros" dialog shows libraries in wrong section medium NEW
92796 [ODF] Writer does not remove unused page background image/bitmap medium NEW
92909 UI - Table creation field data types are displayed in rounded buttons that do not fill grid cell (OSX only) medium NEW
92928 Same network connection error for each call to GETPIVOTDATA medium NEW
93085 Application window is too high, status bar is not visible (1024x600 resolution), size of module launch inherited from the StartCenter which gets a fixed minimum height (comment 14) high NEW
93191 Sheet Tab Controls Right Click Not Working Correctly medium NEEDINFO
93221 "object variable not set" fault with Basic using CryptographicService with 64 bit LibreOffice an 64 bit windows medium NEEDINFO
93352 RTL: Horizontal scrolling for right-to-left Sheet moves in opposite direction medium NEW
93361 Pivot table column total doesn't work high NEW
93660 FILEOPEN: Writer cannot open DOCX with large table again medium NEW
93977 FILEOPEN DOC: Layout of the table of contents looks different in LibO compared to Word medium NEW
94259 X-Y-Graphic in Excel File are shown wrong in version 4.4, 4.5, 5.01 medium NEW
94265 bootstrap exception when running C# application with LibreOffice API medium NEW
94666 Narrow spacing of menu items, and truncated tooltips with lower resolution low NEW
94737 Poor subpixel rendering on Linux medium NEW
94792 Fileopen: performance regression for xlsx with chart (per Comment 4) medium NEW
94816 MAILMERGE: Writer freeze in Serial letter assistant step 6. with German UI high NEEDINFO
94826 Big image in header completely freezes modern hardware medium NEW
94942 Corrupt dialog under 5.0.2.2 (Format -> Cells -> Border-> Width) medium NEW
94950 FILEOPEN: text line inside .doc file showed in wrong position medium NEW
95150 Border of tablecontrol doesn't show color medium NEW
95191 Forms and dialogs opened via macros on ODB file open no longer work high NEW
95476 FILESAFE: Fileexport to xhtml shows display:condition; for section, which is defined as displayed with condition medium NEW
95539 Option + arrow does not work properly when RTL is enabled medium NEW
95658 Repagination caused by changing print settings in File > Print .. under "Range and Copies" is too slow low NEW
95665 Text "Search key not found" vertically misaligned within Find toolbar medium NEW
95680 Convert - Metafile - Save Image... No function. Must use Alt+R medium NEW
95960 UI - Displaced field value widget in User-Defined Document Properties medium NEW
96095 Performance regression when opening "Getting Started" in 5.0+ medium NEW
96176 DRAW: Slow performance opening and editing a file medium NEW
96341 Version 5.1 takes a lot more RAM than version 5.0 in Windows medium NEW
96500 EDITING: flicker during mouse-move of table control (form design mode) medium NEW
96753 KDE: big file opening and navigating is slow vs. 4.3 medium NEW
96826 Typewriter attribute given too much weight when finding font based on attributed medium NEW
96897 Function window always displayed medium NEW
96947 Check marks on menus (Windows 7 Aero theme, OpenGL enabled) miss their borders medium NEW
97129 No feedback while saving a document medium NEW
97226 Can not open Remote Files in Google Drive on Windows XP medium NEW
97231 QueryValue not working as expected medium NEW
97395 FILEOPEN: Libre Office odb crashes when try to open tables high NEW
97548 EDITING: When selecting cells for a formula, Ctrl is incorrectly behaving like Shift medium NEW
97579 Unable to link Access 2010 database causing LireOffice to freeze and requiring force kill high NEW
97768 working with onscreen-keyboard LO misunderstands AltGr+E (Euro) as Alt+E medium NEW
98079 FILESAVE CALC crashes sporadically when saving multiple sheet workbook (Win x64) high NEW
98094 NAVIGATOR: Unable to drag header in 'Content View', if first was tried to drag a header with all Categories visible. medium NEW
98173 Regular quotation marks in italics in Times New Roman overwrite text - hebrew text medium NEW
98243 cell scroll highlighting breaks cell border display when navigating up / down at document magnification of at least 150% low NEW
98349 Handling of PUA mapped "symbol fonts" like Chess fonts, some characters not mapping back into keyboard when used in LO medium NEW
98454 Visual corruption when viewing a document in web view, help documentation, and text based controls medium UNCONFIRMED
98491 Split hsqldb Base application will not start under OSX10.11.x causes LO to freeze requiring force kill high NEW
98546 EDITING sheet links created using a macro are not updated immediately medium NEW
98598 VIEWING: Background image and web layout medium NEW
98967 Paragraph "Fixed" line spacings using Point units are incorrectly being rounded--unable to set spacing to tenths or hundredths of a Point precision medium NEW
98993 Rounding off entries distorts line spacing medium NEW
99071 Navigator heading is Truncated when mouse click medium NEW
99088 Export to PNG messes up with DPI medium NEW
99403 FILESAVE: progress bar intermittent in showing progress of save medium NEW
99685 Slides are flashing buffer of prior slide at the end of "Fade" transition, or end of "Fine Dissolve" transition medium NEW
99707 Impress stalls presentation on 3-liner text block medium NEW
99720 Custom animation "Put on the Brakes" doesnt show correctly in slide show medium NEW
99784 EXTENSION MANAGER - freeze / hang when attempting to install extensions on OSX 10.11 and macOS 10.12 by double-click on oxt file highest NEW
99822 FILEOPEN: Objects in tables located in wrong position medium NEW
99912 Writer creates a new hyperlink when proofing hyperlink text medium NEW
99924 Impress does not display GDI correctly in OSX high NEW
99927 Changing font in certain cases causes visual glitches in UI (eg. blank menus and dialogs) medium NEW
99946 Calendar control shows misaligned day abbreviations (OSX) high NEW
100050 UI / VIEWING: Tango icons for saving flicker while saving medium NEW
100117 LibO crashes when switching Sidebar decks in Writer/Calc if two windows visible (KDE4) medium NEW
100119 Korean Autocorrect does not work in OS X 10.11 medium NEW
100291 When using the Move/Copy Sheet to a different workbook actions are grayed out (only when 1 sheet exists in source document) medium NEW
100477 Report builder generates empty pages high NEW
100611 DATALOSS: Document with chart can be saved as chart, and can't be opened again without manual fix medium NEW
100622 Menu shortcuts don't work with right ALT key medium NEW
100756 FileOpen: Opening documents via LibreOffice (from other applications) medium NEW
100925 When overwriting a cell, new content is displayed on top of the old content until finishing the edit highest NEW
100996 Tools -> Solver... does not display message anymore if no compatible Java installed medium NEW
101293 Bad marker icons in Galaxy and Highcontrast theme low NEW
101305 UI - dark blue selection within template browser as well as within recent documents view medium NEW
101313 Copy-paste a Table With Merged Cells: Loss of formatting (Cells Placed in Wrong Position) medium NEW
101528 Crashes on 3D chart modification high NEEDINFO
101895 Paragraph dialog units shown as "ch" and "lines" when "Enable char unit" is enabled by default medium NEW
101915 F11 shortcut should shift keyboard focus to control sidebar content panel of Styles and Formatting deck medium NEW
101948 Search fails to find specified indentation in imported Word document medium NEW
102156 Start center not navigational by keyboard when launched medium NEW
102320 Draw: Vectorizing text results in garbage output high NEW
102405 Repagination from web view causes TOC to appear on more pages medium NEEDINFO
103056 Endless scrolling upwards after reaching top of document medium NEW
103151 SVG rendered wrong in the presenter console medium NEW
103218 Imported XSLX wrong image boundaries medium NEW
103236 Some Impress slide transitions crashes LO high NEW
103252 macOS: Time to open StartCenter high NEW
103375 MAILMERGE: unpredictable failures: sometimes looses embedded image, or not all mails sent, when e-mailing as pdf medium NEW
103412 Base report builder: "font color" and "background color" buttons in conditional formatting are broken medium NEW
103474 FORMATTING: bracket shape not shown correctly medium NEW
103476 FORMATTING: bad paragraph and line spacing in textbox medium NEW
103493 FILESAVE Write Error when a sheet with images, drawing objects and note/comment captions has been copied twice before itself, or even application crash medium REOPENED
103553 FORMATTING: Writer show zero page borders in specific docx file medium NEW
103567 FILESAVE: Hyperlinks inserted in gallery objects are not persistent in ODP files medium NEW
103570 Java not selected automatically when a database is opened for the first time low UNCONFIRMED
103586 Assigning shortcuts to macro in context LibreOffice (not a module) does not work anymore medium NEW
103658 FILESAVE: Incredibly slow to save on first change after opening medium NEW
103753 create a new sheet while cell is in edit mode medium NEW
103800 FILEOPEN PPTX: Dark red font opens as white in specific text box medium NEW
103806 Slide sorter does bizarre things medium NEW
103830 Form field names not update in python macro medium NEW
103919 Mailmerge e-mail does not send all merged documents high NEW
104020 FILESAVE Saving business cards loses positioning of image medium NEW
104031 CPU usage when hovering over Start Center buttons medium NEW
104110 FILEOPEN RTF: The text within a textbox contained in a drawing canvas is invisible medium NEW
104111 FILEOPEN RTF: Three red dots appear if a textbox is placed in a drawing canvas medium NEW
104225 VIEWING: sluggishness while scrolling medium NEW
104261 VIEWING: Calc red comment indicator is changing shape low NEW
104296 VIEWING: Relatively slow scrolling in GDI mode if a simple GIF-file is embedded medium NEW
104312 Severe lag on drawing canvas in "Draw" high NEW
104381 Missing accessible text-caret-moved events in Writer paragraphs medium NEW
104384 FILEOPEN DOCX: Textboxes in drawing canvas are corrupted medium NEW
104385 FILEOPEN DOCX:VML Cloud Callout Not Imported medium NEW
104421 Input Field of type Function requires double-click to edit instead of single-click low NEW
104423 UI: Edit > Duplicate (Shft+F3) doesn't any longer remember last used values medium NEW
104447 FILEOPEN: Very slow opening of specific file medium NEW
104469 accessibillity features are missing in dialogs medium NEW
104482 FILESAVE: saving progress status bar not displayed while saving medium NEW
104500 ODT file does not print background medium NEW
104568 After being deactivated/hidden, various tool bars can't be activated again (in the same document session) medium NEW
104582 Relatively high DISK I/O for Writer document containing an animated gif medium NEW
104598 Black separator/border lines appear in UI around panes (Side Bar, Navigator) and rulers (part in margin) medium NEW
104655 FILEOPEN ODS: Memory usage for opening file has increased with factor 5 medium NEW
104677 FILEOPEN DOCX: ww8 uses fallback font when rendering WMF to canvas different at LibO5.0 compared to Lib4.4 and Word medium NEW
104716 General performance problems - UI freezes, slow scrolling, etc. in files with big/complex images medium NEW
104741 PDF EXPORT: Extra line added to the output file medium ASSIGNED
104781 FILEOPEN DOCX: Time to open file has increased compared to LibO4463 low NEW
104816 Inserting a user field will lose the condition of section medium NEW
104878 Impress works very slow with large sized GIF medium NEW
104892 Adding a bunch of very small images (jpg/png) will cause a high CPU load low NEW
104901 Image object cumulative placement error affecting display/print & cell anchor behavior with irregular row height medium NEW
104920 Clone Formatting in Writer Does Not Properly Copy Attributes Sometimes medium NEW
104952 Crash printed file with input on printer HP LaserJet 1020 (Printed in pdf normal, empty file printed normal) medium NEW
105003 Rotating a PNG image is quite slow medium NEW
105076 [PRINTING] Replaced Cyrillic font with printing on network printer MFP HP 1212nf medium NEEDINFO
105095 Tab after autonumbering footnote vanishes in docx format - not in odt medium NEW
105114 Conditional formatting dialog closes while editing any condition 2nd time medium NEW
105162 OPENGL: Crash when trying to drag some text in Writer high NEW
105276 New line issue in long equations (no wrap option) medium NEW
105362 FILEEXPORT: Slides nearly black and unreadable when exported to PNG with transparency enabled medium NEW

196 Open; 0 Resolved; 196 Total (0% complete)


In addition all unresolved regressions in LibreOffice might benefit from bibisecting.

Finding bugs already bibisected

For the links to bugzilla queries, please see here.

With the above info added to the bug a developer knows that the regression was introduced between the commits fb754a0df859e30255c25af8fa19bfaa75f257e7 (good) and 2d19e9bb07ccff3134f855812dddfda5c07b1fe4 (bad) on master. A:

git log fb754a0df859e30255c25af8fa19bfaa75f257e7..2d19e9bb07ccff3134f855812dddfda5c07b1fe4

on the source repository will then show the 128 commits including the one that introduced the bug, making it a lot easier for the developer to close in on the culprit. For more details, see:

Special situations

running with a separate user profile

To create separate profile directory in /tmp/ (which will be automatically removed during the next machine boot):

./opt/program/soffice -env:UserInstallation=file:///tmp/soffice-bisect

If you want to start fresh for each bisect iteration, you can add git commit hash to directory name. For example:

./opt/program/soffice \
-env:UserInstallation=file:///tmp/soffice-bisect-$(git rev-parse HEAD)

Finding a fix

Sometimes a bug is a collateral damage of another (possibly minor) bug, in a non obvious way. In such cases that other bug might be fixed on master, but was not backported to the release branch. You can find the fix, just like finding a bug, except that you need to type

git bisect good

for

git bisect bad

and vice versa. If the fix is small and important enough, it likely can be easily backported to the release branch.

Troubleshooting

Please add your problems and how you solved them. This section covers issues that affect more than one OS. For OS-specific issues, please see the individual OS wiki pages.

Unable to extract files

If you can't extract files (you should get an error message). Try to rename the file from *.tar.lzma to *.tar.xz.

Installed LibreOffice became modified unexpectedly

Problem / Messages:

   error: Your local changes to the following files would be overwritten by checkout:
     opt/program/pythonloader.pyc
     opt/program/uno.pyc
     opt/program/unohelper.pyc
   Please, commit your changes or stash them before you can switch branches.
   Aborting

Solutions:

git checkout .

will repair the problem. Afterwards you can start the next test run as usual with

git bisect good # or git bisect bad

If that does not solve the problem, you can try

git clean -fd
git reset --hard

The ultimate repair is to delete everything except one

.git/

hidden folder, and do

git checkout oldest

.


Unable to start soffice

Problem: Upon trying to run soffice within bibisect you receive the error "unexpected operator terminate called after throwing an instance of 'com::sun::star::uno::DeploymentException'

Solution: Reset your libreoffice profile located in ~/.config/libreoffice. Make sure to backup the folder if you have any settings you'd like to preserve for use with your stable libreoffice release. After you're done with bibisect you can try to return your backed up profile to it's original location (ie. delete the new profile folder and replace it with the backed up one)


Further information

notes

For details on how the bibisect archives are generated:

For information on merging all of our bibisect repositories into one big one, see

We want to make a GUI for bibisect: