QA/Bibisect

From The Document Foundation Wiki
< QA(Redirected from QA/HowToBibisect)
Jump to: navigation, search

Introduction

Bibisect stands for "binary bisect". It's a procedure that helps identify commits that have caused regressions. Regressions are a most annoying artifact that unfortunately come with software development and QA. We want to deal with regressions quickly before time passes and they get harder and harder to triage and fix.

How does Bibisecting Work?

Each bibisect package is a git repository populated with hundreds or thousands of builds of LibreOffice. Added to the repository in chronological order, the builds provide a fast and simple way for us to test a bug against many different versions without having to compile LibreOffice each time we switch to a different version.

We can leverage the tools built into git for binary search through commits (e.g. git bisect) as well as benefit from the compression and de-duplication functions included in git. For example, in a series of builds that each might take up 500MB of disk space, through de-duplication we can potentially use only 25MB per additional build.

Watch Effective Bisection and Bibisection (Matthew Francis's talk at LibOCon 2015 on YouTube) for both an introduction, and practical details.

Details

A successful bibisect will leave developers with only a few commits to search through to find the 'culprit' commit that we believe introduced a regression. Depending upon the particular bibisect repository, this range might be 60, 100, or even 2 commits. (We also have some very 'coarse' bibisect repositories that use released versions + beta + RC builds, and these may have ranges spanning multiple hundreds of commits)

QA can go the additional step of fully bisecting (determining the exact commit) or can leave this to the developers. It's often easy to find the commit that caused the issue after bibisecting the bug.

With approximately 10 gigs, you'll have hundreds of builds to work with. A bibisect can take as little as 15 minutes.

Currently, bibisecting is possible on:

General Instructions

Note: Each operating system is different, please read the full instructions that correspond to your specific operating system (GNU/Linux, OS X, or Windows) for more complete instructions, instructional videos, and other related information.

Bibisecting relies on 6 steps:

  1. Setting up your environment, including installing any extra software
  2. Downloading the bibisect package
  3. Bibisecting the bug
  4. Getting a bibisect log
  5. Attaching the bibisect log to the bug report
  6. Updating the bug report fields

Limitations

Performing a bibisect is only possible if we have a repository available for the correct OS, covering the correct range of LibreOffice commits. We have very good coverage on GNU/Linux, and are improving our coverage on Windows and OS X.

Some regressions are difficult to reproduce, and may require multiple test runs for a bug to appear. As you may imagine, trying to bibisect such regressions is very difficult, and requires one to perform many test runs after each checkout of a new version of LibreOffice to give some security / statistics that the bug is indeed present or not present with the given version.

We do not currently have bibisect support for

Versions

Versions are listed on each OS page. See #Details above.

Not Bibisectable

Some bugs are not bibisectable. These include

  • Bugs that don't show up on a supported platform (see #Limitations)
  • Bugs that predate our bibisect builds (We're working on extending the range covered)
  • Bugs that depend upon build options not enabled in the TDF builds (e.g. -kde)

If a bug with bibisectRequest Keyword is found to be not bibisectable, leave a comment on the bug explaining why we're not able to use bibisect to track down the introduction of this problem, and replace Keyword bibisectRequest with notBibisectable.

Finding bugs

Finding bugs needing bibisect

Following Bugs with Keyword bibisectRequest are waiting for a bibisect:

ID Summary Priority Status
43715 Impress: presentation minimizer not reduces resolution of images (regression after 3.3) medium NEW
43822 Ctrl-W does not close document when the focus is in the Navigator medium NEW
51119 EDITING: OLE objects from writer are impossible to manage medium NEW
54703 Hidden Sections are no longer hidden when printing or exporting pdf. medium NEW
56173 FILESAVE impossible for password protected shared .ods medium NEW
60428 Duplex Printing - HP Officejet Pro 8600 Plus medium NEW
61083 Underlines are not imported from EMF medium NEW
64276 SVG: Problem with some svg file. medium NEW
64423 Import of XML Excel file into Calc fails with General input/output error. medium NEW
65535 Adding a comment in the middle of a word makes it recognised as misspelt by spellchecker medium NEW
66054 FORMATTING: font work medium NEW
66237 Document preview of tables shows overlayed toolbar icons and misplaced column headers medium NEW
66617 chart2 unoapi test fails on sch.ChXDiagram::com::sun::star::chart::ChartStatistics::RegressionCurves medium NEW
67744 FORMATTING: Adobe Courier Std font is always bold on Mac OS X medium NEW
68695 EDITING: Comments cannot be resized while editing - missing squares around the comment/note medium NEW
69254 FORMATTING: Custom font handling medium NEW
69461 SLIDESHOW: Inserted Movie files show a white screen for about one frame at the start high NEW
69631 PDF: Diagrams from DOC files not exported to PDF unless clicked in medium NEW
69895 SVG rendering incorrect (primarily placement of elements withing drawing) medium NEW
69932 VIEWING: LibO 4.x.x flicker low NEW
70317 FORMATTING: Awkward kerning of first word in a justified paragraph before an "italicised" text medium NEW
70607 LibO uses 100% CPU and crashes when quit during extension update high NEW
70710 filled areas are not filled for WMF images medium NEW
71828 Distorted eps-exports with LO Draw medium NEW
75218 Letter Wizard should be improved to follow locale customs medium NEW
75260 double line border styles wrongly displayed high NEW
75471 Newly opened LibO document frames stay in background (i. e. LibO does not get focus) medium NEW
75650 Formulas with boolean result change cell format while other formulas keep it medium NEW
75958 FILEOPEN: Badly misplaced graphics in MS Word .doc files when imported into LibreOffice Writer medium NEW
76644 extended tips missing for menu items medium NEW
77444 LO is open, and opening any document by double-click takes a long time requiring mouse movement to display it highest NEW
79063 cell height of spreadsheet printed different after cut/paste to Writer as object when row height set using dialog low NEW
79250 Cell in 'Linked cell...' within "List Box" change to 'Text-@' format medium NEW
79452 Themes with a separator line per toolbar incorrectly places separator line when toolbar is on bottom low NEW
80087 FILESAVE: File Corruption: Roundtrip file gets corrupted medium NEW
80473 Wrong style applied to DOCX paragraph when open in CJK UI (because of wrong style hereditability) medium NEW
80989 FILESAVE: Text-Box is getting lost in round Trip high NEW
82091 Spelling dialog <F7> correction of Calc cells does not remove red wavy underline, but context menu spelling correction does medium NEW
82113 FILEOPEN: Text in EMF not displayed medium NEW
82658 FILEOPEN: A parallelogram in DOCX canvas become a triangle medium NEW
83306 FILEOPEN: Artifacts when importing .docx files with Autoshape images (Windows only) highest NEW
87012 REPORT-BUILDER: Charts without content if chart contain date or time high NEW
87763 Problem with Impress View Presentation Mode medium NEW
87912 EDITING: Image is hidden, when anchor is character and another image is inserted on the same page medium NEW
89147 Calc comment text invisible when using missing font in .xls file medium NEW
89497 EDITING, FORMATTING: Text frame gets lower, sometimes making text invisible, when switching templates medium NEW
89507 FILEOPEN: LibO can't identify which user has the file open medium NEEDINFO
90081 FORMATTING: Document index style has changed and can't be set to the previous one medium NEW
90414 Text inside Folded Corner rendering at Draw is different when export to SVG medium NEW
90580 VBA modules break valid StarBasic code; cannot delete "VBA context" medium NEW
90621 FORMATTING: Custom character styles are not applied in bibliographies medium NEW
90811 Images are not properly placed when viewing attached document high NEW
91115 Gallery Drag & Drop Feature Broken (Ubuntu Unity only) high NEW
91237 CALC hangs on web table paste high NEW
91770 Crash on closing on Windows highest NEW
92052 Black line around focused area within focused widgets medium NEW
92307 FORMATTING: Changing case when all cells are selected does not take effect medium NEW
92380 Style Preview broken some styles in Styles Pane on V5.0 medium NEW
92472 FILEOPEN: First checkbox in row has too large dimensions on docx import medium NEW
92617 Base is crashing frequently when EDITING data and switching Forms high REOPENED
92771 "run macros" dialog shows libraries in wrong section medium NEW
92796 [ODF] Writer does not remove unused page background image/bitmap medium NEW
92909 UI - Table creation field data types are displayed in rounded buttons that do not fill grid cell (OSX only) medium NEW
92928 Same network connection error for each call to GETPIVOTDATA medium NEW
93085 Application window is too high, status bar is not visible (1024x600 resolution), size of module launch inherited from the StartCenter which gets a fixed minimum height (comment 14) high NEW
93191 Sheet Tab Controls Right Click Not Working Correctly medium NEW
93352 RTL: Horizontal scrolling for right-to-left Sheet moves in opposite direction medium NEW
93361 Pivot table column total doesn't work high NEW
93366 Table jumps to the next page when saving DOC file medium NEW
93660 FILEOPEN: Writer cannot open DOCX with large table again medium NEW
93874 FILEOPEN: DOCX import table: colors and structure lost high NEW
94088 FILESAVE: paragraph background colour not saved in HTML file medium ASSIGNED
94186 Let A, B empty cells than A+B/2=2 under certain conditions medium NEW
94259 X-Y-Graphic in Excel File are shown wrong in version 4.4, 4.5, 5.01 medium NEW
94265 bootstrap exception when running C# application with LibreOffice API medium NEW
94448 UI: Draw’s context menu lacks Text option medium NEW
94625 FORMATTING: rotated drawing do not anchored to cell medium NEW
94666 Narrow spacing of menu items, and truncated tooltips with lower resolution low NEW
94737 Poor subpixel rendering on Linux medium NEW
94792 Fileopen: performance regression for xlsx with chart (per Comment 4) medium NEW
94921 Picture at wrong position after sheet copy medium NEW
94942 Corrupt dialog under 5.0.2.2 (Format -> Cells -> Border-> Width) medium NEW
94950 FILEOPEN: text line inside .doc file showed in wrong position medium NEW
95150 Border of tablecontrol doesn't show color medium NEW
95191 Forms and dialogs opened via macros on ODB file open no longer work high NEW
95209 Broken font rendering of some characters in Opus Standard font (was ok in LO4.4.2) medium NEW
95431 Insert big image get black box medium NEW
95476 FILESAFE: Fileexport to xhtml shows display:condition; for section, which is defined as displayed with condition medium NEW
95539 Option + arrow does not work properly when RTL is enabled medium NEW
95658 Repagination caused by changing print settings in File > Print .. under "Range and Copies" is too slow low NEW
95680 Convert - Metafile - Save Image... No function. Must use Alt+R medium NEW
95737 Fontwork Object incompatible with Type 1 fonts medium NEW
95960 UI - Displaced field value widget in User-Defined Document Properties medium NEW
96095 Performance regression when opening "Getting Started" in 5.0+ medium NEW
96176 DRAW: Slow performance opening and editing a file medium NEW
96341 Version 5.1 takes a lot more RAM than version 5.0 in Windows medium NEW
96500 EDITING: flicker during mouse-move of table control (form design mode) medium NEW
96650 VIEWING: If you setup borders of different widths, the edges don't line properly medium NEW
96658 selection of objects in Draw and Impress awfully slow medium NEEDINFO
96748 Default item of Languages/User interface reflects current LibreOffice setting instead of the OS environment medium NEW
96753 KDE: big file opening and navigating is slow vs. 4.3 medium NEW
96826 Typewriter attribute given too much weight when finding font based on attributed medium NEW
96829 CJK Ext-B characters with font PMingLiU-ExtB rendered wrongly, as empty boxes, overdrawn, or not at all medium NEW
96897 Function window always displayed medium NEW
96947 Check marks on menus (Windows 7 Aero theme, OpenGL enabled) miss their borders medium NEW
97070 Localized TABLE formulas written stopped working medium NEW
97122 Modes tab bar activates "Slide Master" when clicking "Normal" medium NEW
97129 No feedback while saving a document medium NEW
97226 Can not open Remote Files in Google Drive on Windows XP medium NEW
97231 QueryValue not working as expected medium NEW
97312 FILEOPEN: Table in a DOC file displayed too wide; outside of page border medium NEW
97337 Background graphics cause artifacts in PDF export medium UNCONFIRMED
97395 FILEOPEN: Libre Office odb crashes when try to open tables high NEW
97548 EDITING: When selecting cells for a formula, Ctrl is incorrectly behaving like Shift medium NEW
97579 Unable to link Access 2010 database causing LireOffice to freeze and requiring force kill high NEW
97581 INSERT: unknown lines appearing in SVG shape medium NEW
97672 Created styles selections invisible. medium NEW
97768 working with onscreen-keyboard LO misunderstands AltGr+E (Euro) as Alt+E medium NEW
97905 Current style does not show up in Sidebar (until clicked) in KDE with integration from 5.1 medium NEW
98079 FILESAVE CALC crashes sporadically when saving multiple sheet workbook (Win x64) high NEW
98085 FILESAVE/OPEN as .xlsx destroys link to other document medium NEW
98094 NAVIGATOR: Unable to drag header in 'Content View', if first was tried to drag a header with all Categories visible. medium NEW
98173 Regular quotation marks in italics in Times New Roman overwrite text - hebrew text medium NEW
98243 cell scroll highlighting breaks cell border display when navigating up / down at document magnification of at least 150% low NEW
98285 FORMATTING: Paragraph background colour in table cells cannot be changed and saved, if whole cell was not selected medium NEW
98349 Handling of PUA mapped "symbol fonts" like Chess fonts, some characters not mapping back into keyboard when used in LO medium NEW
98454 Visual corruption when viewing a document in web view, help documentation, and text based controls medium UNCONFIRMED
98491 Split hsqldb Base application will not start under OSX10.11.x causes LO to freeze requiring force kill high NEW
98546 EDITING sheet links created using a macro are not updated immediately medium NEW
98598 VIEWING: Background image and web layout medium NEW
98967 Paragraph "Fixed" line spacings using Point units are incorrectly being rounded--unable to set spacing to tenths or hundredths of a Point precision medium NEW
98993 Rounding off entries distorts line spacing medium NEW
99071 Navigator heading is Truncated when mouse click medium NEW
99088 Export to PNG messes up with DPI medium NEW
99158 GDI metafiles not displayed correctly medium NEW
99210 Sluggish Editing of Text Boxes medium NEW
99265 If done after cropping, compress displays image stretched medium NEW
99386 Undo with merged cell, needs hard recalculate, no automatic update medium NEW
99403 FILESAVE: progress bar intermittent in showing progress of save medium NEW
99537 SIDEBAR: Content panel setting not saved between context changes medium NEW
99607 High CPU load with filter active high NEW
99631 EDITING: DOCX with embedded XLSX loaded but incorrectly scaled when edit medium NEW
99707 Impress stalls presentation on 3-liner text block medium NEW
99784 EXTENSION MANAGER - freeze / hang when attempting to install extensions on OSX 10.11 by double-click on oxt file highest NEEDINFO
99912 Writer creates a new hyperlink when proofing hyperlink text medium NEW
99924 Impress does not display GDI correctly in OSX high NEW
99927 Changing font in certain cases causes visual glitches in UI (eg. blank menus and dialogs) medium NEW
99946 Calendar control shows misaligned day abbreviations (OSX) high NEW
100007 Progress slider and progress status controls don't update during playback medium NEW
100050 UI / VIEWING: Tango icons for saving flicker while saving medium NEW
100119 Korean Autocorrect does not work in OS X 10.11 medium NEW
100291 When using the Move/Copy Sheet to a different workbook actions are grayed out (only when 1 sheet exists in source document) medium NEW
100300 Crash upon start (LibreOffice recovers, but there's an (empty) Document Recovery dialog ..) medium NEW
100477 Report builder generates empty pages high NEW
100611 DATALOSS: Document with chart can be saved as chart, and can't be opened again without manual fix medium NEW
100622 Menu shortcuts don't work with right ALT key medium NEW
100626 SID_NEWDOCDIRECT / .uno:AddDirect assigned N_MOD1 and File menu action, opens a new document but does not direct edit cursor focus into the document medium NEW
100627 Create a table, and the cursor disappears medium NEW
100756 FileOpen: Opening documents via LibreOffice (from other applications) medium NEW
100813 libreoffice crashes immediately after openning this .docx file: medium NEW
100925 When overwriting a cell, new content is displayed on top of the old content until finishing the edit highest NEW
101038 Error when previewing print or printing medium NEEDINFO
101293 Bad marker icons in Galaxy and Highcontrast theme low NEW
101305 UI - dark blue selection within template browser as well as within recent documents view medium NEW
101317 File saved with wrong mime type medium NEW
101364 “Untitled” not translated in Chinese Simplified UI medium NEW
101413 FILEOPEN: Missing checkboxes on DOCX import medium NEW
101433 Embedded StarView metafile is corrupted when inserted into document canvas medium NEW
101470 Drop down Apply Style (formatting toolbar) doesn't allow (any more) to edit a style that is not applied in the text having the focus low NEW
101475 Format > Cells window unnaturally wide in v5.2 (Calc) low NEW
101520 Moving or resizing playing or paused media clip leaves impression in original place medium NEW
101583 Range name references updated incorrectly (for single row/column ranges) medium NEW
101598 Not possible to add a background music on Impress presentation medium NEW
101638 [FILEOPEN,DOC] Bullets in a graph are too large medium NEW
101711 Background image turns to solid blue for all slides after copy/paste medium NEW
101798 VIEWING: Text boundaries broken when text is entered high NEW
101832 Contour Editor no longer centers image in the dialog medium UNCONFIRMED
101873 Searching for text doesn't find it the first time after searching for text that is not in the document medium NEW
101884 User defined border control has problems with keyboard navigation medium NEW
101895 Paragraph dialog units shown as "ch" and "lines" when "Enable char unit" is enabled by default medium NEW
101898 Saving a presentation on Ubuntu 16.04 hangs (regression) high NEW
101915 F11 shortcut should shift keyboard focus to control sidebar content panel of Styles and Formatting deck medium NEW
101953 Controls in the dialog Bullets and Numbering, tab Options, become wider/smaller when switching from numbers to graphics medium NEW
101988 LOOKUP of null value returns #NA medium REOPENED
102152 Setting "bookmarks and page" pdf export option not working medium UNCONFIRMED
102177 Document focus lost when context menu opened medium NEW
102279 SLIDE PANE: White line on the right and bottom of preview medium UNCONFIRMED
102320 Draw: Vectorizing text results in garbage output high NEW
102334 Fileopen DOC: hidden text with table inserted afterwards shouldn't be visible medium UNCONFIRMED
102343 Unable to undo text formatting in textbox edit mode medium NEW
102377 FILESAVE: Saving as XLSX resulted in Corrupted file medium UNCONFIRMED
102405 Repagination from web view causes TOC to appear on more pages medium UNCONFIRMED

192 Open; 0 Resolved; 192 Total (0% complete)


In addition all unresolved regressions in LibreOffice might benefit from bibisecting.

Finding bugs already bibisected

For the links to bugzilla queries, please see here.

With the above info added to the bug a developer knows that the regression was introduced between the commits fb754a0df859e30255c25af8fa19bfaa75f257e7 (good) and 2d19e9bb07ccff3134f855812dddfda5c07b1fe4 (bad) on master. A:

git log fb754a0df859e30255c25af8fa19bfaa75f257e7..2d19e9bb07ccff3134f855812dddfda5c07b1fe4

on the source repository will then show the 128 commits including the one that introduced the bug, making it a lot easier for the developer to close in on the culprit. For more details, see:

Special situations

running with a separate user profile

To create separate profile directory in /tmp/ (which will be automatically removed during the next machine boot):

./opt/program/soffice -env:UserInstallation=file:///tmp/soffice-bisect

If you want to start fresh for each bisect iteration, you can add git commit hash to directory name. For example:

./opt/program/soffice \
-env:UserInstallation=file:///tmp/soffice-bisect-$(git rev-parse HEAD)

Finding a fix

Sometimes a bug is a collateral damage of another (possibly minor) bug, in a non obvious way. In such cases that other bug might be fixed on master, but was not backported to the release branch. You can find the fix, just like finding a bug, except that you need to type

git bisect good

for

git bisect bad

and vice versa. If the fix is small and important enough, it likely can be easily backported to the release branch.

Troubleshooting

Please add your problems and how you solved them. This section covers issues that affect more than one OS. For OS-specific issues, please see the individual OS wiki pages.

Unable to extract files

If you can't extract files (you should get an error message). Try to rename the file from *.tar.lzma to *.tar.xz.

Installed LibreOffice became modified unexpectedly

Problem / Messages:

   error: Your local changes to the following files would be overwritten by checkout:
     opt/program/pythonloader.pyc
     opt/program/uno.pyc
     opt/program/unohelper.pyc
   Please, commit your changes or stash them before you can switch branches.
   Aborting

Solutions:

git checkout .

will repair the problem. Afterwards you can start the next test run as usual with

git bisect good # or git bisect bad

If that does not solve the problem, you can try

git clean -fd
git reset --hard

The ultimate repair is to delete everything except one

.git/

hidden folder, and do

git checkout oldest

.


Unable to start soffice

Problem: Upon trying to run soffice within bibisect you receive the error "unexpected operator terminate called after throwing an instance of 'com::sun::star::uno::DeploymentException'

Solution: Reset your libreoffice profile located in ~/.config/libreoffice. Make sure to backup the folder if you have any settings you'd like to preserve for use with your stable libreoffice release. After you're done with bibisect you can try to return your backed up profile to it's original location (ie. delete the new profile folder and replace it with the backed up one)


Further information

notes

For details on how the bibisect archives are generated:

For information on merging all of our bibisect repositories into one big one, see

We want to make a GUI for bibisect: