QA/Bibisect

From The Document Foundation Wiki
< QA(Redirected from QA/HowToBibisect)
Jump to: navigation, search

Introduction

Bibisect stands for "binary bisect". It's a procedure that helps identify commits that have caused regressions. Regressions are a most annoying artifact that unfortunately come with software development and QA. We want to deal with regressions quickly before time passes and they get harder and harder to triage and fix.

How does Bibisecting Work?

Each bibisect package is a git repository populated with hundreds or thousands of builds of LibreOffice. Added to the repository in chronological order, the builds provide a fast and simple way for us to test a bug against many different versions without having to compile LibreOffice each time we switch to a different version.

We can leverage the tools built into git for binary search through commits (e.g. git bisect) as well as benefit from the compression and de-duplication functions included in git. For example, in a series of builds that each might take up 500MB of disk space, through de-duplication we can potentially use only 25MB per additional build.

Watch Effective Bisection and Bibisection (Matthew Francis's talk at LibOCon 2015 on YouTube) for both an introduction, and practical details.

Details

A successful bibisect will leave developers with only a few commits to search through to find the 'culprit' commit that we believe introduced a regression. Depending upon the particular bibisect repository, this range might be 60, 100, or even 2 commits. (We also have some very 'coarse' bibisect repositories that use released versions + beta + RC builds, and these may have ranges spanning multiple hundreds of commits)

QA can go the additional step of fully bisecting (determining the exact commit) or can leave this to the developers. It's often easy to find the commit that caused the issue after bibisecting the bug.

With approximately 10 gigs, you'll have hundreds of builds to work with. A bibisect can take as little as 15 minutes.

Currently, bibisecting is possible on:

General Instructions

Note: Each operating system is different, please read the full instructions that correspond to your specific operating system (GNU/Linux, OS X, or Windows) for more complete instructions, instructional videos, and other related information.

Bibisecting relies on 6 steps:

  1. Setting up your environment, including installing any extra software
  2. Downloading the bibisect package
  3. Bibisecting the bug
  4. Getting a bibisect log
  5. Attaching the bibisect log to the bug report
  6. Updating the bug report fields

Limitations

Performing a bibisect is only possible if we have a repository available for the correct OS, covering the correct range of LibreOffice commits. We have very good coverage on GNU/Linux, and are improving our coverage on Windows and OS X.

Some regressions are difficult to reproduce, and may require multiple test runs for a bug to appear. As you may imagine, trying to bibisect such regressions is very difficult, and requires one to perform many test runs after each checkout of a new version of LibreOffice to give some security / statistics that the bug is indeed present or not present with the given version.

We do not currently have bibisect support for

Versions

Versions are listed on each OS page. See #Details above.

Not Bibisectable

Some bugs are not bibisectable. These include

  • Bugs that don't show up on a supported platform (see #Limitations)
  • Bugs that predate our bibisect builds (We're working on extending the range covered)
  • Bugs that depend upon build options not enabled in the TDF builds (e.g. -kde)

If a bug with bibisectRequest Keyword is found to be not bibisectable, leave a comment on the bug explaining why we're not able to use bibisect to track down the introduction of this problem, and replace Keyword bibisectRequest with notBibisectable.

Finding bugs

Finding bugs needing bibisect

Following Bugs with Keyword bibisectRequest are waiting for a bibisect:

Full Query
ID Summary Priority Status
43715 Impress: presentation minimizer not reduces resolution of images (regression after 3.3) medium NEW
43822 Ctrl-W does not close document when the focus is in the Navigator medium NEW
47471 Clear Direct Formatting damages the document's styles medium NEW
51119 EDITING: OLE objects from writer are impossible to manage medium NEW
54551 FILEOPEN: long time to load .DOC file medium NEW
54703 Hidden Sections are no longer hidden when printing or exporting pdf. medium NEW
60428 Duplex Printing - HP Officejet Pro 8600 Plus medium NEW
64423 Import of XML Excel file into Calc fails with General input/output error. medium NEW
65200 FORMATTING: Autofit width not behave properly if cell(s) contain text wrapping medium NEW
65535 Adding a comment in the middle of a word makes it recognised as misspelt by spellchecker medium NEW
66054 FORMATTING: font work medium NEW
68784 cell comment pop-up window doesn't disappears completely when mouse moves away medium NEW
69069 [EDITING, VIEWING] 100% CPU while Scrolling and copying Columns medium NEW
69254 LibreOffice unable to correctly handle regular font weights/styles on Mac medium NEW
69461 SLIDESHOW: Inserted Movie files show a white screen for about one frame at the start high NEW
69479 Formula is getting corrupted after input medium NEW
69932 VIEWING: LibO 4.x.x flicker low NEW
70607 LibO uses 100% CPU and crashes when quit during extension update high NEW
70710 filled areas are not filled for WMF images medium NEW
70838 FILESAVE: doc: wrong size and position exporting rotated shapes medium NEW
71828 Distorted eps-exports with LO Draw medium NEW
74179 .xls table linked in .ppt file is not correctly printed or exported to .pdf medium NEW
75218 Letter Wizard should be improved to follow locale customs medium NEW
75260 double line border styles wrongly displayed high NEW
75331 FILEOPEN: DOCX - table has a text wrapping of 'around' (see Comment 15 and Comment 20) medium NEW
76644 extended tips missing for menu items medium NEW
77444 LO closed then opening any document by double-click never opens the file. mouse movement required for the file to open highest NEW
77929 Fill series should detect the last cell to be filled, rather than fill to row #1648576 medium NEW
79063 cell height of spreadsheet printed different after cut/paste to Writer as object when row height set using dialog low NEW
79452 Themes with a separator line per toolbar incorrectly places separator line at bottom when toolbar docked at bottom low NEW
81943 FILEOPEN: RTF - Image should be placed behind table (wrap through-in background) medium NEW
82091 Spelling dialog <F7> correction of Calc cells does not remove red wavy underline, but context menu spelling correction does medium NEW
82113 FILEOPEN: EmfPlusRecordTypeDrawDriverString Text in EMF not displayed medium ASSIGNED
86321 EDITING, FORMATTING: diagram didn't automatic update when change variable (recalculation not triggered) highest NEW
86919 Writer loses focus when toggling between full screen mode medium NEW
87165 FILEOPEN: text missing in tables in .rtf medium NEW
87218 FILEOPEN: table displayed in wrong position in .rtf medium NEW
87219 FILEOPEN: line at the top displayed at the bottom in .rtf medium NEW
87680 No accessible caret-moved events in Writer tables medium NEW
87912 EDITING: Image is hidden, when anchor is character and another image is inserted on the same page medium NEW
89497 EDITING, FORMATTING: Text frame gets lower, sometimes making text invisible, when switching templates medium NEW
89696 unknown software exception 0x40000015 at address 0x0036327c high NEW
90414 Text inside Folded Corner rendering at Draw is different when export to SVG medium NEW
90580 VBA modules break valid StarBasic code; cannot delete "VBA context" medium NEW
90621 FORMATTING: Custom character styles are not applied in bibliographies medium NEW
91237 CALC hangs on web table paste (intermittent) high NEW
91292 FORMATTING: DOCX, Paragraph background color set to 'No Fill' not persisted on save medium ASSIGNED
92617 Base is crashing frequently when EDITING data and switching Forms high REOPENED
92796 [ODF] Writer does not remove unused page background image/bitmap medium NEW
92909 UI - Table creation field data types are displayed in rounded buttons that do not fill grid cell (OSX only) low NEW
92928 Same network connection error for each call to GETPIVOTDATA medium NEW
92960 Calc doesn't merge cells when pasting merged content from HTML medium NEW
93085 Application window is too high, status bar is not visible (1024x600 resolution), size of module launch inherited from the StartCenter which gets a fixed minimum height (comment 14) high NEW
93361 Pivot table column total doesn't work high NEEDINFO
93660 FILEOPEN: Writer cannot open DOCX with large table again (writerfilterlo.dll) medium NEW
93977 FILEOPEN DOC: Layout of the table of contents looks different in LibO compared to Word medium NEW
94259 X-Y-Graphic in Excel File are shown wrong in version 4.4, 4.5, 5.01 medium NEW
94265 bootstrap exception when running C# application with LibreOffice API medium REOPENED
94666 Narrow spacing of menu items, and truncated tooltips with lower resolution low NEW
94737 Poor subpixel rendering on Linux medium NEW
94792 Fileopen: performance regression for xlsx with chart (per Comment 4) medium NEW
94826 Big image in header completely freezes modern hardware medium NEW
95150 Border of tablecontrol doesn't show color medium NEW
95191 Forms and dialogs opened via macros on ODB file open no longer work high NEW
95476 FILESAFE: Fileexport to xhtml shows display:condition; for section, which is defined as displayed with condition medium NEW
95539 Option + arrow does not work properly when RTL is enabled medium NEW
95658 Repagination caused by changing print settings in File > Print .. under "Range and Copies" is too slow low NEW
95680 Convert - Metafile - Save Image... No function. Must use Alt+R medium NEW
96176 DRAW: Slow performance opening and editing a file medium NEW
96341 Version 5.1 takes a lot more RAM than version 5.0 in Windows medium NEW
96500 EDITING: flicker during mouse-move of table control (form design mode) medium NEW
96753 KDE: big file opening and navigating is slow vs. 4.3 medium NEW
96826 Typewriter attribute given too much weight when finding font based on attributed medium NEW
97129 No feedback while saving a document medium NEW
97231 QueryValue not working as expected medium NEW
97266 Calc goes too far selecting range in chart medium NEW
97395 FILEOPEN: Libre Office odb crashes when try to open tables from Access MDB file high NEW
97768 working with onscreen-keyboard LO misunderstands AltGr+E (Euro) as Alt+E medium NEW
97905 Current style does not show up in Sidebar (until clicked) in KDE with integration from 5.1 medium NEW
98017 PRINTING: Multiple Notes Per Page Bug medium NEW
98079 FILESAVE CALC crashes sporadically when saving multiple sheet workbook (Win x64) high NEW
98094 NAVIGATOR: Unable to drag header in 'Content View', if first was tried to drag a header with all Categories visible. medium NEW
98243 cell scroll highlighting breaks cell border display when navigating up / down at document magnification of at least 150% low NEW
98491 Split hsqldb Base application will not start under OSX10.11.x causes LO to freeze requiring force kill high NEW
98546 EDITING sheet links created using a macro are not updated immediately medium NEW
98598 VIEWING: Background image and web layout medium NEW
98665 Saving document is quite a bit slower than before medium NEW
99088 Export to PNG messes up with DPI medium NEW
99403 FILESAVE: progress bar intermittent in showing progress of save medium NEW
99685 Slides are flashing buffer of prior slide at the end of "Fade" transition, or end of "Fine Dissolve" transition medium NEW
99707 Impress stalls presentation on 3-liner text block medium NEW
99838 EDITING: Converting text to curve doesn't convert characters not found in font. medium NEW
99912 Writer creates a new hyperlink when proofing hyperlink text medium NEW
100050 UI / VIEWING: Tango icons for saving flicker while saving medium NEW
100119 Korean Autocorrect does not work in OS X 10.11 medium NEW
100477 Report builder generates empty pages high NEW
100756 FileOpen: Opening documents via LibreOffice (from other applications) medium NEW
100925 When overwriting a cell, new content is displayed on top of the old content until finishing the edit highest NEW
100937 Description: LO Freezed when I insert a very long text in the Description box medium NEW
100961 After inserting Fixed Date, and saving in DOC format, the date is not fixed anymore medium NEW
101224 VIEWING Cell view not updated with split screen and macros medium NEW
101293 Bad marker icons in Galaxy and Highcontrast theme low NEW
101895 Paragraph dialog units shown as "ch" and "lines" when "Enable char unit" is enabled by default medium NEW
101915 F11 shortcut should shift keyboard focus to control sidebar content panel of Styles and Formatting deck medium NEW
103056 Endless scrolling upwards after reaching top of document medium NEW
103151 SVG rendered wrong in the presenter console medium NEW
103218 Imported XSLX wrong image boundaries medium NEW
103312 Using format brush to copy conditional formats to some rows generates thousands (?) additional conditional formats making the spreadsheet unusable slow high NEW
103375 MAILMERGE: unpredictable failures: sometimes looses embedded image, or not all mails sent, when e-mailing as pdf medium NEW
103476 FORMATTING: bad paragraph and line spacing in textbox medium NEW
103553 FORMATTING: Writer show zero page borders in specific docx file medium NEW
103570 Java not selected automatically when a database is opened for the first time low NEW
103586 Assigning shortcuts to macro in context LibreOffice (not a module) does not work anymore medium NEW
103806 Slide sorter does bizarre things medium NEW
103919 Mailmerge e-mail does not send all merged documents high NEW
104020 FILESAVE Saving business cards loses positioning of image medium NEW
104031 CPU usage when hovering over Start Center buttons medium NEW
104110 FILEOPEN RTF: The text within a textbox contained in a drawing canvas is invisible medium NEW
104111 FILEOPEN RTF: Three red dots appear if a textbox is placed in a drawing canvas medium NEW
104225 VIEWING: sluggishness while scrolling medium NEW
104261 VIEWING: Calc red comment indicator is changing shape low NEW
104384 FILEOPEN DOCX: Textboxes in drawing canvas are corrupted medium NEW
104421 Input Field of type Function requires double-click to edit instead of single-click low NEW
104469 accessibillity features are missing in dialogs medium NEW
104500 ODT file does not print background medium NEW
104677 FILEOPEN DOCX: ww8 uses fallback font when rendering WMF to canvas different at LibO5.0 compared to Lib4.4 and Word medium NEW
104716 General performance problems - UI freezes, slow scrolling, etc. in files with big/complex images medium NEW
104802 Performance - jerky cursor behavior, lags key strokes, 500mb memory use high NEW
104816 Inserting a user field will lose the condition of section medium NEW
104892 Adding a bunch of very small images (jpg/png) will cause a high CPU load low NEW
104901 Image object cumulative placement error affecting display/print & cell anchor behavior with irregular row height medium NEW
105328 Undoing adding a table into a single table cell, will collapse the table cell height medium NEW
105444 Extra blank lines inserted at end of Comments in DOCX on save when text added in document before comment (steps in Comment 8) low NEW
105487 Screen readers have trouble reading tables that span multiple pages medium NEW
105489 Scaling algorithm makes icons look ugly high NEW
105541 Charts rendering stutters on bargraphs + hatched-areas medium NEW
105580 Redraw problems when using fonts with long underlengths (see comment 6) medium NEW
105690 FILESAVE: Button dissapears after saving in Calc medium NEW
105714 Link dialog icons and fields are too big medium NEW
105799 Undoing applied table style (aka autoformat) doesnt return table text formatting back to original medium NEW
105881 Problem with <Shift>+TAB keyboard Navigation on the Standard toolbar medium NEW
106074 'Link to External Data...' data duplicated in new cells on opening file and updating link high NEW
106113 EDITING: Text in cells has different vertical alignment when being edited medium NEW
106154 Extremely slow basic operations on macOS high NEW
106484 UI: Print Properties Device tab dropdown items not applied or remembered medium NEW
106611 EDITING: click on status bar does not change the overwrite mode medium NEW
106617 Writer memory leak medium NEW
106651 PDF/A documents fail validation - ISO 19005-1:2005 violations (see comment #5) medium NEW
106703 Poor Performance on macOS 10.12.3 high NEW
106898 Flash during slide transition with GTK VCL plugin (not with GTK3) medium NEW
106958 Documents continually freeze for several seconds. Happens every few minutes. high NEW
106959 MAILMERGE: if output saved as individual documents the second document is identical to the first one if document starts with table low NEW
106962 Clicking text or text box on macOS Sierra results in sluggish UI high NEW
106991 Highlighting remains after select and no fill. medium NEW
107039 Crash on Close after acknowledging save changes - mutex issue (steps in comment 21) high NEW
107248 Improvements to the version control dialog medium NEW
107305 EDITING: Blue page break line disappears after updating TOC (and reappears after scrolling) medium NEW
107376 Displaying false preview when using pptx files. medium NEW
107391 FILEOPEN: Images set as 'no wrap' appear floating over text medium NEW
107435 Non-exact paragraph line spacing has problems for CTL and CJK fonts medium NEW
107443 FILEOPEN: "Open" in "Document in Use" dialog for non-default file format documents opens read-only medium NEW
107511 Performance regression in hunspell medium NEW
107574 Blue throbbing highlight of comments appear over comment at different window views medium NEW
107913 Text animation order in Impress slide show wrong for hypelinked text since LO 3.6 medium NEW
107944 BASIC IDE makes useless, annoying refresh while selecting text medium NEW
107966 OPENGL: Format Cells/Borders/Line Styles - not drawn medium NEW
107972 Slow scrolling/data entry/line insertion - deletion after line 1000 or so medium NEW
107987 EDITING: Resizing an image is slow depending on slide design medium NEW
108000 FILEOPEN DOC: File is opening very slow medium NEW
108235 Pivot table update on undo lags a step behind medium NEW
108266 FILESAVE: RTF: paragraphs preceded by endnote with custom character lose formatting medium NEW
108370 Impress Crash with videos high NEW
108532 EDITING: It takes a while before the document gets responsive after saving medium NEW
108537 EDITING: Selecting images within more or less complex document has a small delay medium NEW
108560 EDITING: Slow Edit or Open Large Cells medium NEW
108573 EDITING: No 'preview' while moving complex drawing object medium NEW
108583 MAILMERGE: Time required to save individual mailmerge documents has increased medium NEW
108637 Memory usage increases considerably when deleting rows in a document with a 1000 empty sheets (see comment 3) medium NEW
108638 FORMATTING Text size is not scaled correctly according to the zoom factor medium NEW
108757 PDF Export of a particular spreadsheet is slower than it has been before medium NEW
108911 Cannot toggle cursor focus between formula and worksheet medium NEW
108937 Copy Style Tool makes CPU go to 100 % high NEW
108969 Saving should warn about losing signatures medium NEW
109062 Two-finger scroll with touchpad is not working on Mac medium NEW
109132 FILESAVE: ODS: VBA Macro name in Cell are converted to lowercase and macro is unknown medium NEW
109353 Listbox doesn't show value (background) if value is just selected (Linux/KDE) medium NEW
111394 Bad export from Draw to SVG-format - not similar export to PNG-format high NEW
111522 SdrUndoMoveObj::Undo() Crash When multi windows medium NEW
111874 Regression: glitch in fade->smoothly transition in 5.4 wrt 5.3 (linux, intel graphics) medium NEW
111936 CHARTS: Bad display on data label in percent stacked chart medium NEW
112081 UI: Checkbox labels getting cropped in various language setting options medium NEW
112313 FILESAVE Text below image moves up next to it when file saved as DOCX medium NEW
112351 Cursor does not move up in long word after removing paragraph break medium NEW
112568 Slow typing in Writer after inserting png image, image repaints all the time medium NEW
112811 FILESAVE: DOCX - Dots separator in index not fully preserved medium NEW
112950 PDF: Hebrew characters overlapping or very close together with David CLM font medium NEW
113014 FILEOPEN DOCX Numbering in a table is displayed outside the visible area medium NEW
113100 Crash in: libc-2.19.so medium NEW
113170 Japanese vertical text moves up its container. medium NEW
113228 Untranslatable UI tokens in sidebar Calc medium NEW
113351 Navigator help missing nv02.png (or a link) from helpimg icon set medium NEW
113500 AutoFill with combined cell content incorrectly increments value when starting from the same content medium NEW
113539 FILEOPEN ODT: Slower opening of file containing a large table (25 rows 2000 columns) medium NEW
113594 Glue points can not be selected and therefore not edited (move, delete, orientation) medium NEW
113616 Mouse pointer actions are offset vertically to actual mouse location (Win10, with OpenGL enabled) medium NEW
113620 Conversion of Draw documents to html fails with command line. medium NEW
113622 Impress can't play MP4 videos highest NEW
113623 FILESAVE ODT: Error saving the document. Write Error. Error in writing sub-document styles.xml. medium NEEDINFO
113721 One CPU core stuck at 100% after saving a file medium NEW
113785 On filtered list an autofill upwards erases data. medium NEW
113849 FILESAVE: Saving a file as .docx format adds spurious extra page breaks medium NEW
113984 Content of a sub-table in another table is hidden by page footer when breaking across pages medium NEW
114022 Selection in Input Line, of cell containing also an email address as link, is not synchronized properly in the cell medium NEW
114023 Insert -> Field -> More Fields, or <ctrl>+<f2>, Fields dialog showing incorrect selection listbox contents high NEW

214 Total; 214 Open (100%); 0 Resolved (0%); 0 Verified (0%);


In addition all unresolved regressions in LibreOffice might benefit from bibisecting.

Finding bugs already bibisected

For the links to bugzilla queries, please see here.

With the above info added to the bug a developer knows that the regression was introduced between the commits fb754a0df859e30255c25af8fa19bfaa75f257e7 (good) and 2d19e9bb07ccff3134f855812dddfda5c07b1fe4 (bad) on master. A:

git log fb754a0df859e30255c25af8fa19bfaa75f257e7..2d19e9bb07ccff3134f855812dddfda5c07b1fe4

on the source repository will then show the 128 commits including the one that introduced the bug, making it a lot easier for the developer to close in on the culprit. For more details, see:

Special situations

running with a separate user profile

To create separate profile directory in /tmp/ (which will be automatically removed during the next machine boot):

./opt/program/soffice -env:UserInstallation=file:///tmp/soffice-bisect

If you want to start fresh for each bisect iteration, you can add git commit hash to directory name. For example:

./opt/program/soffice \
-env:UserInstallation=file:///tmp/soffice-bisect-$(git rev-parse HEAD)

Finding a fix

Sometimes a bug is a collateral damage of another (possibly minor) bug, in a non obvious way. In such cases that other bug might be fixed on master, but was not backported to the release branch. You can find the fix, just like finding a bug, except that you need to type
git bisect good
for
git bisect bad
and vice versa. If the fix is small and important enough, it likely can be easily backported to the release branch.

Troubleshooting

Please add your problems and how you solved them. This section covers issues that affect more than one OS. For OS-specific issues, please see the individual OS wiki pages.

Unable to extract files

If you can't extract files (you should get an error message). Try to rename the file from *.tar.lzma to *.tar.xz.

Installed LibreOffice became modified unexpectedly

Problem / Messages:

   error: Your local changes to the following files would be overwritten by checkout:
     opt/program/pythonloader.pyc
     opt/program/uno.pyc
     opt/program/unohelper.pyc
   Please, commit your changes or stash them before you can switch branches.
   Aborting

Solutions:

git checkout .

will repair the problem. Afterwards you can start the next test run as usual with

git bisect good # or git bisect bad

If that does not solve the problem, you can try

git clean -fd
git reset --hard
The ultimate repair is to delete everything except one
.git/
hidden folder, and do
git checkout oldest
.


Unable to start soffice

Problem: Upon trying to run soffice within bibisect you receive the error "unexpected operator terminate called after throwing an instance of 'com::sun::star::uno::DeploymentException'

Solution: Reset your libreoffice profile located in ~/.config/libreoffice. Make sure to backup the folder if you have any settings you'd like to preserve for use with your stable libreoffice release. After you're done with bibisect you can try to return your backed up profile to it's original location (ie. delete the new profile folder and replace it with the backed up one)


Further information

notes

For details on how the bibisect archives are generated:

For information on merging all of our bibisect repositories into one big one, see

We want to make a GUI for bibisect: