JM/Bibisect

From The Document Foundation Wiki
< QA
Jump to navigation Jump to search
This page is a translated version of the page QA/Bibisect and the translation is 17% complete.
Other languages:
Bahasa Indonesia • ‎English • ‎dansk • ‎русский • ‎中文(简体)‎ • ‎日本語

Pengantar

Bibisect merupakan singkatan dari "binary bisect", yaitu prosedur yang membantu mengidentifikasi commit yang menyebabkan regresi. Regresi adalah artefak yang paling menjengkelkan yang sayangnya muncul serta bersamaan dengan pengembangan perangkat lunak dan PK. Kita ingin menangani regresi secepat mungkin karena seiring waktu berlalu, semakin sulit untuk melakukan triase dan perbaikan.

Bagaimana Bibisecting Bekerja?

Setiap paket bibisect adalah repositori git yang diisi dengan ratusan atau ribuan build LibreOffice. Ditambahkan ke repositori dalam urutan kronologis, build ini menyediakan cara cepat dan sederhana bagi kita untuk menguji kutu pada banyak versi berbeda tanpa harus mengkompilasi LibreOffice setiap kali kita beralih ke versi yang berbeda.

Kita dapat memanfaatkan alat-alat yang dibangun di dalam git untuk pencarian biner melalui commit (misalnya git bisect) serta mendapatkan manfaat dari fungsi kompresi dan de-duplikasi yang termasuk dalam git. Misalnya, dalam serangkaian build yang masing-masing mungkin membutuhkan ruang disk 500MB, melalui de-duplikasi kita berpotensi hanya menggunakan 25MB per build tambahan.

Tonton Bisection dan Bibisection yang Efektif (Presentasi Matthew Francis pada LibOCon 2015 di YouTube) sebagai pengantar, dan detail praktis.

Rincian

Bibisect yang berhasil akan membantu pengembang hanya memiliki sedikit commit untuk dicari untuk menemukan commit 'pelaku' yang kita yakini memperkenalkan regresi. PK dapat melakukan langkah tambahan membagi dua sepenuhnya (menentukan commit yang tepat) atau dapat menyerahkan ini kepada pengembang. Seringkali mudah untuk menemukan commit yang menyebabkan masalah setelah melakukan bibisect kutu.

Dengan sekitar 10 giga, Anda akan memiliki ratusan build untuk dikerjakan. Bibisect dapat memakan waktu hanya 15 menit.

Saat ini, bibisecting dimungkinkan pada:

Seperti yang dapat dilihat dari Commit dalam kisaran dan Jumlah build, tergantung pada repositori bibisect tertentu, kisaran ini mungkin rata-rata 80 commit untuk repositori "43max" lama (mencapai ratusan dari commit sumber dalam beberapa build) menjadi hanya 1 atau 2 commit untuk repositori yang lebih baru. (Kami juga memiliki beberapa repositori bibisect yang sangat 'kasar' yang menggunakan versi yang dirilis + beta + RC builds, dan ini mungkin memiliki rentang yang mencakup ratusan commit).

Jika commit build atau rentang ditemukan, kutu ditandai dengan kata kunci "bibisected", dan jika sumber commit ditemukan, kutu juga ditandai "bisected".

Petunjuk Umum

Bisa jadi ide yang baik bagi pemula untuk memulai dengan menelusuri langkah-langkah bibisect yang sudah selesai dengan sukses. Untuk tujuan ini kami memiliki tutorial bibisecting.

Catatan: Setiap sistem operasi berbeda, harap baca instruksi lengkap yang sesuai dengan sistem operasi spesifik Anda (GNU/Linux, macOS, atau Windows) untuk instruksi yang lebih lengkap, video instruksional, dan informasi terkait lainnya.

Bibisecting bergantung pada 6 langkah:

  1. Mengatur lingkungan Anda, termasuk memasang perangkat lunak tambahan apa pun
  2. Mengunduh paket bibisect
  3. Melakukan bibisecting kutu
  4. Mendapatkan log bibisect
  5. Attaching the bibisect log to the bug report
  6. Remove keyword 'bibisectRequest' from Keywords field and add the keyword 'bibisected'
  7. If the commit causing the regression has been identified, also add the keyword 'bisected', add the developer as CC and add the comment 'Adding Cc: to <developer's name>'

Note that you should only add the original committer to the CC, not any reviewers. If a developer has retired from the project (check the date of their last commits), there is no point in adding anyone to the CC. Rather in this case, you should bump the priority of the bug report.

Some bibisect repositories have ranges of source commits folded into a single chunk. If after completing a bibisect you are presented with only a single one of these chunk commits, you can get the range by first copying the hash (note: not the source hash) and using it in a git log command like so:

git log c3f9dc54188b316a61dd871b63734d49b05f5d7f

Now you can copy the source hashes of the first and previous displayed commits and use them to construct a range query like so: https://git.libreoffice.org/core/log/7c4d3ea6ba4d42b4dda5148a00c8c411b5d7703d..5b195fbcf7a441aeb193f6abd08b877e580938e0 This will allow you to skim commit messages looking for related subjects. For deeper scrutiny, see the section on detective work.

Limitations

Performing a bibisect is only possible if we have a repository available for the correct OS, covering the correct range of LibreOffice commits. We have very good coverage on GNU/Linux, and are improving our coverage on Windows and macOS.

Some regressions are difficult to reproduce, and may require multiple test runs for a bug to appear. As you may imagine, trying to bibisect such regressions is very difficult, and requires one to perform many test runs after each checkout of a new version of LibreOffice to give some security / statistics that the bug is indeed present or not present with the given version.

We do not currently have bibisect support for

Versions

Versions are listed on each OS page. See #Details above.

Not Bibisectable

Some bugs are not bibisectable. These include

  • Bugs that don't show up on a supported platform (see #Limitations)
  • Bugs that predate our bibisect builds (We're working on extending the range covered)
  • Bugs that depend upon build options not enabled in the TDF builds (e.g. -kde)

If a bug with bibisectRequest Keyword is found to be not bibisectable, leave a comment on the bug explaining why we're not able to use bibisect to track down the introduction of this problem, and replace Keyword bibisectRequest with notBibisectable.

Finding bugs

Menemukan kutu yang membutuhkan bibisect

Following Bugs with Keyword bibisectRequest are waiting for a bibisect:

In addition all unresolved regressions in LibreOffice might benefit from bibisecting.

Finding bugs already bibisected

For the links to bugzilla queries, please see here.

With the above info added to the bug a developer knows that the regression was introduced between the commits fb754a0df859e30255c25af8fa19bfaa75f257e7 (good) and 2d19e9bb07ccff3134f855812dddfda5c07b1fe4 (bad) on master. A:

git log fb754a0df859e30255c25af8fa19bfaa75f257e7..2d19e9bb07ccff3134f855812dddfda5c07b1fe4

on the source repository will then show the 128 commits including the one that introduced the bug, making it a lot easier for the developer to close in on the culprit. For more details, see:

Special situations

Bibisecting language-specific bugs

The TDF bibisect repositories provided do not include any lang-packs.

To bibisect bugs which work good with english UI, but bad with non-english UI (like CJKs): one should copy-paste the following files, from a release build which has your language, to the same location of the bibisect repo:

  1. program/resource/<your_lang_tag>
  2. share/registry/Langpack-<your_lang_tag>.xcd
  3. share/registry/res/fcfg_langpack_<your_lang_tag>.xcd
  4. share/registry/res/registry_<your_lang_tag>.xcd

The suggested language resources you should copy is the one for the same major branch. For example, for the bibisect-linux-64-6.0 repo, you can copy from 6.0.1.1 release.

Running with a separate user profile

To create separate profile directory in /tmp/ (which will be automatically removed during the next machine boot):

./opt/program/soffice -env:UserInstallation=file:///tmp/soffice-bisect

If you want to start fresh for each bisect iteration, you can add git commit hash to directory name. For example:

./opt/program/soffice \
-env:UserInstallation=file:///tmp/soffice-bisect-$(git rev-parse HEAD)

Finding a fix

Sometimes a bug is a collateral damage of another (possibly minor) bug, in a non obvious way. In such cases that other bug might be fixed on master, but was not backported to the release branch. To identify a problem-fixing commit you can use:

git bisect start --term-new=fixed --term-old=unfixed
git bisect fixed master
git bisect unfixed <some-old-sha> # you can use "git log --reverse" to detect the first revision in the bibisected package

As you test, say

git bisect fixed

or

git bisect unfixed

as appropriate.

If the fix is small and important enough, it likely can be easily backported to the release branch.

Checking out a specific commit based on source hash

Sometimes you already know a specific source commit has/does not have the bug you are hunting.

Copy the source hash from the LibreOffice git history and use it to grep in the bibisect repo log like so:

git log --all --grep='29a9f433c268414747d8ec7343fc2b5987971738'

Copy the commit hash of the bibisect repo and do a checkout for it to verify your assumption:

git checkout 38758b70f5278d4d8292a2e857e80a9a1130f38e

You can use the repo commit hash when determining the bad-good range to bisect:

git bisect start 38758b70f5278d4d8292a2e857e80a9a1130f38e oldest

Doing detective work inside a commit range

In case your bibisect result is a range of commits and you have trouble locating the offending commit, you might make your life easier by searching with a keyword.

An example for a case, where we know the problem is in Calc's Navigator view:

git log --stat --pretty=short --graph 169bd7718264b0e312052757f9bbd2321e1399c2...ff8b873936aa72b17309da4bfc2775573a5b1f55 | grep -B 10 -A 10 --color=always "sc/source/ui/navipi" | less -R

Adjust the -B and -A grep parameters to control how many lines before and after the hit you want to display. When piping to less, -R preserves the colouring of search results.

Bisect skip hell

Sometimes we get stuck in a miserable situation, where we have to use git bisect skip over and over again due to various reasons. There are ways of getting out of these infuriating messes and here we will show one of them.

Example bug: tdf#123095

Bug appeared in 4.0-4.1 range.

There are lots of commits, where the autofilter is not shown at all, so have to skip those.

We can use git bisect visualize to show us the remaining suspected commits:

git bisect visualize --oneline > /some/path/bisect-viz-123095.txt

The earliest bad results are at the top while the ones you have skipped are at the bottom. The commit below the bottom is the last known good one.

My top results included

  b842ac1 source-hash-7b4d76772ef76a8de852ed647ed0cad368f70189
  40ef04e source-hash-dc173b7f2a550185404aacbc6da744cb6d1880fc
  8e697de source-hash-eb96e4325278f31b9e6fbc1d5c6a01543204ded6

So I did git checkout b842ac1, test, continue with the next etc. Turns out the third one from the top was already a skippable commit.

I made a range query with the source hashes in the style bottomcommit..firstbadcommit.

In the displayed results I searched for "filter" and found a suspicious commit about sorting autofilter popup items.

If soffice is failing with a non-zero exit code--this probably means a crash--a solution under #Tidbits for efficiency offers a script to ease the pain of repeatedly skipping failed probes.

Finding source commits from bibisect

If bibisect was done, but not likely to be correct or in a repository known to have large range of source commits, we can check the range by finding a previous build commit from the found one and then find the source range.

$ git log -1 1f14665c5624bc7a502738aa8f4f2bd70a211e72^
$ git log --oneline b6c016da23d309b4ac7d154bc33a22397974ed73..d85fd8a85501547d5bb87822d2589a07aed7f2d6

Troubleshooting

Please add your problems and how you solved them. This section covers issues that affect more than one OS. For OS-specific issues, please see the individual OS wiki pages.

Unable to extract files

If you can't extract files (you should get an error message). Try to rename the file from *.tar.lzma to *.tar.xz.

Installed LibreOffice became modified unexpectedly

Problem / Messages:

   error: Your local changes to the following files would be overwritten by checkout:
     opt/program/pythonloader.pyc
     opt/program/uno.pyc
     opt/program/unohelper.pyc
   Please, commit your changes or stash them before you can switch branches.
   Aborting

Solutions:

git checkout .

will repair the problem. Afterwards you can start the next test run as usual with

git bisect good # or git bisect bad

If that does not solve the problem, you can try

git clean -fd
git reset --hard

The ultimate repair is to delete everything except one

.git/

hidden folder, and do

git checkout oldest

Unable to start soffice

Problem 1: Upon trying to run soffice within bibisect you receive the error "unexpected operator terminate called after throwing an instance of 'com::sun::star::uno::DeploymentException'

Solution: Reset your libreoffice profile located in ~/.config/libreoffice. Make sure to backup the folder if you have any settings you'd like to preserve for use with your stable libreoffice release. After you're done with bibisect you can try to return your backed up profile to it's original location (ie. delete the new profile folder and replace it with the backed up one)

Problem 2: Trying to run soffice within bibisect gives frequent errors such as "Application Error" and skipping via 'git bisect skip' is tedious.

Solution: Upon an error, run one-line script to skip commit on error message and try the next one. Script will stop when soffice is successfully run and then you proceed with 'git bisect good' or 'git bisect bad'. If error is repeated, script can be run again.

 while (git clean -f -X && git bisect skip && ! instdir/program/soffice); do : ; done

How to bisect RTL bugs

In case you need to bisect a RTL ( Right-To-Left ) bug, please import this variable before calling LibreOffice:

SAL_RTL_ENABLED=1

How to bisect File Recovery bugs

In case you need to bisect File Recovery bugs ( e.g. tdf#129210 ), comment out line OOO_DISABLE_RECOVERY=1 in file instdir/program/ooenv

You may also use user control:

soffice --norestore file-that-recovers.ext

How to bisect file hanging / CPU or memory exhaustion bugs

In case you need to bisect file hanging/memory exhaustion bugs, you may limit memory or CPU or time execution with timeout utility.
For example, if LibreOffice 'good' commit opens a file with 1 GB of memory, you may set soffice memory limit to 1.500.000 kilobytes to avoid 'bad' commit hanging your computer.

timeout -m 1500000 instdir/program/soffice

Or, if time required to open a file is less than 10 seconds (can be seen with 'time' command), time limit can be set:

timeout -t 10 instdir/program/soffice

Tidbits for efficiency

The repetitive nature of bibisecting gets tedious fast, so every trick you can use to make the process quicker is valuable. A typical way is to define aliases for complex commands in your .bashrc file.

The .bashrc of an active bibisecter might include a block such as this:

alias bad='git bisect bad && instdir/program/soffice'
alias good='git bisect good && instdir/program/soffice'
alias skip='git bisect skip && instdir/program/soffice'
alias next='git log --reverse --pretty=%H master | grep -A 1 $(git rev-parse HEAD) | tail -n1 | xargs git checkout'
alias prev='git checkout HEAD^1'
alias gallery='rm instdir/program/gengal'
alias pycache='find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf'
alias source='if [[ $(git log -1) =~ sha:([[:alnum:]]+) ]]; then git --git-dir ~/libreoffice/.git log -1 ${BASH_REMATCH[1]}; fi'
alias sof="\$( if [[ -d instdir ]] ; then echo instdir ; else if [[ -d opt ]] ; then echo opt ; fi ; fi )/program/soffice --norestore --nologo"

We have commands for jumping to the next or previous commit, "pycache" for dealing with a Windows-specific annoyance and finally "source" for quickly displaying source commit information. The command named "source" assumes you are in a bibisect repository, grabs the source hash of the currently checked out commit and feeds it to a git command targeting a full clone of the LibreOffice source code. To get a breakdown of the command, see the Bash Reference Manual.

The above examples are not exclusive to *nix systems, but work fine in a cygwin environment on Windows. If you want to open documents from the cygwin command line, enclose the ordinary Windows path in single quotes like so:

 instdir/program/soffice 'c:\users\test\downloads\example.ods'

Further information

notes

For details on how the bibisect archives are generated:

We want to make a GUI for bibisect: