质控/二进制二分查找(Bibisect)

    From The Document Foundation Wiki
    < QA
    This page is a translated version of the page QA/Bibisect and the translation is 38% complete.
    Outdated translations are marked like this.

    简介

    二进制二分查找(Bibisect,即 "binary bisect"),是一整套在软件中定位导致了某项功能“倒退(regressions)”的源代码提交(commit)的方法。“倒退”问题是软件开发以及质量控制中最应该避免的错误,它是指在原来的版本中表现完好的某项功能,在后续的版本中却突然无法使用或者无法正常工作的现象,这通常是由于在之前能正常工作的版本与最近不能正常工作的版本之间,存在一个或多个有缺陷的源代码提交。我们需要尽早发现“倒退”问题,并尽快定位导致该问题的有缺陷的源代码,这是因为在问题出现的早期通常会很容易定位该问题,而时间过的越旧,查找它就会越困难。

    If you are new to the topic of bisecting, it is recommended to first complete our tutorial and only then return to this article. Also, while you are free to study the topic independently, mentoring is available for all.

    Bibisect 二进制二分查找原理

    每个 bibisect 软件包(或者称为 bibisect 仓库)其实都是一个 git 仓库,该仓库中包含了成百上千个(次)从源代码编译的 LibreOffice 二进制程序,这些二进制程序在放入该 git 仓库时是按照对应的源代码提交先后顺序排列的。按照这种方式组织的 LibreOffice 二进制包的集合,是我们在多个版本中定位导致某个“倒退”问题源代码的绝佳方式。有了 bibisect 仓库,我们在测试时不需要每次切换到某个版本之后运行程序时重新编译源代码了。

    采用这种方式,我们一方面可以使用 git 本身的二分查找工具(即 git bisect)直接基于二进制程序进行二分查找,另一方面又能受益于 git 对文件系统良好的差分压缩、去重以及文件组织方式。例如,一次完整的 LibreOffice 源代码编译后生成的二进制可执行程序至少会占用500MB磁盘空间,但是如果采用 git, 则虽然第一次添加占用的空间相同,但是后续每次添加产生的增量磁盘空间可能只有 25MB 大小。

    详情

    一次成功的“二进制二分查找”(bibisect)会将导致“倒退”问题的可能源代码范围缩小至很少的几个commit,开发者可以从这个小范围内迅速找到引起“倒退”问题的真正“罪魁祸首”。 Bibisect 之后,QA 团队可以进行额外的步骤进行 bisect,从而将范围缩小到某个具体的源代码 commit, 或者将这项任务留给开发者完成。通常情况下,只要进行了成功的 bibisect, 则找到具体的导致“倒退”问题的具体 commit 会非常容易。

    使用不到 10G 的 bibisect 仓库,您可以用于成百上千个编译后的 LibreOffice 版本进行测试。通常情况下完成一个 bibisect 二分查找过程只需要不到15分钟。

    当前,“二进制二分查找”可在以下操作系统上进行:

    涵盖的源代码范围及构建数章节所示, 取决于您所用的 bibisect 仓库,其涵盖的源代码 commit 范围,对于类似“43max”这样的仓库可能是80个一次构建(某些仓库可能达到几百个);而对于比较新的仓库,可能每隔1个或2个 commit 就有一次构建。(还有一些比较“粗糙”的仓库只是对每次的已发布版本 + beta 版本 + RC 版本进行了构建,这些仓库可能每个二进制构建涵盖上百个源代码 commit)。

    一般性说明

    对于初学者,我们建议大家从已经完成了 bibisect 过程的示例中进行学习。为此,我们准备了Bibisect 入门教程

    Note pin.svg

    Note:
    Each operating system is different, so please read the page that corresponds to your operating system (GNU/Linux, macOS or Windows) for more specific instructions, tips and troubleshooting.

    首次进行二进制二分查找前,您需要:

    The bibisecting articles for each operating system contain tables listing the available versions. In the "Download links" column of the table you will find links to pages explaining how to acquire the repositories. The repositories for old versions no longer receiving updates are offered as bundles. The repositories for still active versions can be periodically updated with the command git checkout master && git pull.

    If you are testing version 5.x or older, you have to read carefully the article for the operating system you are using and also check the comments column in the Version table. The older versions require various special considerations that this general article does not mention.

    Use the cd (change directory) command in your terminal to move between the different repositories. Jump to the oldest commit in a repository by saying git checkout oldest and to the newest by saying git checkout master. In some older repositories you have to use latest instead of master. Even if the bug report already points out the first version with the bug, always start by testing with both the oldest and the newest commits.

    The commands to launch LibreOffice from the bibisect repositories are as follows in the different operating systems:

    # Windows
    instdir\program\soffice
    </div>
    
    <div lang="en" dir="ltr" class="mw-content-ltr">
    # Linux
    instdir/program/soffice
    </div>
    
    <div lang="en" dir="ltr" class="mw-content-ltr">
    # macOS
    open LibreOffice.app/

    When you don't see the bug in oldest, but do see it in the newest commit, start bibisecting by saying git bisect start master oldest.

    After testing in LibreOffice and seeing the bug, say git bisect bad. If you don't see the bug, say git bisect good. If you were unable to test, say git bisect skip (note that there are various tips related to these scenarios in this and other articles).

    After the final step in the bibisecting process, you are presented with the first bad commit. Within the information given, you are interested in the source hash, which is a combination of letters and numbers. Now you can study the commit to see what it's about and who wrote it. If you have the LibreOffice source code on your computer, you can change to its directory and say git show 22fc8c634c5f9b09d45aff0403503f4d8226328d, the last part being the source hash. Otherwise you can examine the commit in a web browser by creating a URL like so: https://git.libreoffice.org/core/commit/22fc8c634c5f9b09d45aff0403503f4d8226328d

    Now update the bug report as follows (can be done in a single change):

    1. Write a comment mentioning the bibisect repository you used and the source hash (it will become a link in Bugzilla automatically). Any other useful information is welcome, of course.
    2. Update the Version field, if needed (see also #Finding the earliest version affected
    3. Replace the keyword bibisectRequest with bibisected and bisected (only include the last one, if you found the exact commit and not just a range of commits)
    4. Add the name of the commit author to the Regression By field (for this you need contributor rights in Bugzilla)
    5. Add the commit author to the CC List of the report

    需要注意的是,您应当将最初的代码作者(author)添加到 CC 列表,而不是审阅者(Reviewer)。如果某个开发者已经不再参与这个项目了(您可以检查他/她最后一次提交commit的日期来判断),那么就没有必要将任何人添加到 CC 列表,此时您应当提高这个 bug 的优先级。

    Commit ranges

    Some bibisect repositories have ranges of source commits folded into a single chunk. If after completing a bibisect you are presented with only a single one of these chunk commits, you can get the range by first copying the hash (note: not the source hash) and using it in a git log command like so:

    git log -2 c3f9dc54188b316a61dd871b63734d49b05f5d7f

    现在,您可以复制第一个和前一个显示的二进制 commit 对应的源代码散列值(source hash)并使用..分隔,以构造出类似下面的查询:

    https://git.libreoffice.org/core/log/7c4d3ea6ba4d42b4dda5148a00c8c411b5d7703d..5b195fbcf7a441aeb193f6abd08b877e580938e0
    

    得到这个范围之后,您就能通过范围内源代码 commit 的描述和变化内容来进行更深层次的筛查。见:深层次筛查部分的内容

    Finding the earliest version affected

    If you have a local copy of the core repository, you can update the Version field in the bug report thanks to some git commands:

    1. First, make sure you have all the tags:

    git fetch --tag

    2. Then, use `name-rev` to find the context (more precisely, by how many revisions it preceded the earliest tag that contains the commit):

    git name-rev <commit>

    3. If that didn't give a useful tag that matches a LibreOffice version, you can list all tags that contain the commit and have "libreoffice" in the name with:

    git tag --contains <commit> | grep libreoffice

    限制

    要进行 bibisect,您必须已经下载了针对您的操作系统的二进制软件 git 仓库,并且该仓库覆盖了合适的 LibreOffice 源代码 commit 范围。在这方面,我们在 GNU/Linux 操作系统下覆盖很全面。我们正在努力完善 Windows 和 macOS 下的覆盖度。

    Some regressions are difficult to reproduce, and may require multiple test runs for a bug to appear. As you may imagine, trying to bibisect such regressions is very difficult, and requires one to perform many test runs after each checkout of a new version of LibreOffice to give some security / statistics that the bug is indeed present or not present with the given version.

    我们当前不支持以下平台上的 bibisect:

    无法进行 Bibisect 的 bug

    有一些 bug 无法进行 Bibisect,这包括:

    • Bugs that don't show up on a supported platform (see #Limitations)
    • Bugs that predate our bibisect builds (We're working on extending the range covered)
    • Bugs that depend upon build options not enabled in the TDF builds (e.g. -kde)
    • Bugs that don't show up on a supported platform (see #Limitations)
    • Bugs that predate our bibisect builds
    • Bugs that depend upon build options not enabled in the TDF builds

    If a bug with bibisectRequest Keyword is found to be not bibisectable, leave a comment on the bug explaining why we're not able to use bibisect to track down the introduction of this problem, and replace Keyword bibisectRequest with notBibisectable. If a bug predates bibisect builds, replace Keyword bibisectRequest with preBibisect.

    找到 bug

    找到需要进行“二进制二分查找”的 bug

    If you are new to bibisecting, click the Id column header until the list is sorted from highest Id to lowest. Compared to newer ones, when bibisecting older issues you will run into more surprises and greater complexity.

    以下 bug 包含 bibisectRequest 关键字,需要进行二进制二分查找:

    另外,二进制二分查找对于解决LibreOffice中所有未解决的倒退性问题均会有帮助。

    特殊情况

    二分查找与特定语言(比如中日韩语言界面)相关的 bug

    您要二分查找的 bug 可能与特定的界面语言相关。比如 tdf#115637,这个 bug 在中文界面上能够100%出现,而在英文界面上从来不出现。在这种情况下,因为 TDF 提供的二分查找仓库中默认没有提供任何语言包,只有英文界面,因此如果不进行改进,就无法二分查找这样的 bug。

    如果某个 bug 在英文界面上工作正常,但在非英文界面上有问题(例如中日韩界面),您可以将以下文件从一个包含您的界面语言的常规发布版本中,复制到 bibisect 仓库:

    1. program/resource/<您的语言标记>
    2. share/registry/Langpack-<您的语言标记>.xcd
    3. share/registry/res/fcfg_langpack_<您的语言标记>.xcd
    4. share/registry/res/registry_<您的语言标记>.xcd

    复制时,请注意尽量复制主版本号相同的版本中的资源。比如,对于 bibisect-linux-64-6.0 仓库,您可以复制 6.0.1.1 版本中的语言资源。

    以单独的“用户配置文件”运行程序

    如果想将用户配置文件放到 /tmp/ 临时目录中(该目录下的所有内容将在系统重启后被删除):

    ./opt/program/soffice -env:UserInstallation=file:///tmp/soffice-bisect

    如果想在每次的二分查找中都让 libreoffice 以全新的用户配置文件启动,那么可以将 git commit 的希哈值附加到文件夹名称中,例如:

    ./opt/program/soffice \
    -env:UserInstallation=file:///tmp/soffice-bisect-$(git rev-parse HEAD)

    逆向 bibisect(找到修复了某个问题的提交)

    Sometimes a bug is a collateral damage of another (possibly minor) bug, in a non obvious way. In such cases that other bug might be fixed on master, but was not backported to the release branch. To identify a problem-fixing commit you can use:

    git bisect start --term-new=fixed --term-old=unfixed
    git bisect fixed master
    git bisect unfixed <some-old-sha> # you can use "git log --reverse" to detect the first revision in the bibisected package

    As you test, say

    git bisect fixed
    or
    git bisect unfixed
    as appropriate.

    If the fix is small and important enough, it likely can be easily backported to the release branch.

    根据源代码的 hash 值检出某个特定的提交

    有时候,您已经知道,在之前的某个特定的源代码 commit 中不存在您正在测试的 bug。

    这种情况下,复制 LibreOffice 源代码 git 历史中的这个 source hash,并用它再 bibisecct 仓库的 log 中进行 grep:

    git log --all --grep='29a9f433c268414747d8ec7343fc2b5987971738'

    然后复制 bibisect 中的 commit hash,检出这个 commit,然后验证您的假设:

    git checkout 38758b70f5278d4d8292a2e857e80a9a1130f38e

    然后您可以使用 bibisect 仓库的 commit hash 来决定 bisect 过程中的 bad 和 good 的范围:

    git bisect start 38758b70f5278d4d8292a2e857e80a9a1130f38e oldest

    在某个 commit 范围内进行侦察

    In case your bibisect result is a range of commits and you have trouble locating the offending commit, you might make your life easier by searching with a keyword.

    An example for a case, where we know the problem is in Calc's Navigator view:

    git log --stat --pretty=short --graph 169bd7718264b0e312052757f9bbd2321e1399c2...ff8b873936aa72b17309da4bfc2775573a5b1f55 | grep -B 10 -A 10 --color=always "sc/source/ui/navipi" | less -R

    Adjust the -B and -A grep parameters to control how many lines before and after the hit you want to display. When piping to less, -R preserves the colouring of search results.

    This method can also be used as a shortcut in a situation, where you find yourself having to use git bisect skip over and over again due to some broken functionality.

    如果在 bibisect 过程中 soffice 在某个或某几个 commit 下频繁崩溃,让您无法准确判断代码范围,那么 #Tidbits for efficiency 章节提供了解决办法。

    Especially in the older repositories, sometimes you can end up with a list of dozens of potential bad commits. The commits might not be in order, so it is not easy to construct a range query. You can save the list of commits to a text file and run git show for all of them, directing the output to a file:

    while IFS="" read -r p || [ -n "$p" ]
    do
      git show --no-patch "$p" >> /path/to/bisectlog.txt
    done < /path/to/bisect_range.txt

    The older repositories include the original commit message, so this resulting log file will be immediately useful. If the binary repository doesn't include the original messages, you have to first do a pass with git show --oneline --no-patch, tidy up the list of source hashes and then do the final looping against LibreOffice core repository.

    使用 bibisect 找到源代码 commit

    If bibisect was done, but not likely to be correct or in a repository known to have large range of source commits, we can check the range by finding a previous build commit from the found one and then find the source range.

    $ git log -1 1f14665c5624bc7a502738aa8f4f2bd70a211e72^
    $ git log --oneline b6c016da23d309b4ac7d154bc33a22397974ed73..d85fd8a85501547d5bb87822d2589a07aed7f2d6

    故障排除

    This section covers issues that affect more than one OS. For OS-specific issues, please see the individual OS wiki pages.

    无法提取文件

    If you can't extract files (you should get an error message), try to rename the file from *.tar.lzma to *.tar.xz.

    已安装的 LibreOffice 被意外修改

    问题 / 信息:

       error: Your local changes to the following files would be overwritten by checkout:
         opt/program/pythonloader.pyc
         opt/program/uno.pyc
         opt/program/unohelper.pyc
       Please, commit your changes or stash them before you can switch branches.
       Aborting
    

    解决办法:

    git checkout .
    
    # <span lang="en" dir="ltr" class="mw-content-ltr">Sometimes this more specific command is needed:</span>
    find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf;

    The above command will repair the problem. Afterwards you can start the next test run as usual with

    git bisect good # or git bisect bad

    If that does not solve the problem, you can try the following commands:

    git clean -fd
    git reset --hard

    The ultimate repair is to delete everything except one hidden .git/ folder, and do

    git checkout oldest

    无法启动 soffice

    Upon trying to run soffice within bibisect you receive the error "unexpected operator terminate called after throwing an instance of 'com::sun::star::uno::DeploymentException'

    Solution: Reset your Libreoffice profile. Make sure to backup the folder, if you have any settings you'd like to preserve for use with your stable LibreOffice release. After you're done with the bibisect you can try to return your backed up profile to its original location (ie. delete the profile folder and replace it with the backed up one)

    如果 bisect RTL(从右向左)类型的 bug

    如果想要对 RTL ( Right-To-Left,从右向左) 语言相关的 bug, 在运行 LibreOffice 之前请导入如下的变量:

    SAL_RTL_ENABLED=1

    如何 bisect 文件恢复类型的 bug

    In case you need to bisect File Recovery bugs ( e.g. tdf#129210 ), comment out line OOO_DISABLE_RECOVERY=1 in file instdir/program/ooenv

    You may also use user control:

    soffice --norestore file-that-recovers.ext

    How to bisect file hanging / CPU or memory exhaustion bugs

    In case you need to bisect file hanging/memory exhaustion bugs, you may limit memory or CPU or time execution.

    In Linux, timeout utility can be used. To avoid confusion with built-in "timeout" command, here it's renamed to "tmt".

    For example, if LibreOffice 'good' commit opens a file with 1 GB of memory, you may set soffice memory limit to 1.500.000 kilobytes to avoid 'bad' commit hanging your computer.

    tmt -m 1500000 instdir/program/soffice

    Or, to limit CPU+SYS time (where normal time is seen with successful run):

    tmt -t 20 instdir/program/soffice

    More advanced option is to use benchexec utility. User needs to be added to the appropriate group (benchexec). Runexec can be used to execute a single command while measuring its resource consumption, similarly to the tool "time" but with more reliable time measurements and with measurement of memory usage.

    In Windows, process-governor utility can be used.

    更多信息

    Automation

    See the separate article on automating bisecting.

    备注

    Bibisect 仓库是如何生成的: