Development/Lcov

This short tutorial aims to describe how to generate lcov reports.

Example results
Current results can be found here:

http://lcov.libreoffice.org

Details on how these are generated here: https://lists.freedesktop.org/archives/libreoffice/2015-June/068679.html

Historic results can be found here:

https://dev-builds.libreoffice.org/lcov_reports/

Set up build environment
Follow the guidelines at : How to Build to set up your LibreOffice build environment. This guide assumes you have the libreoffice source code in '/usr/local/src/libreoffice'.

Install lcov
You need to use lcov 1.10 or higher. http://ltp.sourceforge.net/coverage/lcov.php

patch 'geninfo'
You may need to apply the following patch to 'geninfo' :

If you're on openSUSE >=12.2, just make sure you have a fully updated system, and lcov will just work. If you're using the latest distribution supplied lcov package on Fedora >=17, this patch may not be required anymore either.

Create lcovrc file
Create an ~/.lcovrc file in your home directory and add the following content :

geninfo_auto_base = 1

This option must be set to prevent errors like :

ERROR: could not read source file /home/user/project/sub-dir1/subdir2/subdir1/subdir2/file.c

This option is required when using lcov on projects built  with libtool or similar build environments that work with multiple base directories, i.e. environments, where the current  working  directories when  invoking the compiler are not the same directories in which the source code file is located. This option specifies that geninfo should try to automatically determine the base-directories when collecting coverage data.

Clean up from previous run
This step only needs to be performed if you already generated the reports at some earlier time. If this is the first time you generate them, this step can safely be skipped.

Remove the tracefiles from the previous run, reset the counters, and make clean.

rm -f /tmp/libreoffice_base.info /tmp/libreoffice_test.info /tmp/libreoffice_filtered.info /tmp/libreoffice_total.info lcov --zerocounters --directory /usr/local/src/libreoffice make distclean ./autogen.sh

Set appropriate FLAGS set and configure
You need to specify additional LDFLAGS and CFLAGS to instrument the build properly. You also need to specify some flags for the ./configure script. Because chances are that you will not have all required prerequisite libraries installed on your system, the basic idea is to use as much of the 'internal' libreoffice libraries (included in the libreoffice source tree) as possible. You can do this by using '--without-system-libs' and '--without-system-headers'.

The build has an online update mechanism that spawns threads that dont properly join again before exit, which is a notorious problem with the tests (that often run quickly enough to terminate soffice.bin while the online update check is still in progress). In order to prevent this, you can run ./configure with '--disable-online-update'.

LDFLAGS+='-fprofile-arcs' CFLAGS+='-fprofile-arcs -ftest-coverage' \ CXXFLAGS+='-fprofile-arcs -ftest-coverage' CPPFLAGS+='-fprofile-arcs -ftest-coverage' \ ./configure --disable-online-update --without-system-libs --without-system-headers

Build LibreOffice
The basic idea is to initially build the project only without running any tests (as much as possible.)

make

Run initial/baseline lcov
Now you need to create an lcov "baseline" before running any tests. The result is a coverage data file that contains zero coverage for every instrumented line of the project. At a later stage, you will combine this data file with coverage data files captured after the test run. This way the percentage of total lines covered will always be correct, even when not all source code files were loaded during the test(s).

lcov --no-external --capture --initial --directory /usr/local/src/libreoffice --output-file /tmp/libreoffice_base.info

Run tests
Now start the tests. The default behaviour is to stop the testing entirely if a single test fails, so that you can troubleshoot the failure. In order to let the tests continue even if one fails, you have to add '-k' to the 'make' command. This instructs it to keep going as much as possible after an error occurs. This is done in order to get as accurate as possible an picture of the overall code coverage of the tests. ('make check' will run 'slowcheck', 'unitcheck', and 'subsequentcheck' as well.)

make -k check

Run lcov again after tests/checks complete
lcov --no-external --capture --directory /usr/local/src/libreoffice --output-file /tmp/libreoffice_test.info

You may get some warnings about no data being found for /usr/include/*.h stuff. As we are not interested in that, this should not be a problem.

Combine lcov tracefiles
Now combine the 'before tests' and 'after tests' snapshots.

lcov --add-tracefile /tmp/libreoffice_base.info --add-tracefile /tmp/libreoffice_test.info \ --output-file /tmp/libreoffice_total.info

Remove / filter out remaining unwanted stuff from tracefile
Now we can remove the (remaining) stuff that we aren't interested in from the tracefile.

lcov --remove /tmp/libreoffice_total.info '/usr/include/*' '/usr/lib/*' '/usr/local/src/libreoffice/*/UnpackedTarball/*' \ '/usr/local/src/libreoffice/workdir/*' '/usr/local/src/libreoffice/instdir/*' '/usr/local/src/libreoffice/external/*' \ -o /tmp/libreoffice_filtered.info

Create a storage directory
Create an empty directory to store the reports.

rm -rf /tmp/libreoffice-lcov/ mkdir /tmp/libreoffice-lcov/

Retrieve software version
It is a good practice to include the version of the software you run the report on somewhere in the report. One way to do this, is to use the '--title' option with the genhtml command. If you used a git version, you could add the commit SHA1 that way. You can retreive this by using the 'git log' command on the directory where you cloned/located the libreoffice source. The first line in the output is the commit SHA1.

cd /usr/local/src/libreoffice git log | head

Create reports
Now generate the reports. Remember to include the git commit SHA1.

genhtml --prefix /usr/local/src/libreoffice --ignore-errors source /tmp/libreoffice_filtered.info \ --legend --title "commit SHA1" --output-directory=/tmp/libreoffice-lcov

Done.
You should now have the generated HTML report files. See Development/RegressionHotspots, and  if they are too red for your taste.