Development/Cpp Unit Tests

CppUnit
LibreOffice uses the CppUnit framework for its C++ unit tests. CppUnit is a C++ port of the JUnit framework and has been forked several times. LibreOffice maintains its own fork of CppUnit, which is, as of today, the last active fork.

Xisco Faulí's presentation from FOSDEM 2021, LibreOffice QA - how to write your first test, walks through the process of creating tests.

Types of Unit tests in the LibreOffice code
There are many types of unit tests in LibreOffice; if you are unsure which one to use, please ask on the LibreOffice development mailing list, or on the IRC - see "Talk to developers" at https://www.libreoffice.org/community/developers/.

Testing one function/method
Take the function/method, and feed it with various data, and assert that the result is correct.

Example: test::oustring::StringConcat::checkConcat in https://cgit.freedesktop.org/libreoffice/core/tree/sal/qa/rtl/strings/test_oustring_concat.cxx#n57

Import test
For an import test, two things are needed:
 * a test document - a minimized version of the document that shown the bug in the first place. Best if you can re-create the document from scratch, to avoid any confidential data in it.  Check the similar tests in the area where the document should be stored, usually it is in a 'data/' subdirectory.
 * the test itself

The import test then loads the document as a first thing, and then you assert exactly the property that was fixed.

Example: testFdo50665 in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/rtfimport/rtfimport.cxx#n606

Export test
Export test extends the import test by actually re-importing twice:


 * first time, the document is read, then saved using the filter that we want to test, and
 * second time, the saved result is read again, to be able to assert that the saving was done correctly.

The asserting of the saved result can be done the same way as in the import test (see above) or using assertXPath method (see below).

Example: testDMLSolidfillAlpha in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/ooxmlexport/ooxmlexport6.cxx#n115

assertXPath test
In many cases, we operate on XML:


 * ODF and OOXML are XML in a container
 * we dump data as XML (various debugging output - like dump of the Writer layout, EMF/WMF structure, XShapes, etc.)

The trick is then to load the data as DOM, and assert various properties via an XPath expression.

Example: testBehinddoc in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/ooxmlexport/ooxmlexport6.cxx#n610

EMF/WMF examples: https://cgit.freedesktop.org/libreoffice/core/tree/emfio/qa/cppunit/wmf/wmfimporttest.cxx

Unit tests dumping a bitmap, and asserting the result
EMF+ is stored in EMF comments, and rendered later. To be able to test that, it is necessary to dump it as a bitmap, and assert that the various pixels have the right values.

To dump the bitmap first (to see what exactly you want to assert), export CPPCANVAS_DEBUG_EMFPLUS_DUMP_TO=/tmp/file.png, and the bitmap will be saved to /tmp/file.png when you execute the unit test.

Example: https://cgit.freedesktop.org/libreoffice/core/tree/cppcanvas/qa/extras/emfplus/emfplus.cxx

Layout dump test
This is specific to writer. In some cases, it is necessary to test how the document is laid out by the layout engine - eg. some text should appear on the 3rd page, while previously it was showing on the 2nd page, and similar.

These test work so that the file is imported, and then the layout is dumped as XML

Example: testFdo52052 in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/rtfimport/rtfimport.cxx#n755.

More info

XShape test
This can be easily used in chart2 or Impress. The structure of the view (that is represented as XShapes hierarchy) is dumped as XML, and then either compared against a reference dump (less reliable - as the dump method can change in the future) or checked using the assertXPath (the new method).

Example (reference): Chart2XShapeTest::testFdo75075 in https://cgit.freedesktop.org/libreoffice/core/tree/chart2/qa/extras/xshape/chart2xshape.cxx#n80

Example (assertXPath): Chart2XShapeTest::testTdf76649TrendLineBug in https://cgit.freedesktop.org/libreoffice/core/tree/chart2/qa/extras/xshape/chart2xshape.cxx#n144

Complex scenario test
It may happen that a bug needs several steps to reproduce. Even that is testable when the procedure is reliable.

In such cases, usually it is an import test, with additional steps that need to be done - like selecting text, triggering some action on that, etc.

Example: testFdo37606Copy in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/odfimport/odfimport.cxx#n738

LibreOfficeKit (LOK) test
Some functionality (like pressing a key as if it would be real keyboard input) is available via the LibreOfficeKit API, but not via UNO or internal API (e.g. because in Writer the SwEditWin is not a public class from a visibility point of view).

If you want to use it, get a reference to the implementation of the UNO object that represents a document, then you can call member functions on it directly.

Example: testTdf89954 in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/uiwriter/uiwriter7.cxx#n2371

Performance tests
These tests are executed as part of make perfcheck. They require the valgrind-dev package to be installed. They generate log files in workdir/CppunitTest/[testname]/.. that contain valgrind logs.

These can be processed to produce a summary CSV file using bin/parse-perfcheck.py

Example: sc/qa/perf/scperfobj.cxx

Adding a new test
In some cases, the infrastructure for the unit tests is not introduced yet. In these cases, please have a look at areas that already have lots of unit tests for inspiration, and create new unit test like following.

Discovering a test strategy
You might run LibreOffice in a debugger and insert a breakpoint in a relevant part of the the bug fix. Then, you manually do the thing that used to cause the bug and inspect the backtrace provided by the debugger. Finally, in your cppunit test you invoke the function similarly to how you see it invoked in the backtrace.

Defining the test class
The tests should be placed in  subdir in a file named. First, you've to create your class. Here's what can be a basic test skeleton:

Write one or more test functions and test various capabilities of the class. See CppUnit tutorial for further details about testing with CppUnit.

Run test at build time
Now that your test class is ready, you need to compile and run it. This means creating file  (substitute module and test name as appropriate) in the toplevel (module) directory if it does not exist yet:

You will also need to edit  and add the following (or add to the list if it already exists):

To build and run the tests, simply run. If the build succeeds, the tests will run automatically.

So, once your tests are running, all you need to do is fix any bugs they uncover.

Run a single test at a time
CPPUNIT_TEST_NAME is the environment variable that needs to be set and contain the name of the tests. The test names need to be fully qualified to be recognized.

Examples:

You may omit the  variable entirely and just write. Printing to stderr is (always?) hidden if the test passes and displayed if it fails.