Development/Cpp Unit Tests

    From The Document Foundation Wiki

    TDF LibreOffice Document Liberation Project Community Blogs Weblate Nextcloud Redmine Ask LibreOffice Donate

    Other languages:

    CppUnit

    LibreOffice uses the CppUnit framework for its C++ unit tests. CppUnit is a C++ port of the JUnit framework and has been forked several times. LibreOffice maintains its own fork of CppUnit, which is, as of today, the last active fork.

    Xisco Faulí's presentation from FOSDEM 2021, LibreOffice QA - how to write your first test, walks through the process of creating tests.

    Types of Unit tests in the LibreOffice code

    There are many types of unit tests in LibreOffice; if you are unsure which one to use, please ask on the LibreOffice development mailing list, or on the IRC - see "Talk to developers" at https://www.libreoffice.org/community/developers/.

    Testing one function/method

    Take the function/method, and feed it with various data, and assert that the result is correct.

    Example: test::oustring::StringConcat::checkConcat() in https://cgit.freedesktop.org/libreoffice/core/tree/sal/qa/rtl/strings/test_oustring_concat.cxx#n57

    Import test

    For an import test, two things are needed:

    • a test document - a minimized version of the document that shown the bug in the first place. Best if you can re-create the document from scratch, to avoid any confidential data in it. Check the similar tests in the area where the document should be stored, usually it is in a 'data/' subdirectory.
    • the test itself

    The import test then loads the document as a first thing, and then you assert exactly the property that was fixed.

    Example: testFdo50665 in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/rtfimport/rtfimport.cxx#n606

    Export test

    Export test extends the import test by actually re-importing twice:

    • first time, the document is read, then saved using the filter that we want to test, and
    • second time, the saved result is read again, to be able to assert that the saving was done correctly.

    The asserting of the saved result can be done the same way as in the import test (see above) or using assertXPath method (see below).

    Example: testDMLSolidfillAlpha in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/ooxmlexport/ooxmlexport6.cxx#n115

    assertXPath test

    In many cases, we operate on XML:

    • ODF and OOXML are XML in a container
    • we dump data as XML (various debugging output - like dump of the Writer layout, EMF/WMF structure, XShapes, etc.)

    The trick is then to load the data as DOM, and assert various properties via an XPath expression.

    Example: testBehinddoc in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/ooxmlexport/ooxmlexport6.cxx#n610

    EMF/WMF examples: https://cgit.freedesktop.org/libreoffice/core/tree/emfio/qa/cppunit/wmf/wmfimporttest.cxx

    Unit tests dumping a bitmap, and asserting the result

    EMF+ is stored in EMF comments, and rendered later. To be able to test that, it is necessary to dump it as a bitmap, and assert that the various pixels have the right values.

    To dump the bitmap first (to see what exactly you want to assert), export CPPCANVAS_DEBUG_EMFPLUS_DUMP_TO=/tmp/file.png , and the bitmap will be saved to /tmp/file.png when you execute the unit test.

    Example: https://cgit.freedesktop.org/libreoffice/core/tree/cppcanvas/qa/extras/emfplus/emfplus.cxx

    Layout dump test

    This is specific to writer. In some cases, it is necessary to test how the document is laid out by the layout engine - eg. some text should appear on the 3rd page, while previously it was showing on the 2nd page, and similar.

    These test work so that the file is imported, and then the layout is dumped as XML

    Example: testFdo52052 in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/rtfimport/rtfimport.cxx#n755 .

    More info

    XShape test

    This can be easily used in chart2 or Impress. The structure of the view (that is represented as XShapes hierarchy) is dumped as XML, and then either compared against a reference dump (less reliable - as the dump method can change in the future) or checked using the assertXPath (the new method).

    Example (reference): Chart2XShapeTest::testFdo75075() in https://cgit.freedesktop.org/libreoffice/core/tree/chart2/qa/extras/xshape/chart2xshape.cxx#n80

    Example (assertXPath): Chart2XShapeTest::testTdf76649TrendLineBug() in https://cgit.freedesktop.org/libreoffice/core/tree/chart2/qa/extras/xshape/chart2xshape.cxx#n144

    Complex scenario test

    It may happen that a bug needs several steps to reproduce. Even that is testable when the procedure is reliable.

    In such cases, usually it is an import test, with additional steps that need to be done - like selecting text, triggering some action on that, etc.

    Example: testFdo37606Copy in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/odfimport/odfimport.cxx#n738

    LibreOfficeKit (LOK) test

    Some functionality (like pressing a key as if it would be real keyboard input) is available via the LibreOfficeKit API, but not via UNO or internal API (e.g. because in Writer the SwEditWin is not a public class from a visibility point of view).

    If you want to use it, get a reference to the implementation of the UNO object that represents a document, then you can call member functions on it directly.

    Example: testTdf89954 in https://cgit.freedesktop.org/libreoffice/core/tree/sw/qa/extras/uiwriter/uiwriter7.cxx#n2371

    Performance tests

    These tests are executed as part of make perfcheck. They require the valgrind-dev package to be installed. They generate log files in workdir/CppunitTest/[testname]/.. that contain valgrind logs.

    These can be processed to produce a summary CSV file using bin/parse-perfcheck.py

    Example: sc/qa/perf/scperfobj.cxx

    Adding a new test

    In some cases, the infrastructure for the unit tests is not introduced yet. In these cases, please have a look at areas that already have lots of unit tests for inspiration, and create new unit test like following.

    Discovering a test strategy

    You might run LibreOffice in a debugger and insert a breakpoint in a relevant part of the the bug fix. Then, you manually do the thing that used to cause the bug and inspect the backtrace provided by the debugger. Finally, in your cppunit test you invoke the function similarly to how you see it invoked in the backtrace.

    Defining the test class

    The tests should be placed in qa/ subdir in a file named test_xxx.cxx. First, you've to create your class. Here's what can be a basic test skeleton: 

    // copy here a header from the file TEMPLATE.SOURCECODE.HEADER in the root of the source tree.

#include "sal/config.h"
#include "sal/precppunit.hxx"

#include "cppunit/TestAssert.h"
#include "cppunit/TestFixture.h"
#include "cppunit/extensions/HelperMacros.h"
#include "cppunit/plugin/TestPlugIn.h"

namespace
{
/**
 * Perform tests on MyClass.
 */
class MyClassTest : public CppUnit::TestFixture
{
private:
    /**
     * Tests capability Foo of the class.
     */
    void testFoo();

    // Adds code needed to register the test suite
    CPPUNIT_TEST_SUITE(MyClassTest);
    // Declares the method as a test to call
    CPPUNIT_TEST(testFoo);
    // End of test suite definition
    CPPUNIT_TEST_SUITE_END();
};

void MyClassTest::testFoo()
{   
    CPPUNIT_ASSERT(true); // do the tests here
}

// Put the test suite in the registry
CPPUNIT_TEST_SUITE_REGISTRATION(MyClassTest);

} // namespace
CPPUNIT_PLUGIN_IMPLEMENT();

    Write one or more test functions and test various capabilities of the class. See CppUnit tutorial for further details about testing with CppUnit.

    Run test at build time

    Now that your test class is ready, you need to compile and run it. This means creating file CppunitTest_module_test.mk (substitute module and test name as appropriate) in the toplevel (module) directory if it does not exist yet: 

    # copy here a header from the file TEMPLATE.SOURCECODE.HEADER in the root of the source tree.

$(eval $(call gb_CppunitTest_CppunitTest,module_test))

$(eval $(call gb_CppunitTest_add_exception_objects,module_test, \
module/qa/cppunit/test_xxx \
))

# add a list of all needed libraries here
$(eval $(call gb_CppunitTest_add_linked_libs,module_test, \
sal \
$(gb_STDLIBS) \
))

$(eval $(call gb_CppunitTest_set_include,module_test,\
$$(INCLUDE) \
-I$(OUTDIR)/inc \
))

    You will also need to edit Module_module.mk and add the following (or add to the list if it already exists): 

    $(eval $(call gb_Module_add_check_targets,module,\
     CppunitTest_module_test \
))

    To build and run the tests, simply run make check. If the build succeeds, the tests will run automatically.

    So, once your tests are running, all you need to do is fix any bugs they uncover.

    Run a single test at a time

    CPPUNIT_TEST_NAME is the environment variable that needs to be set and contain the name of the tests. The test names need to be fully qualified to be recognized.

    Example: 

    CPPUNIT_TEST_NAME="testOutlineODS testPrintRangeODS" make CppunitTest_sc_subsequent_filters_test2

    Printing to stderr is hidden if the test passes and displayed if it fails.

    If you use a single string as a value of CPPUNIT_TEST_NAME, that has to be a class name. If that class has multiple tests, then all tests inside that class will be ran. The CPPUNIT_TEST_FIXTURE macro always creates a class with a single test function.

    You can discover the make targets by looking at the *.mk files in modules.

    Retrieved from "https://wiki.documentfoundation.org/index.php?title=Development/Cpp_Unit_Tests&oldid=739414"