Jump to content

Development/How to debug/ar

From The Document Foundation Wiki
This page is a translated version of the page Development/How to debug and the translation is 0% complete.

Debugging options

You need to enable debugging support to do any practical interactive debugging. You can enable it for the entire build with:

./autogen.sh --enable-debug

or

./autogen.sh --enable-dbgutil

If you have done a non-debug build before, you need to run make clean. If you later want to switch back to doing non-debug builds, you have to run the cleanup again.

Using --enable-dbgutil is like --enable-debug and additionally it enables more-or-less useful assertions and additional debugging code, and also the STL debugging mode of libstdc++ on some GCC based platforms (but not macOS because Apple's libstdc++ lacks support for it, and clang libc++ does not appear to have a debug mode), and uses the debug runtimes (including debug STL) with MSVC. Note that it is not possible to mix code built with and without --enable-dbgutil.

Note that a complete build with --enable-debug or --enable-dbgutil for all modules takes a large amount of disk space. If you find this too much you can either use --enable-symbols to enable debugger symbols for just specific parts

./autogen.sh --enable-dbgutil --enable-symbols="sw/ sc/ xmloff/"

or do a full non-debug build and then rebuild just the modules you are interested in:

make <module>.clean && make <module> debug=t

Debugging with an IDE

See Development/IDE for how to setup various IDEs for debugging.

Debugging with gdb

Debugging with lldb

If you don't have gdb available, say because you're using macOS, then you need to use lldb.

The commands are different between gdb and lldb.

There is some helpful python pretty printing that you can load manually to print OUStrings etc., see solenv/lldb/libreoffice/LO.py.

Unfortunately at least Apple's lldb appears to have a nasty habit of truncating command line arguments, so starting CppUnit tests in the usual way may fail with an exception from cppuhelper::ServiceManager::readRdbFile.

Finding a stacktrace of an "ignored" C++ exception

Say your debug build is giving you

warn:linguistic:67283:363302:linguistic/source/gciterator.cxx:679: GrammarCheckingIterator::DequeueAndCheck ignoring N3com3sun4star3uno9ExceptionE msg: C++ code threw St13runtime_error: collate_byname<char>::collate_byname failed to construct for 

and you would like a stacktrace for the exception. The line number above is in a part of the code that catches all exceptions, but we want a stack trace for that std::runtime_error. What we can do is

  1. add a breakpoint at the start of the try block, in this case line 608
  2. once we've broken there, delete that breakpoint and instead add one for all C++ exceptions
  3. continue until we get a C++ exception, then print the backtrace (possibly repeating until we hit the right one)

To do this, first start the lldb debugger (in $LODE_HOME/dev/core if you checked out through lode) with

make debugrun

then when lldb has started, add the initial break point and start the program

(lldb) b linguistic/source/gciterator.cxx:608
# <span lang="en" dir="ltr" class="mw-content-ltr">ignore some warnings about locations not being known yet</span>
(lldb) run --writer --norestore /more/arguments/to/soffice

then remove that breakpoint and start breaking on all C++ exceptions:

(lldb) break set -E cxx
# <span lang="en" dir="ltr" class="mw-content-ltr">optionally delete the old one, e.g. "1" in the list:</span>
(lldb) break list
(lldb) break delete 1
# <span lang="en" dir="ltr" class="mw-content-ltr">and continue and backtrace when you see the exception</span>
(lldb) c
(lldb) bt
# <span lang="en" dir="ltr" class="mw-content-ltr">repeat until you've hit the right one</span>


Debugging with WinDbg or Visual Studio (on Windows)

Debugging with Visual Studio is extremely simple, only a debug build is needed, no need to use IDE integration. Simply open the source file of interest in Visual Studio, add a breakpoint, start soffice.exe, and attach debugger to soffice.bin via Debug ▸ Attach to Process.... Execution will halt when the breakpoint is hit (to stop execution immediately, use Debug ▸ Break All). Visual Studio opens the source files for the executed code automatically during debugging. Call stack is available in the window with the same name.

Using Time Travelling Debugging with windbg

See https://learn.microsoft.com/en-us/windows-hardware/drivers/debuggercmds/time-travel-debugging-overview. You can use this to capture a failure scenario just once, and replay parts of it as needed in the debugger.

Enhancing your debugging experience with Visual Studio

There's lots of configuration options around Visual Studio can will improve your experience and productivity. Most notable are the native visualizers. There is a custom .natvis file in /solenv/vs, which allows to visualize objects of some classes used in LO code easier. The contents of the file is embedded into PDBs for debug builds, so debugging makes use of the visualizers even when one doesn't use IDE integration (generated solution files). Please add your useful additions to the visualizers to that file, and submit patches :)

Also helpful: preventing to step into functions that are trivial one-lines, like the operator-> for smart pointers. Here's the howto: How to *not* step in certain functions? (Stack Overflow)

It's often useful to *disable* "Just My Code" feature of Visual Studio debugger: Options->Debugger->General; locate and uncheck the "Enable Just My Code" item. Otherwise, call stacks would be incomplete and unhelpful.

Debugging release builds

In Visual Studio, the call stacks may at first appear useless; to avoid this, enable the Microsoft Symbol Server in the configuration: Tools ▸ Options ▸ Debugging ▸ Symbols and click on "Microsoft Symbol Servers".

For info on how to set up WinDbg or Visual Studio for debugging TDF release builds with downloaded symbols see How to get a backtrace with WinDbg.

Registering a build as the COM server provider

If you have to debug the COM component process, you can register your build as the COM server provider using the following script:

REM usage:
REM libreoffice-registry-set-com-path.bat soffice_exe_path

reg add HKEY_CLASSES_ROOT\CLSID\{82154420-0FBF-11d4-8313-005004526AB4} /ve /t REG_SZ /d "LibreOffice Service Manager (Ver 1.0)" /f
reg add HKEY_CLASSES_ROOT\CLSID\{82154420-0FBF-11d4-8313-005004526AB4} /v AppID /t REG_SZ /d "{82154420-0FBF-11d4-8313-005004526AB4}" /f
reg add HKEY_CLASSES_ROOT\CLSID\{82154420-0FBF-11d4-8313-005004526AB4}\LocalServer32 /ve /t REG_EXPAND_SZ /d "%* --nodefault --nologo" /f
reg add HKEY_CLASSES_ROOT\CLSID\{82154420-0FBF-11d4-8313-005004526AB4}\NotInsertable /ve /t REG_SZ /d "" /f
reg add HKEY_CLASSES_ROOT\CLSID\{82154420-0FBF-11d4-8313-005004526AB4}\ProgID /ve /t REG_SZ /d "com.sun.star.ServiceManager.1" /f
reg add HKEY_CLASSES_ROOT\CLSID\{82154420-0FBF-11d4-8313-005004526AB4}\Programmable /ve /t REG_SZ /d "" /f
reg add HKEY_CLASSES_ROOT\CLSID\{82154420-0FBF-11d4-8313-005004526AB4}\VersionIndependentProgID /ve /t REG_SZ /d "com.sun.star.ServiceManager" /f
reg add HKEY_CLASSES_ROOT\com.sun.star.ServiceManager /ve /t REG_SZ /d "LibreOffice Service Manager" /f
reg add HKEY_CLASSES_ROOT\com.sun.star.ServiceManager\CLSID /ve /t REG_SZ /d "{82154420-0FBF-11d4-8313-005004526AB4}" /f
reg add HKEY_CLASSES_ROOT\com.sun.star.ServiceManager\CurVer /ve /t REG_SZ /d "com.sun.star.ServiceManager.1" /f
reg add HKEY_CLASSES_ROOT\com.sun.star.ServiceManager\NotInsertable /ve /t REG_SZ /d "" /f
reg add HKEY_CLASSES_ROOT\com.sun.star.ServiceManager.1 /ve /t REG_SZ /d "LibreOffice Service Manager (Ver 1.0)" /f
reg add HKEY_CLASSES_ROOT\com.sun.star.ServiceManager.1\CLSID /ve /t REG_SZ /d "{82154420-0FBF-11d4-8313-005004526AB4}" /f
reg add HKEY_CLASSES_ROOT\com.sun.star.ServiceManager.1\NotInsertable /ve /t REG_SZ /d "" /f

The path argument to the bat file has to be a Windows path, so if you call it from the cygwin shell, you need to use something like cygpath -w -a -l ./instdir/program/soffice.exe.

Using Voltron

Debugging build tools

In case the build breaks when invoking some tool that is actually custom built during the build process, there is likely a bug somewhere in the code of that tool. You can use the BUILDTOOLTRACE environment variable to run the build tool in strace, valgrind or a debugger:

make BUILDTOOLTRACE='strace' PARALLELISM=1              # <span lang="en" dir="ltr" class="mw-content-ltr">run in strace</span>
make BUILDTOOLTRACE='gdb --args' PARALLELISM=1          # <span lang="en" dir="ltr" class="mw-content-ltr">debug with gdb</span>
make BUILDTOOLTRACE='$(DEVENV) /debugexe' PARALLELISM=1 # <span lang="en" dir="ltr" class="mw-content-ltr">debug with Visual Studio</span>

Running cppunit tests

To run cppunnit test foo in module bar, do:

cd bar && make CppunitTest_foo

The foo part is what is shown in a non-verbose build log, and its first component will usually be the module (the bar part); if not, run grep foo */*.mk to find out. E.g. the test run when the build system prints

[build CUT] sw_subsequent_ooxmlexport

can be manually run with

cd sw && make CppunitTest_sw_subsequent_ooxmlexport

Debugging cppunit tests

At build time, if a cppunit test crashes you can get a debugger in there with...

$ export CPPUNITTRACE="gdb --args"

now when you build (forcing a non-parallel build with -j1), gdb will start with the cppunit test loaded, type "run" to execute the test under gdb. This option used to be GDBCPPUNITTRACE before October 2013

On Windows you can start the unit test in Visual studio with:

$ export CPPUNITTRACE="\"<span lang="en" dir="ltr" class="mw-content-ltr">path_to_your_devenv.exe</span>\" /debugexe"

with path_to_your_devenv.exe being the absolute path + devenv.exe. This will start Visual studio and you start the test directly from the UI.

On macOS you no longer have access to gdb which has been replaced by lldb. However you can still use CPPUNITTRACE through

$ export CPPUNITTRACE="lldb --"

Alternatively, if the test crashes and complains about an uncaught execption you can trace a cppunit test to find where the last exception was thrown from with

$ export DEBUGCPPUNIT=TRUE

which will log the throws and catches to gdbtrace.log.

You can run a specific test case method inside the CppUnit make target (which normally has lots of individual test case methods) using

$ CPPUNIT_TEST_NAME=testFDO76163 make CppunitTest_foo

Debugging cppunit tests with strace

If gdb backtraces are not helpful, you can try an strace with

$ make <span lang="en" dir="ltr" class="mw-content-ltr">CppunitTest_Test_Name</span> CPPUNITTRACE="strace -f -s 77" 2>&1 | tee strace.log

The -s parameter increases the character limit to 77

Debugging perfcheck and other cppunit tests that run valgrind

For performance tests (make perfcheck) or tests running under valgrind (see paragraphs below) the tests can not be debugged directly. However valgrind includes a gdbserver and the test can be run after exporting

$ export VALGRIND_GDB=TRUE

then start the test normally and start gdb with

$ gdb workdir/LinkTarget/Executable/cppunittester

and in the gdb prompt

$ target remote | vgdb

Debugging executables

LibreOffice has a number of bundled demo executables in the VCL module. To debug these, you first set the LOTRACE environment variable to point to your debugger. On Linux, this is:

$ export LOTRACE="gdb --args"

On Windows you can start the executable in Visual studio with:

$ export LOTRACE="\"<span lang="en" dir="ltr" class="mw-content-ltr">path_to_your_devenv.exe</span>\" /debugexe"

with path_to_your_devenv.exe being the absolute path + devenv.exe. This will start Visual studio and you start the test directly from the UI.

On macOS you no longer have access to gdb which has been replaced by lldb. However you can still use LOTRACE through

$ export LOTRACE="lldb --"

To run the app, you use the bin/run script:

$ bin/run vcldemo

Valgrinding (memcheck) cppunit tests

At build time, you can memcheck the cppunit and other tests with

$ export VALGRIND=memcheck

This will automatically set G_SLICE=always-malloc and should cause the cppunit tests and the hunspell regression tests to be run under valgrind --tool=memcheck

Valgrinding (memcheck) LibreOffice itself

$ export VALGRIND=memcheck

This will automatically set G_SLICE=always-malloc and cause LibreOffice to run itself under valgrind with --tool=memcheck

Valgrinding (helgrind) cppunit tests

At build time, you can helgrind the cppunit and other tests with

$ export VALGRIND=helgrind

Valgrinding (helgrind) LibreOffice itself

$ export VALGRIND=helgrind

This will cause LibreOffice to run itself under valgrind with --tool=helgrind

Running the subsequent tests

A top-level make check will first do a full build, then run all the subsequent tests, while a top-level make subsequentcheck will only run all the subsequent tests.

You can also do these from within a module, or one module only.

You can run a single subsequent test via its target (look into foo/Module_foo.mk), e.g. cd sw && make -rs JunitTest_sw_complex. The cd foo part is not necessary, but speeds up the process.

If tests fail, it may be due to a locale different from "en-US". In this case, run an "export LANG=C" and give it a new try.

Debugging the subsequent tests

There will be a log file of the failed test, you can look at it with a text editor:

workdir/JunitTest/<module>_<complex|unoapi>/done.log

The log file will contain a Java stack trace of the failed test, and if the soffice.bin crashed and left a core file, then you will see a C++ stack trace of the crash as well.

If there is no crash, then a look at the Java stack trace should point you to the Java test code that fails; the most interesting frames are usually in classes complex.<module>...", and the code for these is in <module>/qa/complex/. This should point you to some interesting UNO API method that is called on the C++ side in soffice.bin.

Now for debugging, this will start the office installation from the instdir inside of gdb:

make debugrun

First make the horrible gdb TUI go away with "C-x a", then set a breakpoint at the offending method, and run, and a start center should pop up.

Then you can run (in another terminal):

make gb_JunitTest_DEBUGRUN=T <module>.subsequentcheck

This will execute the test against the running soffice.bin, and hopefully the breakpoint should be hit, and you can debug the problem from there.

Debugging the qadevOOo/unoapi subsequent tests

The qadevOOo/unoapi tests are somewhat tricky to debug.

If a test fails, then first you need to find out which method it complained about, open this file in a text editor:

workdir/JunitTest/<module>_unoapi/done.log

At the end there is the following summary, pointing out the component test that failed:

Failures that appeared during scenario execution:
    toolkit.AccessibleStatusBarItem
1 of 53 tests failed

Search for FAIL. This shows an error such as:

LOG> getCharacterBounds(6)
LOG> Text at this place:  
LOG> Character bounds outside component
LOG> Character rect: 43, -566, 0, 0
LOG> Component rect: 91, 2, 71, 18
Method getCharacterBounds() finished with state FAILED
LOG> getCharacterBounds(): PASSED.FAILED

If you scroll up a bit you see a line like this:

checking: [toolkit.AccessibleStatusBarItem::com::sun::star::accessibility::XAccessibleText] is iface: [com.sun.star.accessibility.XAccessibleText] testcode: [ifc.accessibility._XAccessibleText]

Which points at the Java test code that is executed here, ifc.accessibility._XAccessibleText, corresponding to qadevOOo/tests/java/ifc/accessibility/_XAccessibleText.java.

There is also a Java setup code specific to the tested component toolkit.AccessibleStatusBarItem in qadevOOo/tests/java/mod/_toolkit/AccessibleStatusBarItem.java.

Now reduce the test a bit for faster testing: edit the corresponding scenario file, usually named <module>/qa/unoapi/<module>.sce, and remove everything except the one line that corresponds to the failing test, here AccessibleStatusBarItem, and check that it still fails:

Now the difficult part in this case is finding out where the failing method is implemented; often (e.g. in Writer) the class will be named almost the same as the tested component, but in this example checkCharacterBounds surprisingly it's not actually in the toolkit module, but git grep points at accessibility/source/standard/vclxaccessiblestatusbaritem.cxx, which contains a VCLXAccessibleStatusBarItem::getCharacterBounds method.

Once you have found out this information, proceed with make debugrun etc. as described in the previous section #Debugging the subsequent tests.

Debugging with rr

rr works fine to debug LibreOffice, even with fancy stuff like in-process JVM. Note that rr currently requires Linux and a recent Intel CPU.

To record LO itself, use:

rr record instdir/program/soffice

(Note: To avoid crashes when using rr versions up to 5.3.0 with recent libc++ versions, setting environment variable SAL_RAND_REPEATABLE=1 (s. the #Environment_Variables section) might help as a workaround (fixed in https://github.com/mozilla/rr/commit/862605a8d4abca6d28d2296ccc6d6148ffc93ff6 ).

To replay, you want to start with the soffice.bin process:

rr replay -p $(rr ps | grep soffice.bin | cut -f 1 | tail -n 1)

On current master towards libreoffice-6-2, all tests (CppUnitTest, JUnitTest, PythonTest, UITest) can be recorded by setting the environment variable RR=1. This requires ~35GB of storage per make check run.

There was some success with getting Eclipse CDT 9.4.3 to use rr as the debugger, following the instructions in the rr documentation, using a Debug Configuration derived from "C/C++ Application" (and with the full path to soffice.bin in the Application field), but with this slightly enhanced rrgdb wrapper script:

#!/bin/bash
dir=/home/foobar/.local/share/rr/latest-trace/
pid=$(rr ps $dir | grep soffice.bin | cut -f 1 | tail -n 1)
exec rr replay -p $pid $dir -- "$@"

In order for Qt Creator (tested with 4.10 release candidate) to find debug info, the sysroot for GDB needs to be explicitly set, which can e.g. be achieved by adding the following line in "Tools" -> "Options" -> "Debugger" -> "GDB" -> "Additional Startup Commands":

set sysroot /

Searching for a memory corruption on Windows using DrMemory

If you have a crasher bug on Windows, and the stack trace is inside 'malloc' or 'free' or 'new' or 'delete' then most likely you have a memory corruption - often intermittent bugs are these too. For these cases, there is a wonder-new-tool (for Windows), called DrMemory; you get it here: http://www.drmemory.org/

You need to install it and enable its insertion into your system path. Then either get a release build from TDF: https://download.documentfoundation.org/libreoffice/stable/ or a daily build from the debug box TB39: https://dev-builds.libreoffice.org/daily/master/Win-x86@39/

Then you need to rename the file soffice.bin to soffice.exe in the LibreOffice's program/ directory. The original soffice.exe is just a trivial wrapper binary.

Finally you'll need a console of some sort; as of now, in order to get anything sensible from the tool, you want to run the following from inside LibreOffice's program/ directory:

drmemory -no_count_leaks -ignore_asserts  -no_check_uninitialized -- soffice.exe

That means you get rather further, hopefully to the point where it crashes with your bug. Since the file-picker crashes drmemory itself, you'll need to use 'recent files' or the command-line to be able to load your document.

Expect it to be -really- slow; that's normal :-) but it is doing some clever things. Hopefully at the end of the day, your bug yields an:

Error #7: UNADDRESSABLE ACCESS: writing 0x2b9ca0f4-0x2b9ca0f8 4 byte(s)

error log, which is a serious error and a very helpful trace around it.

Running CppUnit tests with DrMemory

You can run any CppUnit test with DrMemory for tracking down uninitialized memory accesses and memory management bugs like this:

CPPUNITTRACE="drmemory -no_check_gdi -free_max_frames 30 -suppress C:/Users/xxx/drmemory-suppressions.txt" make CppunitTest_sw_uiwriter

Note that:

  • DrMemory tends to run out of memory itself and die while reporting the copious memory leaks in some of the bigger CppUnit tests in 32-bit builds; to avoid that use the arguments -no_count_leaks -no_check_handle_leaks .
  • DrMemory tends to grind to a halt (or at least, 2 hours of continuous CPU-time were observed with 1.9.0-4 before running out of patience) when running java.exe, which is spawned by several unit tests, notably CppunitTest_dbaccess_hsqldb_test and CppunitTest_dbaccess_RowSetClones and CppunitTest_services. To work around this problem, you can use -no_follow_children, or configure DrMemory to ignore java.exe by running drconfig.exe -quiet -reg java.exe -norun. (Strangely, the in-process jvm.dll is less problematic: millions of errors are produced, but they can easily be suppressed, as described below.)
  • DrMemory does not have an equivalent of valgind's memcheck's --track-origins=yes, so tracking down the root cause of uninitialized memory accesses often involves some additional work; in such cases try if you can reproduce the problem with valgrind on another platform to get a better stack trace.
  • DrMemory reports false positives in JPEG images imported by the SSE2 code in jpeg-turbo https://github.com/DynamoRIO/drmemory/issues/540. Unfortunately it may do so in places far away from the JPEG import filter, for example in CppunitTest_sw_globalfilter the UNINITIALIZED READ errors are reported when exporting the VCL Bitmap to a PNG. The work-around is to force jpeg-turbo to stop using SSE2 by setting an environment variable: export JSIMD_FORCEMMX=1.
  • DrMemory reports various other false positives than can be suppressed via the -suppress argument.

Here is a sample drmemory-suppressions.txt for false positives encountered when running the CppunitTests with DrMemory-1.9.0-4:

UNADDRESSABLE ACCESS
name=suppress all UA in java.exe
java.exe!*

UNINITIALIZED READ
name=suppress all UR in java.exe
java.exe!*

UNADDRESSABLE ACCESS
name=suppress all UA in jvm.dll
jvm.dll!*

UNINITIALIZED READ
name=suppress all UR in jvm.dll
jvm.dll!*

WARNING
name=suppress all warning in jvm.dll
jvm.dll!*

UNADDRESSABLE ACCESS
name=UA in JIT code from jvm.dll
<not in a module>
...
jvm.dll!*

UNINITIALIZED READ
name=UR in JIT code from jvm.dll
<not in a module>
...
jvm.dll!*

INVALID HEAP ARGUMENT
name=https://connect.microsoft.com/VisualStudio/feedback/details/750951/std-locale-implementation-in-crt-assumes-all-facets-to-be-allocated-on-crt-heap-and-crashes-in-destructor-in-debug-mode-if-a-facet-was-allocated-by-a-custom-allocator
drmemorylib.dll!replace_free
*!std::_DebugHeapDelete<>
*!std::_Fac_node::~_Fac_node
*!std::_Fac_node::`scalar deleting destructor'
*!std::_DebugHeapDelete<>
*!std::_Fac_tidy_reg_t::~_Fac_tidy_reg_t
*!std::`dynamic atexit destructor for '_Fac_tidy_reg
*!_CRT_INIT
*!__DllMainCRTStartup
*!_DllMainCRTStartup
ntdll.dll!RtlQueryEnvironmentVariable
ntdll.dll!LdrShutdownProcess
ntdll.dll!RtlExitUserProcess
KERNEL32.dll!ExitProcess

UNINITIALIZED READ
name=https://github.com/DynamoRIO/drmemory/issues/1824 (input UR)
system call NtUserGetClipboardFormatName UNICODE_STRING.MaximumLength
sysdtrans.dll!CDataFormatTranslator::getClipboardFormatName

UNADDRESSABLE ACCESS
name=https://github.com/DynamoRIO/drmemory/issues/1824 (input UA)
system call NtUserGetClipboardFormatName UNICODE_STRING content
sysdtrans.dll!CDataFormatTranslator::getClipboardFormatName

UNINITIALIZED READ
name=https://github.com/DynamoRIO/drmemory/issues/1824 (output1)
sal3.dll!*
sal3.dll!rtl_ustr_compareIgnoreAsciiCase_WithLength
ftransl.dll!rtl::OUString::equalsIgnoreAsciiCase
ftransl.dll!CDataFormatTranslator::findDataFlavorForNativeFormatName
ftransl.dll!CDataFormatTranslator::getDataFlavorFromSystemDataType
sysdtrans.dll!CDataFormatTranslator::getDataFlavorFromFormatEtc
sysdtrans.dll!CDOTransferable::formatEtcToDataFlavor
sysdtrans.dll!CDOTransferable::initFlavorList
sysdtrans.dll!CDTransObjFactory::createTransferableFromDataObj

UNINITIALIZED READ
name=https://github.com/DynamoRIO/drmemory/issues/1824 (output2)
sal3.dll!rtl::compareIgnoreAsciiCase
sal3.dll!rtl_ustr_compareIgnoreAsciiCase_WithLength
sysdtrans.dll!rtl::OUString::equalsIgnoreAsciiCase
sysdtrans.dll!CDataFormatTranslator::isTextHtmlFormat

UNINITIALIZED READ
name=https://github.com/DynamoRIO/drmemory/issues/1825
system call NtGdiAddFontResourceW parameter value #4
GDI32.dll!GdiAddFontResourceW
GDI32.dll!AddFontResourceExW
vcllo.dll!ImplAddTempFont
vcllo.dll!WinSalGraphics::AddTempDevFont
vcllo.dll!OutputDevice::AddTempDevFont

UNINITIALIZED READ
name=https://github.com/DynamoRIO/drmemory/issues/1827
*
KERNELBASE.dll!WaitNamedPipeW
sal3.dll!osl_createPipe

UNINITIALIZED READ
name=CPython custom allocator PyObject_Realloc
python??_d.dll!PyObject_Realloc

UNINITIALIZED READ
name=CPython custom allocator PyObject_Free
python??_d.dll!PyObject_Free

UNINITIALIZED READ
name=CPython custom allocator PyObject_Realloc
python??.dll!PyObject_Realloc

UNINITIALIZED READ
name=CPython custom allocator PyObject_Free
python??.dll!PyObject_Free

WARNING
name=prefetching unaddressable memory in jpeg-turbo
vcllo.dll!jsimd_idct_islow_sse2
vcllo.dll!jsimd_idct_islow
vcllo.dll!decompress_data

Debugging C++ UNO life cycles

The reference count is stored as m_refCount, so e.g. break in the UNO object constructor and add a watch to it to see who takes shared ownership of the object.

(gdb) watch * (&m_refCount)

bin/refcount_leak.py

If acquire() and release() calls on an UNO service are not matched, the object will leak. There is a script that can parse gdb backtraces and try to balance acquire() and release() and sort them by how likely they are.

For usage hints see the comments at the top of bin/refcount_leak.py in the core repository.

Beware that gdb takes a lot of time to print backtraces; 4000 backtraces take > 3 hours on a laptop with a current 15W TDP CPU.

Another disadvantage is that the result of the script requires some manual interpretation, but it can detect bare calls to acquire() that leak.

instrument uno::Reference

There is a patch on gerrit that adds a dummy memory allocation into every uno::Reference so that standard tools like valgrind and address sanitizer can detect when the uno::Reference itself is leaked.

This may be the easiest way to track down a leak, but the disadvantages are that it cannot detect bare acquire() calls and that instrumentation requires a full rebuild; also the added global lock may cause deadlocks with configmgr.

Note that the patch is currently incomplete and may not detect leaks from uno::Any and rtl::Reference (but that could be fixed).

Assertions and Logging

Environment Variables

Note:
To see the complete list of environment variables used in LibreOffice, see Development/Environment variables.

These environment variables are useful for debugging purpose:

  • OOO_DISABLE_RECOVERY=1 disable recovery of corrupted documents on startup.
  • OOO_EXIT_POST_STARTUP=1 exit immediately after opening document. Useful for debugging open performance.
  • SAL_DISABLEGL=1 disable use of OpenGL
  • SAL_DISABLE_OPENCL=1 disable use of OpenCL in calc.
  • SAL_NO_MOUSEGRABS=1 prevents LibreOffice from grabbing the mouse during debugging on X11.
  • SAL_RAND_REPEATABLE=1 makes the random number generator start from a fixed seed, which makes tests that use random numbers predictable.
  • SAL_USE_VCLPLUGIN=gen/kf5/gtk3 force the use of a specific VCL UI backend.
  • SW_DEBUG=1 enable writer document dump keybinds (F12 for layout.xml, Shift+F12 for nodes.xml)
  • SD_DEBUG=1 enable draw graphic object dump keybind (F12 for model.xml)

Macros Controlling Debug Code

  • The NDEBUG macro is the standard way to control the standard assert functionality. It is defined in plain production builds, left undefined for --enable-debug/--enable-dbgutil (note that defining it disables assertions).
  • The SAL_LOG_INFO and SAL_LOG_WARN macros control whether the SAL_INFO and SAL_WARN functionality, resp., from include/sal/log.hxx is activated. They are left undefined in plain production builds, defined for --enable-debug/--enable-dbgutil. If activated, their runtime behaviour is controlled by the SAL_LOG environment variable (see the documentation in include/sal/log.hxx for details).

To enable SAL_INFO and SAL_WARN for sw component type:

export SAL_LOG="+INFO.sw.ww8+WARN"
  • The DBG_UTIL macro enables additional code that potentially affects ABI compatibility, changing public data structures. (So enabling it is an all-or-nothing decision; you generally cannot build just part of LibreOffice with this enabled. For historical reasons, it also controls the obsolete DBG_ASSERT etc. macros from include/tools/debug.hxx.) It is left undefined in production builds, and defined for --enable-dbgutil.
  • The _GLIBCXX_DEBUG macro enables helpful assertions in the libstdc++ STL implementation, which affect ABI compatiblity; it is also enabled by --enable-dbgutil on ELF based GCC platforms (TODO: this could work on all GCC platforms if only somebody tested it).
  • The OSL_DEBUG_LEVEL macro controls additional, potentially excessively expensive debug code (but which does not affect compatibility). It is defined as 0 in plain production builds, as 1 for --enable-debug/--enable-dbgutil (enabling the obsolete OSL_ASSERT etc. macros from include/osl/diagnose.h), and as 2 or higher with an explicit dbglevel=N argument to make.

Debugging Python components in LibreOffice

To debug the loading of python components, change the line DEBUG=0 in pythonloader.py.

To debug scripts using the Python Script Provider, set the variable PYSCRIPT_LOG_LEVEL=DEBUG and (optionally) PYSCRIPT_LOG_STDOUT=0 to redirect to the file $UserInstallation/Scripts/python/log.txt.

To see the method calls executed by the pyuno Python/UNO bridge, set the environment variables PYUNO_LOGLEVEL=ARGS and (optionally) PYUNO_LOGTARGET=mylogfile.

When starting soffice in a terminal, pdb can be used as a python-level debugger; to invoke it and effectively set a breakpoint, add this line in an appropriate place in your python code, typically during initialisation:

import pdb; pdb.set_trace()

Another option is to use gdb for debugging, which can load a custom pretty-printing file that matches the python library that is used by LO. Then commands such as py-bt, py-bt-full, py-print, py-locals, py-up, py-down, py-list become available at the gdb prompt, in addition to gdb's own C++ debugging commands. See documentation at [١].

For the python that is bundled with LO, gdb debugging is enabled by instdir/program/libpython3.5m.so.1.0-gdb.py; if LO is built against a system python, the file might be in some non-obvious place; on Fedora 29 it can be installed with

sudo dnf --enablerepo=fedora-debuginfo --enablerepo=updates-debuginfo install python3-debuginfo.

Debugging memory leaks with valgrind (including ref-count leaks)

Say you know that a given unit test is leaking.

Then Run a unit test like this:

$ make CppunitTest_cppcanvas_test VALGRIND='memcheck --vgdb=yes --vgdb-error=0 --leak-check=full --suppressions=$$BUILDDIR/solenv/sanitizers/valgrind-suppressions'

then in another terminal do

$ gdb workdir/LinkTarget/Executable/cppunittester

and once that gdb comes, enter the command

(gdb) target remote | vgdb

now you can execute commands like setting breakpoint, or continue, and it will control the program running in the other terminal

Most usefully, you can set a breakpoint at a location like

(gdb) br cppunittester.cxx:473

which is just after the unit test has finished, and then you can set a breakpoint at the constructor of the object you are interested in, like

(gdb) br MyObject::MyObject

then take note of the hex value of the "this" pointer when the breakpoint triggers. Then when the end-of-unit-test breakpoint triggers, valgrind can tell you who is still harbouring a pointer to the object you are interested in with:

(gdb) monitor who_points_at 0xegegegeg

For a summary of the available valgrind memcheck debugging commands, see Memcheck Monitor Commands.

Debugging Java components in LibreOffice

If you want to debug the parts of LibreOffice that are implemented in Java, GDB is not useful as it currently does not support Java as Java runs in it's own virtual machine.

It may occasionally be useful to just get the Java level stack trace, like when there is a deadlock with Java code potentially involved; the jstack tool can print it, just give it the process id of soffice.bin as argument.

For actual debugging an IDE is most convenient (although there is also a command line debugger jdb). To get this set up, you have to use remote Java debugging in eclipse for example.

Preparing LibreOffice to enable Debugging

To enable the Java Virtual Machine to be debugged, you start LibreOffice normally and enable the debugging by adding the jvm version you use to the Tools settings currently under Tools > LibreOfficeDev > Advanced and add the following parameters:

-Xdebug
-Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n

the first of the two enables debugging, the second sets the port of the Virtual Machine, if you enable suspend (writing suspend=y instead of suspend=n) the Machine is halted until a debugger attaches to it which can be used to debug the starting progress of the jvm.

Preparing Eclipse for Debugging

In Eclipse, make sure you opened the libreoffice folder as a Java Project, then you can add a debug configuration of the type Remote Java Application This debug configuration should have:

the Connection type: Standard (socket attach)

and the Connection Properties:

Host: localhost

Port: 8000

This port should be the same as the one you prepared inside Libreoffice. If you then start the module that is in Java, and during that starting progress start the debugging configuration you should find it working at stopping at your breakpoints.

Performance debugging (Callgrind)

Callgrind is the most commonly used tool for searching for performance issues. See the following for instructions:

Visualize results

To visualize results KCachegrind could be used.

  1. Turn off cycle detection. The results are often nearly meaningless for non-trivial cases.
  2. Turn off the 'Relative' button and use absolute cycle counts everywhere or it is very easy to lose a sense of proportion.
  3. Check all counts vs. the total in the bottom status bar for sanity.

For an alternative way to look at the results, you could try gprof2dot.

Performance debugging (perf)

The linux kernel profiler (perf) produces less detailed output than callgrind, but is much cheaper to run (since it's a sampling profiler). With callgrind or perf, you should not use a debug build, but one built with --enable-symbols.

You may need extra permissions i.e.

sudo sh -c "echo 0 > /proc/sys/kernel/kptr_restrict"

or also:

sudo sh -c 'echo 1 >/proc/sys/kernel/perf_event_paranoid'

in order to use kernel symbols in the output.

Use

perf record -g --pid=`pidof soffice.bin`

to capture data, and then

perf report

to see the output.

For a really nice visualization of the result, use KDAB Hotspot.

You can export a flamegraph from Hotspot via View ▸ Export ▸ Flamegraph. Formats available are BMP and SVG.

If you find that the resulting data is too coarse, there are two options

  • Capture data for a longer period of time, perhaps by performing the action more than once
  • Increase the profiler sampling frequency e.g.

perf record -g -F 10000 --pid=`pidof soffice.bin`

If you find that parts of the call stacks are missing, you may need to increase the size of the stack capture with:

perf record --call-graph dwarf,65528 --pid=`pidof soffice.bin`