How to get a backtrace with WinDbg

    From The Document Foundation Wiki

    Introduction

    LibreOffice, as all software, can halt or crash while in use on the Windows operating system. In such an unfortunate event, some debugging data can be gathered by the user and submitted to the LibreOffice developer team, so that they can fight the bug. This debugging data is called a backtrace or stack trace and it can be submitted in a Bugzilla bug report as a text file attachment. If you are a user of LibreOffice 4.2.0 or higher, this article will take you step by step through the process of how to get a backtrace from the Windows debugger tool, WinDbg.

    First Time Setup

    1. Install WinDbg using the Standalone Debugging Tools for Windows, which will take up approximately 300 Mb of disk space
      • Windows 10 Users
        Download the Windows 10 SDK (direct link) and during its installation, clear all checkboxes except 'Debugging Tools for Windows'.
      • Installing WinDbg in Windows 7
        Windows 7 and 8 Users
        Download the Windows SDK for Windows 8.1 [direct link] and during its installation, clear all checkboxes except 'Debugging Tools for Windows'. (This requires that you already have .NET Framework 4.5 redistributable preinstalled, but that seems to be the case for an up-to-date installation of Windows 8.1 at least.)
      • Windows XP and Vista Users
        Download the Windows SDK for .NET 4.0 or .NET 3.5 SP1 and during its installation, clear all checkboxes except 'Debugging Tools for Windows'. This requires that you already have .NET 4.0 or .NET 3.5 SP1 already installed.
      • Installation Notes
        The installation needs to be run with administrator access, so if your user account isnt an administrator, then right-click the .msi file and select "execute as administrator". If you get an error during the installation, then its likely that you need to remove the Visual C++ 2010 Redistributable found on your system.
        WinDbg in Start Menu
    2. Attach Symbol Sources to Debug Tool
      • Using WinDbg
        Start WinDbg from within the Windows Start Menu. It is found in the Debugging Tools for Windows folder. In Windows 10, this folder is inside the Windows Kits folder.

        Note: If the folder is not present, please re-run the debugging tools installer, click the 'Change' option, and then reselect 'Debugging Tools for Windows', and continue with the installation.

        Once WinDbg has been opened, click on File ▸ Symbol File Path... and paste the following into the space provided.
        CACHE*C:\symbols;SRV*https://dev-downloads.libreoffice.org/symstore/symbols;SRV*https://msdl.microsoft.com/download/symbols
        Here https://dev-downloads.libreoffice.org/symstore/symbols is for debugging release builds that have symbols. Alternatively if you wish to also debug both the standard build and master build (Win-x86@39 tinderbox), paste the following instead
        CACHE*C:\symbols;SRV*https://dev-builds.libreoffice.org/daily/master/Win-x86@39/symbols;SRV*https://dev-downloads.libreoffice.org/symstore/symbols;SRV*https://msdl.microsoft.com/download/symbols
        Adding Symbols
        Now press OK and choose File ▸ Save workspace to permanently set the symbols search path.
      • Using Visual Studio
        Backtraces can be done within Visual Studio Express 2005 or later by opening Tools ▸ Options ▸ Debugging ▸ Symbols and adding the below URLs to the symbol servers.
        https://dev-builds.libreoffice.org/daily/master/Win-x86@39/symbols
        https://dev-downloads.libreoffice.org/symstore/symbols

    Debugging

    To begin debugging, start up WinDbg. You will find screenshots at the end of this section to assist you.

    If LibreOffice doesn't crash when it starts, then start LibreOffice and in WinDbg, press F6 or click File ▸ Attach to a Process.... In the dialog that appears, select soffice.bin from the list of filename (it normally is the last one in the list if LibreOffice was the last application you started). If LibreOffice crashes before it fully starts, then in WinDbg, press Ctrl + E or click File ▸ Open Executable.... In the dialog that appears, locate and select soffice.exe from the folder in which it is installed and check the Debug child processes also checkbox at the bottom of the dialog.

    Once you press the OK button in either of the dialogs, a series of lines will be printed to window, the last of which will have int 3 on it.

    Note: The first time you run this after setting up the symbols sources, it will begin downloading the symbols, so you will need an internet connection and it may take a few minutes to complete.

    Microsoft (R) Windows Debugger Version 6.12.0002.633 X86
    Copyright (c) Microsoft Corporation. All rights reserved.
    *** wait with pending attach
    Symbol search path is: c:\lodev\symbols;SRV*c:\Symbols\mssymbols\*https://msdl.microsoft.com/download/symbols
    Executable search path is: 
    ModLoad: 01280000 0132a000   C:\Program Files (x86)\LOdev 3.5\program\soffice.bin
    [...]
    ModLoad: 5f520000 5fa4d000   C:\Program Files (x86)\LOdev 3.5\program\xolo.dll
    (13b8.d64): Break instruction exception - code 80000003 (first chance)
    eax=7efaf000 ebx=00000000 ecx=00000000 edx=77d7f85a esi=00000000 edi=00000000
    eip=77cf000c esp=08d3fa30 ebp=08d3fa5c iopl=0         nv up ei pl zr na pe nc
    cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
    ntdll!DbgBreakPoint:
    77cf000c cc              int     3
    

    You will then see the text cursor blinking in the command field. At this point, you can either press F5, click Debug ▸ Go or type the letter g and press enter, so that the debugging will start. You will need to repeat this a second time if you used Ctrl + E or clicked File ▸ Open Executable... to start LibreOffice. Once debugging has started, the command field will turn gray and Debuggee is running... will be written in it, with the word *BUSY* appearing in the field to its left.

    If LibreOffice hasn't already crashed, switch over to LibreOffice and attempt to reproduce the halt or crash. Once the crash has occurred, switch back to WinDbg and you will see that a series of lines have been printed in the window, the last of which will normally have a number of question marks at its end.

    0:011> g
    ModLoad: 05950000 05bfc000   C:\Program Files (x86)\LOdev 3.5\program\fwklo.dll
    ModLoad: 05950000 05bfc000   C:\Program Files (x86)\LOdev 3.5\program\fwklo.dll
    (13b8.dd8): C++ EH exception - code e06d7363 (first chance) 
    [...]
    ModLoad: 74a30000 74a36000   C:\Windows\SysWOW64\sensapi.dll
    (13b8.dd8): Access violation - code c0000005 (first chance)
    First chance exceptions are reported before any exception handling.
    This exception may be expected and handled.
    eax=9162b873 ebx=00000000 ecx=9162b873 edx=9162b873 esi=602ff600 edi=00000000
    eip=6623b085 esp=00bec8fc ebp=00bec908 iopl=0         nv up ei ng nz na po nc
    cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010282
    sal3!rtl_uString_newFromAscii+0x15:
    6623b085 0fbe11          movsx   edx,byte ptr [ecx]         ds:002b:9162b873=???????
    

    You will notice that the command field has returned to white, so click in the field and type in !analyze -v, followed by pressing enter. WinDbg will then start exception analysis and you will see *BUSY* appear again. Wait patiently for the output to complete.

    0:000> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Exception Analysis                                   *
    *                                                                             *
    *******************************************************************************
    FAULTING_IP: 
    sal3!rtl_uString_newFromAscii+15 [c:\lodev\source\sal\rtl\source\ustring.cxx @ 437]
    6623b085 0fbe11          movsx   edx,byte ptr [ecx]
    EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
    ExceptionAddress: 6623b085 (sal3!rtl_uString_newFromAscii+0x00000015)
       ExceptionCode: c0000005 (Access violation)
      ExceptionFlags: 00000000
    NumberParameters: 2
       Parameter[0]: 00000000
       Parameter[1]: 9162b873
    [...]
    SYMBOL_NAME:  soffice.bin
    FOLLOWUP_NAME:  MachineOwner
    MODULE_NAME: soffice
    DEBUG_FLR_IMAGE_TIMESTAMP:  5358065d
    STACK_COMMAND:  ** Pseudo Context ** ; kb
    FAILURE_BUCKET_ID:  STACKIMMUNE_c0000005_C:_Program Files_(8x6)_LibreOffice_program_soffice.bin!Unknown
    BUCKET_ID:  X64_APPLICATION_FAULT_STACKIMMUNE_NOSOS_INVALID_POINTER_WRITE_ZEROED_STACK_soffice.bin
    WATSON_IBUCKET:  219952521
    WATSON_IBUCKETTABLE:  17
    IMAGE_NAME:  C:\Program Files (x86)\LibreOffice\program\soffice.bin
    WATSON_STAGEONE_URL: https://watson.microsoft.com/StageOne/soffice_bin/4_2_4_2/5358065d/ntdll_dll/.../000222d2.htm?Retriage=1
    Followup: MachineOwner
    ---------
    

    Once the exception analysis has completed, *BUSY* will have disappeared and you can then select the outputted text starting from the beginning of the exception analysis all the way to the end. You can copy the text by pressing Ctrl + C, clicking Edit ▸ Copy or simply right-clicking.

    Now open up Notepad from within the Start Menu and paste the text into the empty area by pressing Ctrl + V, clicking Edit ▸ Paste, or right-click ▸ Paste. Save the file with Ctrl + S or File ▸ Save and name the file backtrace.txt.

    As LibreOffice is still locked by WinDbg, you need to close WinDbg by pressing Alt + F4 or clicking File ▸ Exit. If a dialog appears asking to "Save information for workspace?", choose "No". This will close both WinDbg and LibreOffice.

    Producing a mini dump

    Sometimes WinDbg cannot catch the LO exception properly. In such case we have to produce a mini dump which then can be loaded and parsed in the debugger. To produce such dump you can use free ProcDump utility as following:

    Ensure your LibreOffice debug build is running.

    In the Windows command prompt type:

    d:\procdump4.01>procdump.exe soffice.bin -h
    

    Reproduce a crash.

    When LibreOffice window hung is detected ProcDump will inform about successful mini dump creation.

    d:\procdump4.01>procdump.exe soffice.bin -h
    ProcDump v4.0 - Writes process dump files
    Copyright (C) 2009-2011 Mark Russinovich
    Sysinternals - www.sysinternals.com
    Process:               soffice.bin (1964)
    CPU threshold:         n/a
    Performance counter:   n/a
    Commit threshold:      n/a
    Threshold seconds:     n/a
    Number of dumps:       1
    Hung window check:     Enabled
    Exception monitor:     Disabled
    Terminate monitor:     Disabled
    Dump file:             d:\procdump4.01\soffice.dmp
    [09:02.39] Hung window detected.
    Writing dump file d:\procdump4.01\soffice_120614_090239.dmp ...
    Dump written.
    Dump count reached.
    d:\procdump4.01>
    

    Open WinDbg and load .dmp file (File ▸ Open Crash Dump...).

    Loading Dump File [D:\soffice_120601_075839.dmp]
    User Mini Dump File: Only registers, stack and portions of memory are available
    Comment: '
    *** procdump  soffice.bin -h
    *** Hung window detected: 110946'
    Symbol search path is: c:\lodev\symbols;SRV*c:\Symbols\mssymbols\*https://msdl.microsoft.com/download/symbols 
    Executable search path is: 
    Windows 7 Version 7601 (Service Pack 1) MP (2 procs) Free x86 compatible
    Product: WinNt, suite: SingleUserTS
    Machine Name:
    Debug session time: Fri Jun  1 07:58:39.000 2012 (UTC + 2:00)
    System Uptime: not available
    Process Uptime: 0 days 0:00:44.000
    ................................................................
    ................................................................
    ............................................................
    Loading unloaded module list
    ..............
    eax=00000000 ebx=00e7b174 ecx=00000000 edx=00000000 esi=00000002 edi=00000000
    eip=77d0013d esp=00e7b124 ebp=00e7b1c0 iopl=0         nv up ei pl zr na pe nc
    cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
    ntdll!ZwWaitForMultipleObjects+0x15:
    77d0013d 83c404          add     esp,4
    0:000>
    

    Type !analyze -v in the command window.

    0:000> !analyze -v
    

    WinDbg will start exception analysis. Wait patiently for the output:

    0:000> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Exception Analysis                                   *
    *                                                                             *
    *******************************************************************************
    FAULTING_IP: 
    +612e2faf0463dc04
    00000000 ??              ???
    EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
    ExceptionAddress: 00000000
       ExceptionCode: 80000003 (Break instruction exception)
      ExceptionFlags: 00000000
    NumberParameters: 0
    FAULTING_THREAD:  00000f90 
    DEFAULT_BUCKET_ID:  STATUS_BREAKPOINT
    PROCESS_NAME:  soffice.bin
    
    [...]
    
    Followup: MachineOwner
    ---------
    

    Copy all session output to new text document and attach to the bug report.

    Producing additional debug output

    In certain situations LO developer can ask you to apply a patch to particular source file and gather additional debug information by using OutputDebugString function available in Windows as following:

    Download free DebugView utility.

    Ensure your LibreOffice debug build and DebugView are run.

    Reproduce a crash.

    DebugView will gather debug information:

    00000001	0.00000000	[5112] WinInetBackend A 759F0000	
    00000002	0.00007052	[5112] WinInetBackend B 759F6F21	
    00000003	0.01316426	[5112] WinInetBackend C 0 14 0	
    00000004	0.01324074	[5112] WinInetBackend D 0 14 0 6F0C82FD 00000028
    

    Copy output to new text document and attach to the bug report.

    Automation

    You can prepare a one-click debugging environment, so that WinDbg starts already attached to a designated LO process (for example to production regularly installed LO or to LO master in parallel installation), with the debuggee running. First save workspace with symbols file path in WinDbg ("Workspace-master" used here). Then make a batch file in Windows, using that workspace name. Adjust path to soffice.exe and timeout required for LO to start, just enough for WinDbg to attach to it.

    cd "C:\Program Files (x86)\LibreOffice 4\program"
    start soffice.exe
    timeout 5
    C: # if PATH_TO is not on C: drive
    cd "C:\Program Files (x86)\Windows Kits\8.1\Debuggers\x86\"
    start windbg.exe -W Workspace-master -pn soffice.bin -g

    Additional Information

    Daily (development) builds of the master branch

    You can find the most current development build from Win-x86@39 tinderbox that you can obtain here:
    https://dev-builds.libreoffice.org/daily/master/Win-x86@39/current
    (It will install into a LibreOfficeDev folder separate from any existing LibreOffice released version installation. It does not override the installation of a released version. Open test files from inside LibreOfficeDev.)

    Testing on a non-Windows OS

    If you'd like to run a Windows backtrace from within a non-Windows operating system, you can setup a virtual environment using these Microsoft provided VHD images. These VHD images can be run within Oracle Virtualbox and VMware Workstation. Alternatively, you can run the Windows installation within any virtualization software, including VMware Player, and QEMU/KVM.

    See also