Development/UITests

    From The Document Foundation Wiki

    TDF LibreOffice Document Liberation Project Community Blogs Weblate Nextcloud Redmine Ask LibreOffice Donate

    Other languages:

    This is the preliminary documentation for the UI testing framework with a special focus on the Python side.

    Architecture

    The UI testing consists of two parts, a C++ piece that wraps the LibreOffice UI in an introspection library and a Python framework with the actual test code. The connection between the two parts is done through a simple mostly untyped UNO interface.

    C++ introspection library

    The C++ introspection library consists of an UNO API that is used for communication with the python part of the framework and wrappers for many UI objects. The UNO interface has only a few methods and most of the communication happens through string parameters.

    The wrapper for the UI objects provides about the same methods as the UNO interface, plus some helper methods.

    Python framework

    Writing tests

    Xisco Faulí's presentation from FOSDEM 2021, LibreOffice QA - how to write your first test, walks through the process of creating tests.

    Xisco's presentation from FOSDEM 2022 shows Things you can test in a UITest.

    Makefile part

    A simple makefile for the UI testing looks like the demo makefile. The important part is to use the UITest gbuild class and let the gb_UITest_add_module point to the directories with test files.

    Normally nothing more is required and most is handled in the gbuild UITest class. 

    $(eval $(call gb_UITest_UITest,range_name))

$(eval $(call gb_UITest_add_modules,range_name,$(SRCDIR)/sc/qa/uitest,\
    test_dir1/ \
    test_dir2/ \
))

    The range_name parameter is the name of the test target. The $(SRCDIR/sc/qa/uitest is the base directory and the parameters in that part lists the directories to search for tests.

    Optionally the test can be run with a specific registrymodifications.xcu file to support running with specific configuration settings. Each test has an own user profile in workdir/UITest/test_name/user/. With the following makefile snippet a file can be copied into that directory. 

    $(eval $(call gb_UITest_use_configuration,range_name,$(SRCDIR)/sc/qa/uitest/range_name/registrymodifications.xcu))

    This would copy the file sc/qa/uitest/range_name/registrymodifications.xcu to the user profile and the test would run with these settings.

    Writing the Python code

    Infrastructure

    Commands available on UI elements

    Generic

    Handling the State of UI objects

    The method to get the state of the UI object is provided by all UI elements. The method returns the state as a sequence of PropertyValue. 

    ui_element.getState()

    The UI testing framework provides a method to convert the returned sequence of PropertyValue to a python dictionary. 

    from uitest.uihelper.common import get_state_as_dict

get_state_as_dict(ui_element)

    Sending keyboard input to an element

    from uitest.uihelper.common import type_text

xWriterDoc = self.xUITest.getTopFocusWindow()
xWriterEdit = xWriterDoc.getChild("writer_edit")
type_text(xWriterEdit, "Test")

    Selecting text

    from uitest.uihelper.common import select_text

xEdit = xAddNameDlg.getChild("edit")
select_text(xEdit, from_pos="2", to="12")

    Setting Zoom

    xWriterEdit = xWriterDoc.getChild("writer_edit")
xWriterEdit.executeAction("SET", mkPropertyValues({"ZOOM": "200"}))

    GOTO page

    xWriterEdit = xWriterDoc.getChild("writer_edit")
xWriterEdit.executeAction("GOTO", mkPropertyValues({"PAGE": "1"}))

    Control Actions

    Control Code Assert
    Button 

    xControl.executeAction("CLICK", tuple())

    ---

    TextBox 

    from uitest.uihelper.common import type_pos

type_text(xControl, "Text")

    		 

    self.assertEqual(get_state_as_dict(xTitleText)["Text"], "Text")

    CheckBox 

    xControl.executeAction("CLICK", tuple())

    		 

    self.assertEqual(get_state_as_dict(xControl)["Selected"], "true")

    Charmap Widget 

    from libreoffice.uno.propertyvalue import mkPropertyValues

xControl.executeAction("SELECT", mkPropertyValues({"COLUMN": "2", "ROW": "2"}))

    ---

    TreeList 

    # Unfold/Fold the tree
xTreelist = xExDBDlg.getChild("availablelb")         
xTreeEntry = xTreelist.getChild('0')
xTreeEntry.executeAction("EXPAND", tuple())  #Available Databases
xTreeEntry.executeAction("COLLAPSE", tuple()) #Click on the Bibliography

xTreeEntry2 = xTreeEntry.getChild('0')
xTreeEntry2.executeAction("SELECT", tuple())

    ---

    ComboBox 

    from uitest.uihelper.common import select_pos

select_pos(xCombobox, "1")

    		 

    self.assertEqual(get_state_as_dict(xNumbering)["SelectEntryText"], "List 1")

    Tabs 

    from uitest.uihelper.common import select_pos

xTabs = xDialog.getChild("tabcontrol")
select_pos(xTabs, "0")

    		 

    self.assertEqual(get_state_as_dict(xTabs)["CurrPagePos"], "0")

    ListBox ( by position) 

    from uitest.uihelper.common import select_pos

select_pos(xListBox, "1")

    		 

    self.assertEqual(get_state_as_dict(xListBox)["SelectEntryText"], "List 1")

    ListBox ( by name ) 

    from uitest.uihelper.common import select_pos

categoryLB = xCellsDlg.getChild("categorylb")
        props = {"TEXT": "Time"}

actionProps = mkPropertyValues(props)
        categoryLB.executeAction("SELECT", actionProps)

    		 

    self.assertEqual(get_state_as_dict(categoryLB)["SelectEntryText"], "Time")

    Tabdialog (by name) 

    props = {"NAME": "Borders"}
        propsUNO = mkPropertyValues(props)
        xCellsDlg.executeAction("SELECT", propsUNO)

    ---

    Spinfield 

    xDecimalPlaces.executeAction("UP", tuple())
xDecimalPlaces.executeAction("DOWN", tuple())

    		 

    self.assertEqual(get_state_as_dict(xDecimalPlaces)["Text"], "Title text")

    Radiobutton 

    xSuperscript.executeAction("CLICK", tuple())

    		 

    self.assertEqual(get_state_as_dict(xSuperscript)["Checked"], "true")

    ColorListBox 

    xWidget.executeAction("OPENLIST", tuple())
xFloatWindow = self.xUITest.getFloatWindow()
xPaletteSelector = xFloatWindow.getChild("palette_listbox")
select_by_text(xPaletteSelector, "chart-palettes")  # or some other name
xColorSet = xFloatWindow.getChild("colorset")
xColorSet.executeAction("CHOOSE", mkPropertyValues({"POS": "2"}))  # or some other position

    ---

    MenuButton 

    xWidget.executeAction("OPENFROMLIST", mkPropertyValues({"POS": "0"}))

    ---


    Keyword Actions

    Sample: 

    from libreoffice.uno.propertyvalue import mkPropertyValues

ui_object.executeAction("TYPE", mkPropertyValues({"KEYCODE":"CTRL+A"}))

    KEYCODES:

    • ESC
    • TAB
    • DOWN
    • UP
    • LEFT
    • RIGHT
    • DELETE
    • INSERT
    • BACKSPACE
    • RETURN
    • HOME
    • END
    • PAGEUP
    • PAGEDOWN
    • CTRL
    • ALT
    • SHIFT

    How to extend the introspection library

    UI elements extending vcl::Window

    Other UI elements

    Tools for writing a test

    The difficult part about writing a test is to figure out which UNO commands to send and which action to call on a UI object. Fortunately, it is possible to log of UNO commands.

    1. Use these lines in .bashrc or .cshrc:
      export PYTHONPATH=/"Path of LibreOffice repo"/instdir/program/
export URE_BOOTSTRAP=file:///"Path of LibreOffice repo"/instdir/program/fundamentalrc
    2. Launch LibreOffice with 
      LO_COLLECT_UIINFO="test.log" SAL_USE_VCLPLUGIN=gen instdir/program/soffice
      Note: "LO_COLLECT_UIINFO" is a relative path under "instdir/uitest/".
    3. Simulate what you want to do with the mouse
    4. Close LibreOffice
    5. Open the resulting file in instdir/uitest/test.log
    6. Enter the UI logger directory with this command: 
      cd uitest/ui_logger_dsl/
    7. Make sure that your python has textX library installed
    8. Use the following command 
      python dsl_core.py <path_to_log_file> <path_to_a_new_python_file>
      <path_to_log_file> should be replaced with something like SourceDirectory/core/instdir/uitest/test.log and <path_to_a_new_python_file> can be a location of your choice where you would like to see the generated code.

    You can follow all these steps in this youtube video:

    Please accept this video. By accepting you will be accessing content from YouTube, a service provided by an external third party.

    YouTube privacy policy


    Running the test

    Before we push the test to master we should obviously check that the test is working. A simple way is to execute all UI tests in the uitest module through make uitest.uicheck. However we would like to see the UI while the test is being executed and the tests are run headless by default. This is achieved by running the following in the core directory: 

    (cd sc && make -srj1 UITest_sort UITEST_TEST_NAME="sorting.CalcSorting.test_Sortingbuttons_detect_columnheaders" SAL_USE_VCLPLUGIN=gen)

    It is also possible to use a shell script for starting a test.

    Note that you have to run this script from the source directory of LibreOffice core.

    You might also need a few environment variables to make it work, so the full script would be like: 

    export SRCDIR=$PWD
export PYTHONPATH=$SRCDIR/instdir/program:unotest/source/python
export URE_BOOTSTRAP=file://$SRCDIR/instdir/program/fundamentalrc
export TestUserDir=file:///tmp
export TDOC=$SRCDIR/sw/qa/uitest/data
export SAL_USE_VCLPLUGIN=gen
export LC_ALL=C

rm -rf /tmp/libreoffice/4

python3 "$SRCDIR"/uitest/test_main.py --soffice=path:"$SRCDIR"/instdir/program/soffice --userdir=file:///tmp/libreoffice/4 --file="$SRCDIR"/sc/qa/uitest/sort/sorting.py

    In case the test calls get_url_for_data_file to open a file, change variable TDOC accordingly to point to the directory where the file is. When you execute the test with the visible UI you will notice that the test is too fast to see what is going on. A quick and dirty solution is to add python’s (import time) time.sleep(10) which stops execution for a bit. The other solution that you can actually leave in the code (only leave a few useful ones in a test) in interesting places is uitest.debug.sleep. This method is ignored unless the test is started in debug mode by passing the --debug parameter to the test.

    When you need to run only one test in the test script file, run following command: 

    UITEST_TEST_NAME="FILENAME.CLASS_NAME.TEST_NAME" ./execute.sh

    for example 

    UITEST_TEST_NAME="sorting.CalcSorting.test_Sortingbuttons_detect_columnheaders" ./execute.sh


    Unsupported ui items

    • Base support
    • Slide sorter in Impress
    • Double-clicking on forms. See tdf#133641 for inspiration
    • File -wizards
    • Impress - Presentation minimizer
    • Logging ( LO_COLLECT_UIINFO) under Windows
    • Double-click
    • charts logging
    • bottom info toolbar in Calc
    • better support for shortcuts (tdf#124931)

    List of UI tests in master

    You can check the list of already implemented UItests.

    Finding bugs

    We have the keyword needUITest in Bugzilla to track all the bugs that need a UITest to automatically test it:

    More Information

    Retrieved from "https://wiki.documentfoundation.org/index.php?title=Development/UITests&oldid=768151"