Development/UITests

From The Document Foundation Wiki
Jump to navigation Jump to search

Template:TopMenu/en Template:Menu/en Template:Menu.Development/en

Other languages:

This is the preliminary documentation for the UI testing framework with a special focus on the Python side.

Architecture

The UI testing consists of two parts, a C++ piece that wraps the LibreOffice UI in an introspection library and a Python framework with the actual test code. The connection between the two parts is done through a simple mostly untyped UNO interface.

C++ introspection library

The C++ introspection library consists of an UNO API that is used for communication with the python part of the framework and wrappers for many UI objects. The UNO interface has only a few methods and most of the communication happens through string parameters.

The wrapper for the UI objects provides about the same methods as the UNO interface, plus some helper methods.

Python framework

Writing tests

Xisco Faulí's presentation from FOSDEM 2021, LibreOffice QA - how to write your first test, walks through the process of creating tests.

Makefile part

A simple makefile for the UI testing looks like the demo makefile. The important part is to use the UITest gbuild class and let the gb_UITest_add_module point to the directories with test files.

Normally nothing more is required and most is handled in the gbuild UITest class.

$(eval $(call gb_UITest_UITest,range_name))

$(eval $(call gb_UITest_add_modules,range_name,$(SRCDIR)/sc/qa/uitest,\
    test_dir1/ \
    test_dir2/ \
))

The range_name parameter is the name of the test target. The $(SRCDIR/sc/qa/uitest is the base directory and the parameters in that part lists the directories to search for tests.

Optionally the test can be run with a specific registrymodifications.xcu file to support running with specific configuration settings. Each test has an own user profile in workdir/UITest/test_name/user/. With the following makefile snippet a file can be copied into that directory.

$(eval $(call gb_UITest_use_configuration,range_name,$(SRCDIR)/sc/qa/uitest/range_name/registrymodifications.xcu))

This would copy the file sc/qa/uitest/range_name/registrymodifications.xcu to the user profile and the test would run with these settings.

Writing the Python code

Infrastructure

Commands available on UI elements

Generic

Handling the State of UI objects

The method to get the state of the UI object is provided by all UI elements. The method returns the state as a sequence of PropertyValue.

ui_element.getState()

The UI testing framework provides a method to convert the returned sequence of PropertyValue to a python dictionary.

from uitest.uihelper.common import get_state_as_dict

get_state_as_dict(ui_element)

Sending keyboard input to an element

from uitest.uihelper.common import type_text

xWriterDoc = self.xUITest.getTopFocusWindow()
xWriterEdit = xWriterDoc.getChild("writer_edit")
type_text(xWriterEdit, "Test")

Selecting text

from uitest.uihelper.common import select_text

xEdit = xAddNameDlg.getChild("edit")
select_text(xEdit, from_pos="2", to="12")

Setting Zoom

xWriterEdit = xWriterDoc.getChild("writer_edit")
xWriterEdit.executeAction("SET", mkPropertyValues({"ZOOM": "200"}))

GOTO page

xWriterEdit = xWriterDoc.getChild("writer_edit")
xWriterEdit.executeAction("GOTO", mkPropertyValues({"PAGE": "1"}))

Control Actions

Control Code Assert
Button
xControl.executeAction("CLICK", tuple())

---

TextBox
from uitest.uihelper.common import type_pos

type_text(xControl, "Text")
self.assertEqual(get_state_as_dict(xTitleText)["Text"], "Text")
CheckBox
xControl.executeAction("CLICK", tuple())
self.assertEqual(get_state_as_dict(xControl)["Selected"], "true")
Charmap Widget
from libreoffice.uno.propertyvalue import mkPropertyValues

xControl.executeAction("SELECT", mkPropertyValues({"COLUMN": "2", "ROW": "2"}))

---

TreeList
# Unfold/Fold the tree
xTreelist = xExDBDlg.getChild("availablelb")         
xTreeEntry = xTreelist.getChild('0')
xTreeEntry.executeAction("EXPAND", tuple())  #Available Databases
xTreeEntry.executeAction("COLLAPSE", tuple()) #Click on the Bibliography

xTreeEntry2 = xTreeEntry.getChild('0')
xTreeEntry2.executeAction("SELECT", tuple())

---

ComboBox
from uitest.uihelper.common import select_pos

select_pos(xCombobox, "1")
self.assertEqual(get_state_as_dict(xNumbering)["SelectEntryText"], "List 1")
Tabs
from uitest.uihelper.common import select_pos

xTabs = xDialog.getChild("tabcontrol")
select_pos(xTabs, "0")
self.assertEqual(get_state_as_dict(xTabs)["CurrPagePos"], "0")
ListBox ( by position)
from uitest.uihelper.common import select_pos

select_pos(xListBox, "1")
self.assertEqual(get_state_as_dict(xListBox)["SelectEntryText"], "List 1")
ListBox ( by name )
from uitest.uihelper.common import select_pos

categoryLB = xCellsDlg.getChild("categorylb")
        props = {"TEXT": "Time"}

actionProps = mkPropertyValues(props)
        categoryLB.executeAction("SELECT", actionProps)
self.assertEqual(get_state_as_dict(categoryLB)["SelectEntryText"], "Time")
Tabcontrol
from uitest.uihelper.common import select_pmacOSTabs = xFunctionDlg.getChild("tabs")
        select_pos(xTabs, "1")

---

Tabdialog (by position)
from uitest.uihelper.common import select_pmacOSTabs = xFunctionDlg.getChild("tabs")
        select_pos(xTabs, "1")

---

Tabdialog (by name)
        props = {"NAME": "Borders"}
        propsUNO = mkPropertyValues(props)
        xCellsDlg.executeAction("SELECT", propsUNO)

---

Spinfield
xDecimalPlaces.executeAction("UP", tuple())
xDecimalPlaces.executeAction("DOWN", tuple())
self.assertEqual(get_state_as_dict(xDecimalPlaces)["Text"], "Title text")
Radiobutton
xSuperscript.executeAction("CLICK", tuple())
self.assertEqual(get_state_as_dict(xSuperscript)["Checked"], "true")

Keyword Actions

Sample:

from libreoffice.uno.propertyvalue import mkPropertyValues

ui_object.executeAction("TYPE", mkPropertyValues({"KEYCODE":"CTRL+A"}))

KEYCODES:

  • ESC
  • TAB
  • DOWN
  • UP
  • LEFT
  • RIGHT
  • DELETE
  • INSERT
  • BACKSPACE
  • RETURN
  • HOME
  • END
  • PAGEUP
  • PAGEDOWN
  • CTRL
  • ALT
  • SHIFT

How to extend the introspection library

UI elements extending vcl::Window

Other UI elements

Tools for writing a test

The difficult part about writing a test is to figure out which UNO commands to send and which action to call on an UI object. Fortunately, it is possible to log of UNO commands.

  1. Use these lines in .bashrc or .cshrc:
    export PYTHONPATH=/"Path of LibreOffice repo"/instdir/program/
    export URE_BOOTSTRAP=file:///"Path of LibreOffice repo"/instdir/program/fundamentalrc
    
  2. Launch LibreOffice with
    LO_COLLECT_UIINFO="test.log" SAL_USE_VCLPLUGIN=gen instdir/program/soffice
    
  3. Simulate what you want to do with the mouse
  4. Close LibreOffice
  5. Open the resulting file in instdir/uitest/test.log
  6. Enter the UI logger directory with this Command:
    cd uitest/ui_logger_dsl/
    
  7. Make sure that your python has textX library installed
  8. Use the following Command
    python dsl_core.py <path_to_log_file> <path_to_a_new_python_file>
    
    <path_to_log_file> should be replaced with something like SourceDirectory/core/instdir/uitest/test.log and <path_to_a_new_python_file> can be a location of your choice where you would like to see the generated code.

You can follow all these steps in this youtube video:

Running the test

Before we push the test to master we should obviously check that the test is working. A simple way is to execute all UI tests in the uitest module through make uitest.uicheck. However we would like to see the UI while the test is being executed and the tests are run headless by default. This is achieved by running the following in the core directory:

(cd sc && make -srj1 UITest_sort UITEST_TEST_NAME="sorting.CalcSorting.test_Sortingbuttons_detect_columnheaders" SAL_USE_VCLPLUGIN=gen)

It is also possible to use a shell script for starting a test.

You might also need a few environment variables to make it work, so the full script would be like:

export PYTHONPATH=/home/user/libreoffice/instdir/program/
export PYTHONPATH=$PYTHONPATH:/home/user/libreoffice/unotest/source/python/
export URE_BOOTSTRAP=file:///home/user/libreoffice/instdir/program/fundamentalrc
export TestUserDir=file:///tmp
export TDOC=/home/user/libreoffice/sw/qa/uitest/data
export SAL_USE_VCLPLUGIN=gen

rm -rf /tmp/libreoffice/4

SRCDIR="${SRCDIR:-$(pwd)}"
python3 "$SRCDIR"/uitest/test_main.py --debug --soffice=path:"$SRCDIR"/instdir/program/soffice --userdir=file:///tmp/libreoffice/4 --file="$SRCDIR"/sc/qa/uitest/calc_tests/sorting.py

In case the test calls 'get_url_for_data_file' to open a file, change variable TDOC accordingly to point to the directory where the file is. When you execute the test with the visible UI you will notice that the test is too fast to see what is going on. A quick and dirty solution is to add python’s time.sleep which stops execution for a bit. The other solution that you can actually leave in the code (only leave a few useful ones in a test) in interesting places is uitest.debug.sleep. This method is ignored unless the test is started in debug mode by passing the --debug parameter to the test.

When you need to run only one test in the test script file, run following command:

UITEST_TEST_NAME="FILENAME.CLASS_NAME.TEST_NAME" ./execute.sh

for example

UITEST_TEST_NAME="sorting.CalcSorting.test_Sortingbuttons_detect_columnheaders" ./execute.sh


Unsupported ui items

  • Base support
  • Slide sorter in Impress
  • Double-clicking on forms. See https://bugs.documentfoundation.org/show_bug.cgi?id=133641 for inspiration
  • File -wizards
  • Impress - Presentation minimizer
  • Logging ( LO_COLLECT_UIINFO) under Windows
  • Double-click
  • charts logging
  • bottom info toolbar in Calc
  • better support for shortcuts (tdf124931)

List of UI tests in master

You can check the list of already implemented UItests.

Finding bugs

We have the keyword needUITest in Bugzilla to track all the bugs that need a UITest to automatically test it:

More Information