Development/UITests

This is the preliminary documentation for the UI testing framework with a special focus on the Python side.

Architecture
The UI testing consists of two parts, a C++ piece that wraps the LibreOffice UI in an introspection library and a Python framework with the actual test code. The connection between the two parts is done through a simple mostly untyped UNO interface.

C++ introspection library
The C++ introspection library consists of an UNO API that is used for communication with the python part of the framework and wrappers for many UI objects. The UNO interface has only a few methods and most of the communication happens through string parameters.

The wrapper for the UI objects provides about the same methods as the UNO interface, plus some helper methods.

Writing tests
Xisco Faulí's presentation from FOSDEM 2021, LibreOffice QA - how to write your first test, walks through the process of creating tests.

Xisco's presentation from FOSDEM 2022 shows Things you can test in a UITest.

Makefile part
A simple makefile for the UI testing looks like the demo makefile. The important part is to use the UITest gbuild class and let the gb_UITest_add_module point to the directories with test files.

Normally nothing more is required and most is handled in the gbuild UITest class.

The range_name parameter is the name of the test target. The $(SRCDIR/sc/qa/uitest is the base directory and the parameters in that part lists the directories to search for tests.

Optionally the test can be run with a specific registrymodifications.xcu file to support running with specific configuration settings. Each test has an own user profile in workdir/UITest/test_name/user/. With the following makefile snippet a file can be copied into that directory.

This would copy the file sc/qa/uitest/range_name/registrymodifications.xcu to the user profile and the test would run with these settings.

Handling the State of UI objects
The method to get the state of the UI object is provided by all UI elements. The method returns the state as a sequence of PropertyValue.

The UI testing framework provides a method to convert the returned sequence of PropertyValue to a python dictionary.

Keyword Actions
Sample:

KEYCODES:



Tools for writing a test
The difficult part about writing a test is to figure out which UNO commands to send and which action to call on a UI object. Fortunately, it is possible to log of UNO commands.


 * 1) Use these lines in .bashrc or .cshrc:


 * 1) Launch LibreOffice with


 * 1) Simulate what you want to do with the mouse
 * 2) Close LibreOffice
 * 3) Open the resulting file in instdir/uitest/test.log
 * 4) Enter the UI logger directory with this Command:

 should be replaced with something like SourceDirectory/core/instdir/uitest/test.log and  can be a location of your choice where you would like to see the generated code.
 * 1) Make sure that your python has textX library installed
 * 2) Use the following Command

You can follow all these steps in this youtube video:

Running the test
Before we push the test to master we should obviously check that the test is working. A simple way is to execute all UI tests in the uitest module through make uitest.uicheck. However we would like to see the UI while the test is being executed and the tests are run headless by default. This is achieved by running the following in the core directory:

It is also possible to use a shell script for starting a test.

You might also need a few environment variables to make it work, so the full script would be like:

Note that you have to run this script from the source directory of LibreOffice core.

In case the test calls 'get_url_for_data_file' to open a file, change variable TDOC accordingly to point to the directory where the file is. When you execute the test with the visible UI you will notice that the test is too fast to see what is going on. A quick and dirty solution is to add python’s time.sleep which stops execution for a bit. The other solution that you can actually leave in the code (only leave a few useful ones in a test) in interesting places is uitest.debug.sleep. This method is ignored unless the test is started in debug mode by passing the --debug parameter to the test.

When you need to run only one test in the test script file, run following command:

for example

Unsupported ui items

 * Base support
 * Slide sorter in Impress
 * Double-clicking on forms. See tdf#133641 for inspiration
 * File -wizards
 * Impress - Presentation minimizer
 * Logging ( LO_COLLECT_UIINFO) under Windows
 * Double-click
 * charts logging
 * bottom info toolbar in Calc
 * better support for shortcuts (tdf#124931)

List of UI tests in master
You can check the list of already implemented UItests.

Finding bugs
We have the keyword needUITest in Bugzilla to track all the bugs that need a UITest to automatically test it:

More Information

 * Markus Mohrhard's BlogPlost: UI testing in LibreOffice


 * Markus Mohrhard's BlogPlost: Writing a LibreOffice Calc UI test


 * Markus Mohrhard's BlogPlost: LibreOffice UI test tutorial (part 1): Adding a simple test.


 * Markus Mohrhard's BlogPlost: LibreOffice UI test tutorial (part 2): Improve the introspection library


 * GSOC 2018 Saurav Chirania's blog


 * GSOC 2019 Ahmed El-Shreif's blog


 * How to create test case using your logs (Youtube Video)