QA/BSA/Implementation

TDF LibreOffice Document Liberation Project Community Blogs Weblate Nextcloud Redmine Ask LibreOffice Donate

This page was marked as inactive and is retained for historical reference.
BSA is retired

This page describes the Implementation of the Bug Submission Assistant (BSA).

Introduction

The following should contain all the information a developer need to know to work on the implementation of the Bug Submission Assistant. It covers all that cannot be easily deduced from reading the code or the associated tests.

The general idea is to create a web application that fits in a single HTML page. This page ( https://bugassistant.libreoffice.org/libreoffice/bug/bug.html ) is added to the regular bugzilla web site ( https://bugs.documentfoundation.org/ ). However, because it would make no sense to insert such a specialized page to bugs.documentfoundation.org, it is done using a reverse proxy assembling the two together. This server side setup allows the web application to query bugzilla without violating the same origin policy⁠ and does not require any server code.

JavaScript

All the sources are in JavaScript https://github.com/tdf/www-bugassistant
It uses JQuery
Each line of code must be tested
The core is in [1]
- The behavior of each step is encapsulated in a state_XXXX function that is called by the previous step and is responsible for invoking the state_YYYY function of the next step when it is finished.
- A state function is not given any argument and must not provide any argument to the next state function. For instance, the state_subcomponent will retrieve the name of the component from the DOM and not from its arguments.
- The general idea is to be able to develop within the context of a state function without knowing the behavior of the previous state functions nor trying to figure out an API between state functions.
- The error model is to throw after displaying a message in the .error element
- The $.bug.ajax function is a wrapper for $.ajax function that throw on error and display a message in the .error element
- The $.bug.lookup_result function analyzes a page returned by bugzilla, looking for an error pattern. If one is found, it display a message in the .error element and throws. Otherwise it returns the string found by the first capturing parenthesis of the success pattern.
- The code does not contain or generate any markup.
- The ajax calls must use the ajax().pipe() form for consistency. It is not a cosmetic point, it has impact on the code structure.
- The compatibility function is designed for code that needs to be run once, when the page is loaded, to help with cross browser compatibility
The dropdown selections ( version, component etc. ) are implemented in select.js
The tests are implemented in test.js.
- There is at least one test function per function found in the bug.js file.
- The test.html file partially copies the markup from bug.xhtml. It is intentional and no attempt must be made to run the tests using the exact same markup.
- The test.html file must include the same CSS as bug.xhtml because it will have consequences on the position and visibility of the elements and such consequences must be tested.

Cross-browser

If javascript is disabled, a link to bugzilla is shown
If the browser version is known to fail, a link to bugzilla is shown
For each supported browsers, the tests are run, each ?skin=XXX URL is checked, a fake bug is reported and then deleted from bugzilla. A permanent item is set in the TODO list as this work must be done on a regular basis.

CSS/HTML integration

Each state of the Bug Submission Assistant is shown as a static page built from the skin.js file. It is designed to allow integration without requiring multiple steps to reach a given state, nor the bugzilla backend.

Adding a new component

There are three primary steps to adding a new Component to Bugzilla and the BSA:

Create a new component in Bugzilla

 e.g. 'UX-Advice'

Create a set of wiki pages using the name of the new component (name is case-sensitive).
- The pages use some template/wiki-markup tricks, so copy the content from an existing component to serve as the basis of each page.

 e.g. Create
 Template:BugzAssHlp UX-Advice
 QA/Bugzilla/Components/UX-Advice
 QA/Bugzilla/Components/UX-Advice/Help
 QA/Bugzilla/Components/UX-Advice/Extended Help

 Add the component to the list on this page:
 QA/Bugzilla/Components

Update files in the BSA to include the new component:

 components.txt
 sanity-query.xhtml
 sanity-components.xhtml

(Note: the lists of components are alphabetical)

Installation and maintenance

Updating

Updating https://bugassistant.libreoffice.org/

- ssh wsadmin@kermit.documentfoundation.org
- cd /var/www/sites/bugassistant.libreoffice.org/libreoffice
- git pull
- make
- In https://libreoffice.org/admin change ?version=X in the iframe to invalidate the cache

Installation

- Install two Perl modules
  - XMLRPC::Lite (package libsoap-lite-perl on Ubuntu)
  - HTML::Templates (package libhtml-template-perl on Ubuntu)
- Install additional package support:
  - curl (package curl on Ubuntu) - http://curl.haxx.se/
  - xsltproc (package xsltproc on Ubuntu) - http://xmlsoft.org/XSLT/xsltproc.html
  - tidy (package tidy on Ubuntu) - http://tidy.sourceforge.net/
- git clone https://github.com/tdf/www-bugassistant.git
- cd www-bugassistant
- make
- browser bug/bug.html?skin

libreoffice.org integration

The official entry point (in the draft site before it is published) is implemented as a JS Pagecontaining an iframe within silverstripe

<p><iframe src="/bugassistant/bug/bug.html" width="850" height="800" frameborder="0"> </iframe></p>

Note that the <p> is required to prevent silverstripe from editing the inner HTML. A reverse proxy to https://bugs.documentfoundation.org/ is defined server side (see #System Administration) and the location to which it is bound is defined in a javascript file^{[dead link]} that is attached to the page using the Java Script tab in the admin panel for the Bug page in silverstripe.

Bugzilla Integration

The BSA requires a Bugzilla installation to be running XML-RPC.

We've been running into some problems since upgraded to Bugzilla 4.4.5
For notes on testing, see QA/Bugzilla/Deployment.

System Administration

The bugassistant.libreoffice.org host name is defined as a reverse proxy to https://bugs.documentfoundation.org on kermit

<VirtualHost *:443>
    ServerName          bugassistant.libreoffice.org
    DocumentRoot        /var/www/sites/bugassistant.libreoffice.org

    ProxyPass           /libreoffice !
    ProxyPass           / https://bugs.documentfoundation.org/ retry=1
    ProxyPreserveHost   On
    SSLProxyEngine      On

    SSLEngine on

    SSLCertificateFile /etc/ssl/certs/libreoffice.org.crt
    SSLCertificateKeyFile /etc/ssl/private/libreoffice.org.key
    SSLCertificateChainFile /etc/ssl/certs/libreoffice.org.chain

    CustomLog /var/log/apache2/bugassistant.libreoffice.org.log vhost_combined

</VirtualHost>

Three URLs involved :

The user entry point https://libreoffice.org/get-help/bug/ includes an iframe with https://www.libreoffice.org/bugassistant/libreoffice/bug/bug.html which calls https://bugs.documentfoundation.org/ either thru ajax or via form submissions encapsulated in an iframe. The URL https://www.libreoffice.org/get-help/bug/ is handled by silverstripe, https://bugassistant.libreoffice.org/bug/bug.html^{[dead link]} is defined by the virtual host above and https://www.libreoffice.org/bugassistant/bug/bug.html^{[dead link]} uses a reverse proxy (as well as https://libreoffice.org/bugzilla/) that must be added to the libreoffice.org virtual host definition as follows:

<VirtualHost *:443>
 ...
  Alias /bugassistant/ /var/www/sites/bugassistant.libreoffice.org/
  SSLProxyEngine      On
  # 
  # This location will be used by https://bugassistant.libreoffice.org/bug/bug.html
  # when included in an iframe in https://libreoffice.org/get-help/bug/ to obey the
  # same origin policy and be allowed to explore the DOM of an iframe containing
  # a bug submission form result.
  # 
  ProxyPass           /bugzilla/ https://bugs.documentfoundation.org/ retry=1
  ProxyPreserveHost   On
  #
  # Bugzilla will redirect 
  #   https://libreoffice.org/bugzilla/bug_list.cgi 
  # to
  #   https://libreoffice.org/bug_list.cgi
  # when the user is logged in. To preserve the /bugzilla/ directory
  # the Location: header must be reworked, which is the purpose of
  # the ProxyPassReverse that follow. 
  # Note that bugzilla does *not* return a Location that is 
  #   https://bugs.documentfoundation.org/bug_list.cgi
  # because it creates the Location: header using the hostname from 
  # which the request was originally made, i.e. libreoffice.org or
  # www.libreoffice.org. However, bugzilla has no way to know that the
  # original path of the request was prefixed with /bugzilla/ and it is
  # the responsibility of the reverse proxy to rewrite the location 
  # accordingly.
  #
  ProxyPassReverse   /bugzilla/ https://libreoffice.org/
  ProxyPassReverse   /bugzilla/ https://www.libreoffice.org/
 ...
<VirtualHost *:80>
 ...
  #
  # Bug reporting is forced to https because bugzilla always uses https
  # Any attempt to access it from http would violate the same origin policy.
  #
  RewriteRule ^/get-help/bug/$ https://libreoffice.org/get-help/bug/ [R=301,L,NC]
 ...

The following packages need to be installed on the server:

Internationalization

irc.freenode.net#sophi proposed to relay the need for translation, both on *.libreoffice.org and the wiki.
When the https://libreoffice.org/get-help/bug/ page content is modified, it must be copied over to the *.libreoffice.org sites which are separate silverstripe site instances. The URL in the iframe must be copied over and the bug-XX.html changed to the matching locale in the wiki. For instance,

<iframe src="https://bugassistant.libreoffice.org/libreoffice/bug/bug_en.html" width="850" height="800" frameborder="0"> </iframe>

must be replaced by

<iframe src="https://bugassistant.libreoffice.org/libreoffice/bug/bug_fr.html" width="850" height="800" frameborder="0"> </iframe>

in https://fr.libreoffice.org/get-help/bug/ .

The wiki pages QA/Bugzilla/Components/* contain all the English text used to create the https://bugassistant.libreoffice.org/libreoffice/bug/bug_en.html page. The QA/Bugzilla/Components/*/xx contains all the English text used to create the https://bugassistant.libreoffice.org/libreoffice/bug/bug-xx.html page. The software used to create bug-en.html from these Component pages relies on elements such as

<div class="component"> ... </div>

that can be found in the source of the page. These elements must be preserved and not translated so that the software can identify which texts must be used.

Data sources

https://bugs.documentfoundation.org/query.cgi?product=LibreOffice&query_format=advanced : list of versions.^{[dead link]}
QA/Bugzilla/Components/* extracted using a dedicated microformat: ^{[dead link]}
- XSL^{[dead link]} extraction of components^{[dead link]}
- XSL^{[dead link]} extraction of components comments^{[dead link]}
- XSL^{[dead link]} extraction of subcomponents^{[dead link]}

Consistency

Sanity checks are run each time the Bug Submission Assistant is generated:

https://bugassistant.libreoffice.org/libreoffice/bsa.pl -proc=checkComponents -componentsFile=build_en/components.xhtml

This will check if all components in the BSA exist in bugzilla