Hackfest/Hamburg2018/Make drawing layers ODF conform

TDF LibreOffice Document Liberation Project Community Blogs Weblate Nextcloud Redmine Ask LibreOffice Donate

Report about Hackfest topic "Make drawing layers ODF conform"

Armin Le Grand as mentor and me, Regina Henschel, as learning developer, have worked on this topic.

Drawing layers have a lot of problems. In regard to ODF these are

LibreOffice does not use <draw:layer-set> element in elements <draw:page> and <style:master-page> tdf#101218
LibreOffice does not use attributes draw:protected and draw:display of element <draw:layer> tdf#101242

Sketch out of an End Scenario

1. Each page (more precise the <draw:page> element) may have an own layer-set in file format. That will be possible in LibreOffice too.

2. Each master page (more precise the <style:master-page> element) may have an own layer-set in file format. That will be possible in LibreOffice too.

3. The layer-set of the master page is used together with the layer set, which is defined on a page. In the UI it is possible to offer them side by side, or hide one.
ToDo: Define details. The specification has nothing about this case.

4. According ODF specification the default layer-set is only used, in case there exists neither a layer-set of the page itself nor of its master-page. This is the default situation in LibreOffice and will be kept.

5. The <draw:layer> element in file format contains the information about visible/printable/protected of the layer. In running program a view will have a local (only for this view valid) set of visible/printable/protected of the layers. The information about visible/printed/protected is hold in the view, so that different views can have different settings of visible/printed/protected. On saving the information from the active view is written to file. That is a current feature and will be kept. But the information is no longer written to settings.xml but transformed to attributes of the saved layer.

6. Objects refer its layer by name in ODF file format. Current LibreOffice uses internally a numeric ID of the layer. That will be changed so that LibreOffice uses the layer name internally too.

7. Layers will be identified by an internal name instead of the current numeric ID. This name will be written as draw:name attribute of the layer to file. The name will be fix throughout lifetime of the layer and unique. The user can set a display name. It will be written to file in extended namespace. If the user does not define a name, the internal one will be used in the UI.

8. ToDo: Specify copy&paste behavior of objects in regard of layers. This affects copying to Gallery too, tdf#33170. Possible ways e.g.: If an entire page is copied, it is handled as a saved document, which means, that the page layer-set, the layer-set of the referred master page and the default layer-set is copied together with the page into the clipboard and can be pasted if the user wishes that. New layers might be added on pasting. If single objects are copied, they will keep their layer in case the same page is used.
ToDo: Ask UX for behavior for copy&paste to different page or document, or drag to Gallery.

9. The layer "Dimension lines" will be removed as default layer. It will be no longer automatically added to documents created by other applications and not generated for new own documents.
ToDo: Ask UX about this.

10. The layer "Dimension lines" has currently the special feature, that "Dimension line"-shapes are automatically added to this layer, even if this layer is not the active one. This feature will be removed.
ToDo: Ask UX about this.

11. The layer "Controls" will be removed as default layer. Is will be no longer automatically added to documents created by other applications and not generated for new own documents.
ToDo: Ask UX about this.

12. The current layer "Controls" has the special feature, that objects, which reside in this layer are automatically shown in front of objects from other layers and currently the user cannot change this. This feature will be removed.
ToDo: Ask UX about this.

13. Form-controls in life-mode (non-design mode) jump to front before all other objects because they are real windows. That cannot be changed. To mimic this life-mode behavior the current implementation adds the form-controls automatically to the "Controls"-layer and uses the above described special feature of the "Controls"-layer. The form-controls in design mode will be handled same as other objects in regard to z-order and layer. A newly created form-control will be in front of other existing objects, same as any other newly created object.
ToDo: Ask UX about this.

It might be possible to bind the special behavior for form controls not to the layer name, but to the kind of object itself. Therefore it would be necessary to distinguish a form control from other shapes in design mode ON state. Then the form control must always be forced to front. That needs to be done not only, if the form control is selected, but also if another shape is going to cover the form control. I don't know, whether this is possible and how expensive it is.
ToDo: Ask Armin about this.

14. The tab-list of layers will be only shown, if more than one layer exists. That will be the normal UI for new documents in case UX agree to not automatically generate the layers "Dimension lines" and "Controls".
Note: There is an ongoing discussion about showing the tab bar in issue tdf#90320.

It seems, that the layer "Background" is not used at all. At least it is not available in the UI. Action item for Armin: Check this.

The layer "Background objects" has currently some special features. Action item for Armin: Is it possible to remove such special rules?

Note: If UX agrees to remove the special handling of "Dimension lines" and "Controls", a lot of special handling parts can be removed from the code.

Note: If UX agrees about removing the layer "Dimension lines" and "Controls" as default layer, there would be only one default layer, which can then be hidden.

Transition to End Scenatio

Remover internal use of the bitfield

Change the representation from visible/printable/protected from bitfeld to something (internal class will follow) that holds page name, layer-set of page, active layer remark, visible/printable/protected for each layer. [As of Jule 2018, I have brought the visible/printable/protected information to SdrLayer and mimic the "SetAll" for SdrLayerIDSet by tracking a status. The ODF attributes "draw:display" and "draw:locked" are written and read in addition to the bitfield. It is not clear, whether information about page should be included at all.]

The old bitfield is created from the new structure, when writing the document (likely in xmloff). The old bitfeld is interpreted and converted to the new format on loading (also likely in xmloff). The SdrLayerID from the SdrLayer is removed.

Encountered problems:
(1) SdrLayer, SdrLayerID and SdrLayerAdmin are not only used by Draw/Impress but used by Writer and Calc too. In file format neither text documents nor spreadsheet documents have layer-sets.
(2) Objects call a "SetAll()" method in blank, without specifying the actual existing layers.
(3) It is possible to have different views on the same opened document. These views can have different settings for visible/printable/protected.
(4) Currently visible/printable/locked is saved as config-item in file settings.xml. Currently this information goes directly to the initial view and has no counterpart in the model. And on saving the information from the current view is used for saving it into the config-items.
(5) LibreOffice uses always the layers "Layout", "Background", "Background objects", "Controls", "Dimension lines" in exactly this order. If a document has a different order (which is possible using a macro), it is "repaired" on opening. The bitfields expect existence and order of these layers to get the intended relationship between bit and layer.

Preliminary Result at the Time of September 2018

With commit "tdf#101242 Support draw:display and draw:protect attributes of ODF" LibreOffice uses the attributes draw:display and draw:protected of layers from the default <draw:layer-set> element to initialize its view, in case there exists no visible/printable/locked config-items. On saving the ODF attributes are written in addition to the config-items. For this purpose the class SdrLayer has got the new members mbVisibleODF, mbPrintableODF and mbLockedODF.

While working on it, it becomes obvious, that other modules than Draw/Impress use the bitfield SdrLayerIDSet to manage the state of own SdrLayer objects. The internal use of SdrLayer in these modules does not correspond to a "layer" in ODF. These modules do not have layers in file format and they do not use config-items.

And I encountered the very old bug tdf#119397: The internal bitfield SdrLayerIDSet uses an order by layer ID, but on loading the bitfields of the config-items are interpreted in the order the layers are listed in the <draw:layer-set> element. If the user inserts a layer not at the end, but in between, these orders lose synchronization.

Read the new format

If the attributes draw:display or draw:protected exist in the <draw-layer> element, then read it.

Note: Make sure, that existing draw:display and draw:protected exclude affected settings.xml items.

Extend API to “Layer2” or better name, so that property “VisibleLayers” or similar (e.g. sUno_View_VisibleLayers) can still be read and written and new properties too. Hint: This effects aViewProps.

Testdocuments for all cases for import.

Check in

Note: Now Libreoffice is able to read documents, which are produced by foreign application which do not write a settings.xml, but use only ODF means.

Write the new format

Export is extended to write ODF conform.
ToDo: Investigate and decide, whether only for default layer-set or parallel for master-page and page layer-sets.

Parallel keep old settings.xml for older LO versions.

Deprecate relevant items of settings.xml.

Remove items from settings.xml

To be done when relevant

Note: Further steps cannot be detailed, because there are a lot of UX decisions pending. Step 1 can be done immediately, because it only affects code internals.