Tenders for budget 2018
- 1 Proposal for tenders in budget 2018
- 1.1 QA and Automated Testing
- 1.2 Core Code Refactorings and Cleanups
- 1.2.1 convert Impress slideshow to drawinglayer primitives
- 1.2.2 split signing from the build process
- 1.2.3 Re-thinking how we install language-packs
- 1.2.4 Unify Writer and Draw Images
- 1.2.5 Implement font subsetter for font embedding
- 1.2.6 Text layout performance
- 1.2.7 Migrating to single modern 2D graphics library
- 1.2.8 Curl based HTTP/WebDAV UCP
- 1.3 Development Infrastructure
- 1.4 Blocking features
- 1.4.1 SmartArt implementation
- 1.4.2 Improved built-in, XRay-like profiler & script debug
- 1.4.3 finish the XServiceDocumenter to allow UNO user to learn about core implementations
- 1.4.4 Improve the experience of the SDK
- 1.4.5 UNO API changes subscription/feedback channel
- 1.4.6 UNO API discussion/sniplet dump
- 1.5 UX problem areas
- 2 Already Resourced / Abandoned
- 3 Proposals missing detailed descriptions for tenders in budget 2018
- 4 Proposals for tenders in budget 2018 that are likely better suited as a GSOC
Proposal for tenders in budget 2018
Below are proposal for tenders, to be discussed on the ESC.
Costs are stated in rough guestimate person weeks, so that it is possible to rank the proposals against each other:
QA and Automated Testing
Regular scan-build reports like we do with CppCheck
CostEstimate: 1 week Contact: Luke
Regular scan-build reports like we do with CppCheck. See also: https://wiki.documentfoundation.org/Development/Clang_Code_Analysis
CostEstimate: 2 weeks Contact: Luke
- Uncheck low priority/quality warnings
- Show new detected problems since last run
MSO compatibility test framework using Win API
CostEstimate: 6 weeks + VMs Contact: Tamás Zolnai
- Microsoft published some API to work with Office applications.
- It can be a good idea to create a test framework which tests MSO file format compatibility by opening files both with LibreOffice and Microsoft Office and checking document properties (e.g. number of pages, shapes, tables) using LO API and Windows API.
- Of course this test framework would be Windows only.
- First test can be for example a check for the number of tables in the imported document compared to the table number produced by Microsoft Office import.
- After this simple test is implemented, we can run this on thousands of documents (this also means we can guarantee a specific level of compatibility by adding these tests and running them on lots of documents (also good for regression testing)).
- Later the compatibility level can be increased.
- Another good starting point can be some validation test of LO exported documents.
- Just open saved documents in MS office automatically and check whether MSO can open them without a problem (e.g. tdf#104787)
- Validation tools (like Open XML SDK) often shows false positives and also can happen they don't show a problem which actually makes MSO to fail opening a document. So better to test compatibility directly using MSO.
A reasonably future-proof automated testing plan
MSO compatibility test document set
CostEstimate: 3 weeks Contact: Cor Nouws
- There is an abundance of issues with all kind of interoperability problems.
- Interoperability is by far the single most reported problem in the real world.
- The reported issues will have many shared root causes, but often are complex, a mixture and nasty to analyse.
- Preparing well thought clean test documents will make working on improvements much easier in two ways:
- Much more appealing to take a look at
- Saving much developer time; quoting a dev: " It happens quite frequently that creating a minimal reproducer for a bugreport is almost as much time as fixing the actual bug."
- Focus (docx, pptx ... xlsx ... ppt, doc) to be discussed
A manual document building testing task.
Cleanup & further improve ODF conformance
CostEstimate: 4 weeks Contact: Thorsten Behrens
- address annoying issues highlighted by http://autotests.opendocumentformat.org/
- cleanup https://bugs.documentfoundation.org/buglist.cgi?list_id=582656&query_format=advanced&resolution=---&status_whiteboard=odf_validation&status_whiteboard_type=allwordssubstr
- perhaps also cleanup https://bugs.documentfoundation.org/buglist.cgi?list_id=582655&query_format=advanced&resolution=---&status_whiteboard=odf&status_whiteboard_type=allwordssubstr
- generally, get LibreOffice ready for ODF 1.3
Core Code Refactorings and Cleanups
convert Impress slideshow to drawinglayer primitives
CostEstimate: 4 weeks Contact: Thorsten
This is a copy from Development/GSoC/Ideas#Rework_Impress_slideshow_to_use_DrawingLayer_primitives, which has not happend in 4 years.
The Impress slideshow, while being designed to only interact with Impress via interfaces, had to resort to an ugly hack to be able to render all Impress content. That was ok back in the day, but is becoming a liability these days. Nowadays, what one want to use is the DrawingLayer Primitives (https://wiki.openoffice.org/wiki/DrawingPrimitives), which means porting over slideshow/source/engine/shapes/* to use this kind of abstraction, instead of the StarView Metafile previously in use.
split signing from the build process
CostEstimate: 6 weeks Contact: Norbert
- This should allow signing after the binaries are built rather than during the build process, ideally on a separate signing server.
Re-thinking how we install language-packs
CostEstimate: 8 weeks Contact: Markus
- if we have an auto-updater with signed MAR files.
- could provide translated installer, and download rest during install.
- signing is done on the whole archive with this approach.
Unify Writer and Draw Images
CostEstimate: 8 weeks Contact: Thorsten
- long-standing annoyance
- several attempts, current status (90-degree rotations) is a half-house
- extract Writer border rendering & frame special code, move to drawing layer & make accessible from SdrGrafObj
Implement font subsetter for font embedding
CostEstimate: 4 weeks Contact: Thorsten
While font embedding is implemented for LibreOffice since many years, it's an under-used and under-developed feature due to the fact that unicode-capable fonts are massive, thus blowing up documents beyond useful sizes. Other applications therefore simply subset the fonts embedded, to either exactly the set of glyphs used, or to the code pages / script areas touched by existing document content. This would also help a lot for making our in-tree and out-of-tree bug document collection invariant to installed system fonts.
Text layout performance
CostEstimate: 6 weeks Contact: Khaled
Our text layout performance is abysmal, all over the code base it is assumed that shaping text is cheap and we can do it over and over again. Want to measure the text? Shape it and discard the output afterwards. Want to measure part of the same text? Shape again. Want to find line breaks? Shape again. Want to finally draw it? Shape again. This might have been cheap for Latin script in the olden days when all we did is query font cmap table and put glyphs next to each other, which is not the case anymore and never been the case for more involved scripts. We need to work on this, and there are many possibilities; retaining shaping results much longer, improving the wasteful OutputDevice API, caching etc.
Migrating to single modern 2D graphics library
CostEstimate: 20 weeks Contact: Michael
Migrating to single modern 2D graphics library, be it Cairo or Skia, and sorting out our graphics and text drawing inconsistencies.
Curl based HTTP/WebDAV UCP
CostEstimate: 8 weeks Contact: Michael S
- Problem #1: TDF releases bundle both OpenSSL and NSS, both libraries have high number of security issues
- On macOS and Windows, neither OpenSSL nor NSS integrate with the OS supplied crytographic APIs, so they will use a bundled hard-coded set of trusted CAs that is different from what the OS itself would trust and not user-modifiable
- OpenSSL cannot ever be used from the system because it has no ABI
- Problem #2: There are 2 different HTTP/WebDAV UCPs:
- one (which is used by everybody including TDF releases) using a bundled Neon WebDAV library
- requires OpenSSL
- cannot be used on a hypothetical Apple iOS port
- another (for a hypothetical Apple iOS port) using a bundled Serf library
- requires OpenSSL
- Serf does not actually support WebDAV directly, only HTTP, so the UCP itself impleements the additional WebDAV protocol features
- complicated to build, drags in 2 other bundled external libraries
- cannot be upgraded to current version without introducing a new build dependency on "scons" tool
- one (which is used by everybody including TDF releases) using a bundled Neon WebDAV library
- TDF releases also bundle the Curl library
- can do HTTP, likely similar to Serf
- can use native OS cryptographic APIs and trusted CAs on Windows, macOS and Linux
- can be used on Apple iOS without problems
=> implementing a HTTP/WebDAV UCP with Curl, possibly based on code from the Serf UCP, would solve these issues, get rid of 4 bundled external libraries and 1 hard OpenSSL dependency
Native IDE build / simpler building
CostEstimate: 50 weeks Contact: JanI
One of the biggest show-stoppers for keeping new contributors (we have a nice inflow, so no problem there) is our development environment.
Universities today teach students to use IDE, and solely work with that. When I reach out to people who have stopped contributing, the lack of a modern IDE is a major response.
The new “gbuildtojson” that Bjoern has made, is a step in the right direction, but we are still far from a goal, where a contributor clone the repo and open his preferred IDE to start working.
Investing in: - removing cygwin in Windows - making a auto installer (lode++) - complete gbuildtojson and xx-ide-integration to include prepared solutions for popular IDE (msc, Xcode, eclipse)
would help a lot to keep the contributors contributing, and thus help grow the community.
This project has a very wide scope.
More CI hardware to get quicker build-times
CostEstimate: Eur 20k Contact: Noel
- consider cloud hardware cost; scale on-demand ? (Bjoern)
- Norbert considers this to be mostly a problem of stopping builds jamming existing hardware
Preconfigured Cloud VMs for Newcomers
CostEstimate: Eur 10k Contact: Bjoern
Have ready-to-go VM with a preinstalled Dev Env including IDE integration with debugging and code completion for the most popular platform (Windows=MSVS, Linux=kdevelop, macOS=XCode) and in the best case a prebuild LibreOffice. Those imagine should be able to be spun up by the dev-mentor at a minutes notice for newcomers to submit their first 3-5 patches. This is asking for budget for the hosting (e.g. cloudonmac or cloudshare) only, I have hope that both volunteers and the dev-mentor will be able to script the setup of these base images.
(This item likely can be folded into the infra budget in the end, which is fine: Then we wouldnt need to vote on this separately.)
Collaboration Dictionary Project
CostEstimate: 8 weeks Contact: DennisRoczek
Collaboration Dictionary Project was discussed at I10N: a platform for creating and updating dictionaries and automatically pack extensions from word lists (and upload them to the web pages).
CI for Online
CostEstimate: ? Contact: Samuel M.
There should be a Jenkins job that runs "make check" for each gerrit patch for LO Online, as we do for LO core. https://redmine.documentfoundation.org/issues/2541
Actually, the same is true for Android...
CostEstimate: 16 weeks Contact: Thorsten
Our existing smart-art import uses the fallback-stream in OOX files, and gives us only the draw shapes that are imported - thus loosing the original layout. In older file versions we don't have the cached shapes - so can render nothing. We really would like schema driven diagram layout as a core feature as well. This should be interoperable with OOX, and have suitable extensions for ODF. It should layout interoperably, and allow editing of the underlying data, and selection of a schema.
- Note** this has been significantly improved already during GSoC 2017, possibly obsolete, at the very least much reduced effort
Improved built-in, XRay-like profiler & script debug
CostEstimate: 12 weeks Contact: Michael
XRay-alike, to be built-in as native code, with better core hooks and usability etc. Modern browsers come with significant, helpful built-in JS developer tools, and we are far behind this in terms of ease-of-use for development of extensions and (more critically) unit tests using the UNO APIs.
It should be possible to press F12 (or whatever) - and get access to the rich object model, browse it, and see how both scripts, and the app is performing - to help both scripters, and also those interested in improving LibreOffice performance & responsiveness.
Porting the XRay functionality from StarBasic to C++, and including it - along with providing a cleaner, browser-like UI at the bottom of the screen would be ideal.
In particular - the ability to select an item in an arbitrary document eg. an image, or a shape - and get its UNO peer, and be able to inspect attributes on it would be excessively helpful.
Similarly logging UNO events emitted by the core, and allowing inspection of their details in a console-alike.
finish the XServiceDocumenter to allow UNO user to learn about core implementations
CostEstimate: 2 weeks Contact: Bjoern
Finish the com.sun.star.script.XServiceDocumenter/theServiceDocumenter to allow UNO and StarBasic users to use a call to xDocumenter.showCoreDocs() to get directly to docs.libreoffice.org showing the C++ classes implementing a specific UNO service. This should smooth the migration of interested people to move from UNO/extension code to core code and scratch their itch.
Note that the showServiceDocs()/showInterfaceDocs() methods of the service documenter (that bring you to the abstract UNO docs on api.libreoffice.org) are already implemented, so not in scope for this task.
(the XServiceDocumenter.showServiceDocs()/showInterfaceDocs() could also be a base for "Improved scripting debugging". Possibly merge these?)
see: https://www.youtube.com/watch?v=WBNG6bVZPzw for details
Improve the experience of the SDK
CostEstimate: 4 weeks Contact: Bjoern
- see https://studiofreya.com/2016/10/11/integrating-libreoffice-into-cpp/ https://studiofreya.com/2016/10/19/errors-connecting-to-libreoffice-with-cpp-errors-part-2/ https://studiofreya.com/2016/10/21/open-libreoffice-calc-with-cpp-part-3/ as reference of the pain this currently is
- undo hugely painful gnumake-ness etc.
- make it much more usable, and ideally from IDEs
- provide a 10 minutes "Hello World extension for LibreOffice from scratch" video as proof how simple stuff is after this is fixed.
UNO API changes subscription/feedback channel
CostEstimate: 1 week Contact: Bjoern
The simplest solution (KISS for a start) would be e.g.:
- an archived mailing list (uno-changes@ ...)
- an bot that puts every API change on that list, cc'ing the author of the change,
- someone replying to the API change also reaches the author of the change
- bot that filters for the API change discussions and put notifications and RSS feeds on api.libreoffice.org
UNO API discussion/sniplet dump
CostEstimate: 4 weeks Contact: Bjoern
PHP is a horrible language, still it succeeded in capturing a huge amount of the market. I assume, the "user contributed notes" on their API documentation played a major part in that (see e.g.: http://php.net/manual/en/function.strlen.php). This proposal is to add a similar user-contributed code examples directly to a specific part of the UNO API documentation at api.libreoffice.org. The discussion software should have a low barrier to get into (OpenID?) and support e.g. code highlighting. Askbot might be a candidate.
This is pretty much a superset of the "UNO API changes subscription/feedback channel" propoal: those bots and RSS feeds could use the discussion forum.
UX problem areas
CostEstimate: 12 weeks Contact: Kendy
Re-run the User Metrics tender as before.
Suggestion from Samuel: Limit the client side implementation as the scope is quite big already. New client metrics can be added by the community once the infrastructure is there. Modify the list like this e.g.:
3. a client part, that not only counts how often features have been used, but also provides further metrics; some samples of items that need tracking are
- the location of the click action in the menu, as sometimes duplicates exist
- which context menu was used
whether a certain feature was invoked by a single click, by click and hold, by a drop down click or by a multi click from which application clicking on the close document/window ‘X’ (.CloseDoc) and close application ‘X’ (.Quit) occurs whether the user used the enter key, mouse click or an accelerator to open a menu item how the app was opened (via command line, start menu, start center, or by opening a document)
- in which toolbar a button was clicked, as some buttons are in multiple toolbars, and users can add buttons to toolbars individually
which slide transitions and object animations are used most in Impress the concrete action/command sequence: which action was used by the user, and which was the next action used after that (e.g. inserting an image and then adding a caption) which menu bar keystroke sequences are used (e.g. Alt+F + O) which icon theme, font list and theme name the user has active
Redesign of chart wizard UI
CostEstimate: 6 weeks Contact: Bubli
- Redesign of chart wizard UI, pretty sexy mockups from Heiko Tietze here: http://user-prompt.com/libreoffice-design-session-inserting-a-chart/
- Related to that - chart styles. Was a part of this OSBA thingit, but never found funding: http://osb-alliance.de/fileadmin/Working_Groups/OfficeInteroperability/Project2/SpecificationMajorFeatureImprovements_final.pdf
Improving printing UI
CostEstimate: 20 weeks Contact: Bubli
- Improving printing UI, more sexy mockup from Heiko here: http://pad.documentfoundation.org/p/UX-PrintDialog
Already Resourced / Abandoned
Automatic Detection and Bisection of Regressions
CostEstimate: in-progess Contact: Luke Document and improve the https://github.com/milossramek/office-interoperability-tools tools described at https://www.youtube.com/watch?v=cfzMP4oTZwo for the QA team. Add into LO QA framework. Have an end goal of reaching a state where it could run automatic weekly reports against a pool of test documents. Xisco: I'm already working on it: https://github.com/x1sc0/office-interoperability-tools
UPDATE 02-22-2018 (Xisco) The office-interoperability tool is already working in a vm testing rtf, doc, docx, odt (export), ppt, pptx and odp (export) against master builds using 8,450 files in total. The issues found by the tool can be found here: https://bugs.documentfoundation.org/buglist.cgi?list_id=778569&longdesc=office-interoperability-tools&longdesc_type=casesubstring&query_format=advanced
could we have a 'tips' scheme
CostEstimate: ? Contact: Heiko
- KDE side, use pay-pal only https://www.kde.org/fundraisers/yearend2016/
- sounds like re-inventing freedom sponsors (Bubli)
- if this happens - do it outside the foundation to avoid issues (Norbert)
- like barnstars but with a financial 'tip' - is the idea.
abandoned - Norbert (TDF treasurer) is not a fan; so lets do it outside TDF funding.
patch update code
CostEstimate: unclear yet / just for Mac? Contact: Markus
- allow pushing patches, lots of details to sort out
- have a FOSDEM talk for this in the dev-room
- talk to releng & devs there @ the hack-fest.
=> not sure it will work out as a tender; already 70% done.
- Windows & Linux ~done; no Mac so not tested
defer - until after FOSDEM.
Proposals missing detailed descriptions for tenders in budget 2018
Better integration of extensions (Heiko)
Design-team page of topics for GSOC (Heiko)
Proposals for tenders in budget 2018 that are likely better suited as a GSOC
Subpixel and stable glyph positioning
Fixing the bad text spacing we have due to lack of subpixel positioning. All graphics system we use (except GDI, but this is dying) support subpixel positioning but we make no use of it as all our glyphs positions are stored using integers (not to mention the rounding errors all over the place). This results in very bad text spacing and it gets worse on hidpi screens. Thinking about this more, I think it can be a GSoC project.
Add Impress remote capabilities for simple ad-hoc fingerpainting ad-hoc sketches
Contact: Bjoern Likely better suited to GSOC -- adding it here as we collect budget extensions. The idea is basically to allow to do simple ad-hoc sketches on a tablet with the impress remote, e.g. to answer an audience question on the spot. Allowing the tablet to use the projector as a modern chalkboard.
Make Calc/Base/Chart handle small set of geographic data
Allow Calc/Base/Chart to handle regional subsets of geographic data, e.g. import/edit/draw a small region of OpenStreetMap data.