Feature Tests

    From The Document Foundation Wiki
    < QA

    This page was marked as inactive and is retained for historical reference.
    We no longer do manual testing. All the test cases were moved to automated tests. Feature testing should now live in bug hunting session pages.

    Why Feature Testing?

    Features are changes that add new functionality or significantly modify existing functionality. They are designed to be useful, effective, and attractive. Though, users are inventive and use them in unexpected ways. Also the application is complex and all pieces have to cooperate with each other. Any change might have surprising side effects in the related parts. Finally, developers are just humans and they might simply forget to implement something.

    Feature testing should catch the above-mentioned problems and make sure that users will be happy with the new stuff.

    In practice, we need to define a new test set for feature-testing in each release. Most of the feature-tests are expected to be tested once per release. For those extremely important and generally-used new features, which we need to test through each build in future, we have to consider putting them into regression test.

    How to Test Features?

    It is a pretty interesting job because it maps something new and unexplored. Here are some ideas how to do it:

    • find new feature:
      • find one at the release notes page for the upcoming release; FIXME: how to check that nobody tested it yet? note on the wiki page? entry in TestLink? (See TestLink for more information.)
      • monitor [1] mailing list; developers ask there for help from time to time, because it is easier to fix problems when they work on the feature and have the code in mind
    • make others aware that it is being tested:
      • FIXME: how? in wiki? in TestLink?
    • understand the feature:
      • read the description in the release notes or the mail from the developer
      • if you are unsure, contact the author; it should be mentioned in brackets in the release notes; if you can’t find them, ask on the developers mailing list
    • (optional) ask the developer for pointers to potential problems and side effects
      • they might have interesting ideas regarding what needs to be checked
    • download and install build where the feature is included:
      • features mentioned in the release notes are available in the daily builds
      • features mentioned in the mailing list might be available only in a branch; there is a need to ask the developer for the status; they might need to provide extra builds
      • during the beta/rc phase, the features are available also in official builds
    • play with the new functionality;
    • try various expected and unexpected scenarios;
    • watch if it works as expected;
    • be careful for side-effects in the related pieces; and
    • report found problems.

    IMPORTANT: There is no need to repeat tests from the full feature test. They will be done anyway.