Why Feature Testing?
Features are changes that add new functionality or significantly modify existing functionality. They are designed to be useful, effective, and attracting. Though, users are inventive and use the functionality unexpected ways. Also the application is complex and all pieces have to cooperate with each other. Any change might have surprising side effects in the related parts. Finally, developers are just humans and they might simply forget to implement a piece.
Feature testing should catch the above mentioned problems and make sure that users will be happy with the new stuff.
In practice, we need to define a new test set for feature testing in each release. Most of the feature tests are expected to be tested once per release. For those extremely important and generally used new feature need to test through each build in future, we have to consider to put them into regression test.
How to Test Feature?
It is a pretty interesting job because it maps something new and unexplored. Here are some ideas how to do it:
- find new feature:
- find one at the release notes page for the upcoming release; FIXME: how to check that nobody tested it yet? note on the wiki page? entry in MozTrap?
- monitor firstname.lastname@example.org mailing list; developers ask there for help from time to time because it is easier to fix problems when they work on the feature and have the code in mind
- make others aware that it is being tested:
- FIXME: how? in wiki? in MozTrap?
- understand the feature:
- read the description in the release notes or the mail from the developer
- if you are unsure, contact the author; it should be mentioned in brackets in the release notes; if you can't find her, ask on the developers mailing list
- (optional) ask the developer for pointers to potential problems and side effects
- they might have interesting ideas regarding what needs to be checked
- download and install build where the feature is included:
- features mentioned in the release notes are available in the daily builds
- features mentioned in the mailing list might be available only in a branch; there is a need to ask the developer for the status; they might need to provide extra build
- during the beta/rc phase, the features are available also in official builds
- play with the new functionality
- try various expected and unexpected scenarios
- watch if it works as expected
- be careful for side effects in the related pieces
- report found problems
IMPORTANT: There is no need to repeat tests from the full feature test. They will be done anyway.