The GeoTools Module Matrix makes use of a gold star system, just like in school. 3 stars or more is great, an X is used to indicate non working modules.
This test is something quick, accessable and visisble to end users.
Testing a Module
Here is how a library (aka part of the geotools library API) may earn a star:
Passes IP check documented in review.txt file, basically has correct headers
Releasable - has no non blocking bugs in jira
Quality Assurance - Test Case coverage
Stability - based on reviewed GeoAPI interfaces or latest ISO/OGC spec (example referencing)
Supported - user docs, module maintainer watches user list, answers email etc.. (example referencing)
Testing a Plugin
Here is how a plugin (aka hooks into the geotools library) may earn a star:
Passes IP check, basically has correct headers
Releasable - has no non blocking bugs in jira
Used in anger - Used by GeoServer or uDig on large real world datasets
Optimized - has been tuned to meet hard performance requirements (example shapefile)
Supported - user docs, module maintainer watches user list, answers email etc.. (example referencing)
Testing an Extension
Here is how a extension (aka build on top of the geotools library) may earn a star:
Passes IP check, basically has correct headers
Releasable - has no non blocking bugs in jira
Quality Assurance - Test Case coverage
Stability - based onreviewed GeoAPI interfaces, or at least a ISO/OGC specification
Supported - user docs, module maintainer watches user list, answers email etc.. (example graph)