openstack/karma-subunit-reporter
Code Issues Proposed changes

Add normalization documentation to README

Browse Source 
This adds details about test name normalization to the README, along
with additional documentation on potential output differences and
other caveats involved with this plugin's output.

Change-Id: Id2718ead4a572bd33796706c8d5c04bd671c19c1
This commit is contained in:
Tim Buckley 2016-06-15 10:26:33 -06:00
parent 27a6ef7e06
commit 0477320662
1 changed files with 57 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
README.md
View File

@ -41,10 +41,64 @@ module.exports = function(config) {
subunitReporter: { subunitReporter: {
outputFile: 'karma.subunit', outputFile: 'karma.subunit',
tags: [], // tag strings to append to all tests tags: [], // tag strings to append to all tests
slug: false, // convert whitespace to '_' slug: false, // if true, convert whitespace to '_'
separator: '.' // separator for suite components + test name separator: '.', // separator for suite components + test name
normalize: null // an optional normalization function, see below
} }
}); });
}; };
``` ```
### Normalizing Test Names
Given that JavaScript's test name formatting conventions are dissimilar to those
in other subunit-supported languages (particularly, Python) it may be desirable
to convert tokens to something that more closely resembles the other languages
in your environment. Additionally, tooling may require that test names be
reasonably well-formed for parsing and categorization purposes.
In all cases, test name tokens are gathered from karma's `suite` field along
with the test's description. For Jasmine tests, `suite` is an ordered list of
enclosing `describe()` strings for each test. The list of tokens (i.e. suites
plus description) are then passed to some mapping function.
If not otherwise configured, this mapping function simply returns the list
of tokens unaltered; however, it can also be configured to perform simple
conversions from Jasmine-style test descriptions to simple, dot-separated
Python-ish test names. If configured with `slug: true`, spaces will be collapsed
and converted to `_`, which yields decent results in some cases.
If more advanced conversion is needed, a custom mapping function can be provided
using the `normalize` parameter, which accepts a list of tokens and returns a
modified list of tokens.
As a final step, mapped tokens are then joined using the `separator` parameter
(`.` by default).
## Tags
The plugin automatically appends certain tags to each test: the browser ID
(`browser-*`) and the spec ID (`spec-*`). If multiple browsers are tested
concurrently these tags can be used to match results across browsers.
Constant tags to apply to all tests can be also be specified using the `tags`
config option. Each string in the list is unconditionally appended to all tests,
and can be used as a potential compatibility measure (for example, to support
tools that assume a `worker-*` tag should exist).
## Output Differences
While in theory this plugin's output should work correctly with all
subunit-compatible tooling, in practice it may violate some assumptions that
can be made safely in other languages.
For one, tests only run in a single worker thread (and unless configured, no
`worker-#` tag is present). Additionally, within the single worker, tests are
(at least potentially) run asynchronously and may have overlapping execution
timelines.
Some other minor differences also exist. Currently file attachments aren't
supported, so log output, error messages, and tracebacks won't be included in
the subunit output. There's also no 'setUpClass' or other events that might be
logged in Python tests.