Bullseye Testing Technology

BullseyeCoverage Measurement Technique

Some products report several similar code coverage metrics and leave you to work out which one to use. BullseyeCoverage solves this problem by reporting only the best two metrics: function coverage for quickly assessing overall coverage, and condition/decision coverage for detailed testing.

Function Coverage

Function coverage tells whether each function was invoked or not. Looking at your function coverage allows you to quickly know what major areas of your software are untested. To find defects quickly, begin testing by attaining function coverage in all areas of your software before delving into detailed testing of any one area.

Condition/Decision Coverage

Condition/decision coverage measures whether you exercise every control structure with every possible decision outcome as well as every possible condition outcome. A decision is the whole expression affecting the flow of control in a control structure. A decision may contain conditions, which are sub-expressions separated by logical-and (&&) and logical-or (||) operators. Condition/decision coverage basically measures whether you tested all the logic of your control structures.

Condition/decision coverage is the overall best coverage metric for detailed white-box code testing. Condition/decision coverage balances usability with thoroughness. Simpler coverage metrics are blind to many obvious paths through the source code. For example, compared to statement coverage, branch coverage, and condition coverage, condition/decision coverage is stronger and just as easy to use. Condition/decision coverage sees more test cases than these metrics and therefore gives you a better picture of what you are not testing. More thorough coverage metrics require a significant increase in complexity combined with diminishing probability of exposing a bug. For example, compared to path coverage and its variants such as LCSAJ coverage and data flow coverage, condition/decision coverage is much, much easier to use.

Source Code Instrumentation

BullseyeCoverage uses source code instrumentation, which is required for the best coverage analysis. It also makes our technology easily adaptable to a wide variety of processors and platforms. But we did not sacrifice performance; our build-time and run-time speed ranks among the very best.

With Microsoft C++, our smart preprocessor minimizes performance impact while mimicking Microsoft C++ idiosyncracies. BullseyeCoverage recognizes macro invocations and adjusts the measurement to reflect what really needs testing.

Automatic Saving

BullseyeCoverage automatically saves coverage once per second. There is no trade-off between buffering and performance.

Performance

Execution time increases 1.2-1.5x. Code size increase 1.4x. Build time increases 2x with most Microsoft C++ projects, 3-8x with other compilers.