.

Tags:

Achieving a good code coverage is a useful practice in today’s software craftsmanship. For JavaScript applications, whether it is for the browser or for the server, many tools to check the statement coverage are available. What about branch coverage? Fortunately, such a tool is finally available: Istanbul.

Made by Krishnan Anantheswaran from the YUI team Yahoo! Cocktail team, Istanbul is extremely easy to use. It is suitable for both browser-based instrumentation or Node.js application analysis. Under the hood, Istanbul uses Esprima to parse JavaScript code and Escodegen to generate the instrumented version of the code. Istanbul is pure JavaScript, there is no native code or other kind of native libraries wrapper involved.

Istanbul requires Node.js, the package can be installed as:

npm install istanbul

Using Istanbul with command-line Node.js application is fairly straightforward. Assuming you have test.js like this:

x =42;
if(false)
     x =-1;

We can analyze the coverage by running:

istanbul cover test.js

which gives the following outcome:

=============================== Coverage summary ===============================
Statements   : 66.67% ( 2/3 )
Branches     : 50% ( 1/2 )
Functions    : 100% ( 0/0 )
Lines        : 66.67% ( 2/3 )
================================================================================

Even if you put all the above code in just one line, Istanbul can still give the correct coverage result. This is because Istanbul does not work based on line-by-line coverage, it does fully understands JavaScript syntax. Beside the quick text-based report, Istanbul also produces the coverage report in LCOV format. This way, the coverage report can be nicely shown, with color coding and other fancy features:

What about something more complicated? I wrote some time ago about the trap of checking only statement coverage, the example which demonstrated that is the following fragment. Now, if your unit test only verify the result of running inc(4) and you forgot you also need the check for inc(4,2), you won’t see the bug easily. This example is of course just an oversimplified example, in real-world application the problem is much harder to spotted.

function inc(p, q){
    if(q ==undefined) q =1;
    return p + q/q;
}

However, if we run Istanbul so that we can see the branch coverage, the report reveals the issue. Here the marker ‘E’ indicates that the else path of the branch is never executed and that should provoke a suspicion.

How does Istanbul work under the hood? It takes a JavaScript program and passes it through Esprima to get the syntax tree. Then, it injects some instrumentation by wrapping various syntax constructs. After that, a new JavaScript program is generated from the syntax tree by using Escodegen. When the program runs, the injected annotation is also executed accordingly and thereby the statistics of the program execution can be gathered.

While Istanbul has a mechanism to automate running Node.js application with instrumentation, nothing stops you from using the instrumented code in other JavaScript environments (such as web browsers). For this purpose, Istanbul has a special instrument command to perform only the instrumentation. It is up to you to process the coverage hooks and reports.

With Istanbul, tracking JavaScript code coverage has never been easier!

  • Krishnan Anantheswaran

    Hey, thanks for the free marketing! I already see a spike in the number of repo stargazers :) We should meet soon to collaborate on related stuff. Perhaps early next year?

    • http://ariya.ofilabs.com/ Ariya Hidayat

      Thanks for the awesome tool, Krishnan! And yes, catching up early next year would be fantastic.

  • millermedeiros

    It can be tricky to get Istanbul working with some test runners, it only works if tests are executed programmatically – since it add hooks to the native `require()` calls – so if your test runner assumes you are using it from the command-line it can be a problem.

    jasmine-node doesn’t expose the `TerminalReporter` on the `jasmine` object [1] and `istanbul` currently doesn’t work well with scripts loaded with RequireJS [2]. I opened issues on both projects explaining the problems and with solutions.

    Istanbul is indeed a nice tool and it can help to find potential problems on the tests and code. But beware that 100% code coverage doesn’t mean that your tests/code are actually good, it just means that the tests executed every single line/branch/statement (maybe your code and tests are missing edge-cases). Code coverage is just another tool to help find problems, it doesn’t ensure that the code have no bugs. Use it in your favor, don’t become a slave.

    1. https://github.com/mhevery/jasmine-node/issues/184
    2. https://github.com/gotwarlost/istanbul/issues/23

    PS: Esprima opened the doors for so many cool projects, AST became “mainstream”. :D

  • OliverJAsh

    > While Istanbul has a mechanism to automate running Node.js application with instrumentation, nothing stops you from using the instrumented code in other JavaScript environments (such as web browsers).

    Is there any reason why you would choose to do this?

    • http://ariya.ofilabs.com/ Ariya Hidayat

      See e.g. istanbul-middleware. It is easier to do the instrumentation as part of the pre-processing, especially if the code is supposed to support both Node.js and browser anyway.

  • http://winkpress.com/ MK Safi

    How is it possible to have line coverage at 100% but branch coverage at 75%? If all lines are covered, all branches are covered. But Istanbul is giving me exactly this: one file has line coverage of 100% but branch coverage of 75%!