1 Coverage Analyser
The module
cover
provides a set of functions which can be used for coverage analysis of Erlang programs. Coverage analysis consists of monitoring executing programs and observing how often every line of code is executed. Statistics gained from the monitoring process can be used for various purposes. The most important are:
- Finding the most frequently called functions. This is useful for "tuning" code.
- Finding if parts of the code are never executed. This is useful during testing to find out how much of the code that is covered with test cases.
This section describes the Coverage Analyser and shows examples of typical coverage analysis. Refer to the Reference Manual, tools,
cover
for the complete details about the functions available with this tool.1.1 Functions
The module cover exports the following functions.
1.2 Using the Coverage Analyser
Follow these steps to perform coverage analysis of Erlang programs:
- Make sure that the modules you want to analyse contain no syntax errors, and that they can be run with reasonable test data.
- Create or change to a directory where the coverage analysis will be run (see example below).
- Prepare the modules for coverage analysis by evaluating
cover:file(File [,Options])
orcover:all_files(Dir[,Options])
(see Note Prepare Modules below this list).- Restart the system from the directory where the coverage analysis will be run.
- Run a set of test queries or test suites.
- Use the functions
most_called/1
,quick_analyse/1
,analyse/1
to analyse the results of the test run.
Files with extensions
.COVER.erl
must not be changed or moved after they have been created, or the coverage analysis will not work. They can be deleted when the coverage analysis has been completed.The following example illustrates how to perform coverage analysis. The module analysed
io_lib
, which is part of the standard Erlang distribution.> cd ~/erl > mkdir cover_data > cd cover_data > erl -nostick ..... > cover:file(io_lib). Input file:otp_root/lib/stdlib-1.0/src/io_lib Output file:io_lib.COVER.erl Transforming otp_root/lib/stdlib-1.0/src/io_lib.erl for profiling scan/3 scan/1 scan/2 reserved_word/1 fwrite/2 fread/2 fread/3 format/2 ... ... Now compiling module io_lib Module io_lib will now be purged and re-loaded {module,io_lib}This run creates the files
io_lib.COVER.erl
andio_lib.jam
in the current directory, which is the directorycover_data
in the example shown.
io_lib.COVER.erl
is the transformed version of the originalio_lib.erl
io_lib.jam
is the compiled object code for the transformed code. The code in io_lib.jam, when evaluated, will log coverage statistics.
The
-nostick
option is given to theerl
command becauseio_lib
is a library module which resides in a "sticky" directory which means that the system is not normally allowed to load the module from any other place. The option-nostick
removes this restriction. The option is used here only becauseio_lib
provides a good example.
A "sticky" directory is a directory which contains system files which cannot be moved or modified.
Do not use the
-nostick
option unless you are really sure what you are doing.The next step is to force usage of code in
io_lib
. In the example shown, the shell internal functioni()
is evaluated. This forces use of the i/o system. The functioncover:most_called
is then used to analyse the results.2> i(). Pid Initial Call Current Function <0.0.0> {init,boot,1} {init,loop,8} ... <0.2.0> {erl_prim_loader,start_ {erl_prim_loader,loop,5}... <0.4.0> {gen_event,init_it,6} {gen_event,loop,4} ... <0.5.0> {application_controller {gen_server,loop,6} ... ... 3> cover:most_called(io_lib). 1 3844 69.07 69.07 io_lib:deep_char_list/2 2 613 11.02 80.09 io_lib:quote_atom/1 3 256 4.60 84.69 io_lib:write/2 4 243 4.37 89.06 io_lib:write_atom/1 5 243 4.37 93.42 io_lib:quote_atom/2 6 128 2.30 95.72 io_lib:deep_char_list/1 7 114 2.05 97.77 io_lib:write_tail/2 8 99 1.78 99.55 io_lib:write/1 9 23 0.41 99.96 io_lib:format/2 10 2 0.04 100.00 io_lib:fwrite/2 ...The function
cover:most_called(io_lib)
analyses the results and prints out a sorted list of the most called functions inio_lib
. Printout from cover:most_called/1 shows the meaning of the numbers printed to the left of the function names.
Printout from cover:most_called/1For example,
io_lib:deep_char_list/2
accounted for 69.07% of all calls, and the first four calls of the list accounted for almost 90% of all calls.A brief summary of the coverage of the code in
io_lib
can be obtained by evaluatingcover:quick_analyse(io_lib)
.4> cover:quick_analyse(io_lib). COVERAGE ANALYSIS (QUICK) FOR MODULE io_lib *** 22 PERCENT *** of the statements are covered NOT COVERED:85 COVERED: 24 TOTAL NUMBER OF STATEMENTS: 109 COVERAGE FOR FUNCTIONS IN MODULE io_lib *** NOT CALLED *** collect_chars/3 scan/2 scan/1 write_char/1 print/1 write_string/1 indentation/2 char_list/1 nl/0 collect_line/2 collect_chars1/3 write_char/3 ... *** 28 PERCENT COVERED *** write/2 *** 43 PERCENT COVERED *** quote_atom/1 *** 50 PERCENT COVERED *** write_tail/2 *** 60 PERCENT COVERED *** quote_atom/2 *** 75 PERCENT COVERED *** write_atom/1 *** 80 PERCENT COVERED *** deep_char_list/2 *** 100 PERCENT COVERED *** write/1 deep_char_list/1 format/2 fwrite/2 okThe functions are sorted in groups according to the amount of code which has been covered. Functions grouped as
NOT CALLED
did not execute at all. Functions grouped as100 PERCENT COVERED
have executed every line of code at least once. The remainder of the functions are listed in groups which indicate the percentage of code within the function which has been executed and evaluated.A more detailed analysis can be obtained by running
cover:analyse
as shown in the following example:5> cover:analyse(io_lib). Found Object code in:./io_lib.jam Object code last modified:{1996,11,19,13,9,37} Found source file in:io_lib.COVER.erl Source code last modified:{1996,11,19,13,9,32} Analysing Source:io_lib.COVER.erl module:io_lib Output File:io_lib.COVER.erl.out scan/3 scan/1 scan/2 reserved_word/1 fwrite/2 fread/2 fread/3 format/2 print/1 print/4 indentation/2 write/1 write/3 write/2 write_tail/2 ...The evaluation of
cover:analyse(io_lib)
produces the fileio_lib.COVER.erl.out
in the current directory. This file contains a detailed listing of the number of times each line of the code inio_lib.erl
was called.The following example shows first few lines in
io_lib.COVER.erl.out
:$ more io_lib.COVER.erl.out scan(Cont,Chars,Pos) -> "**** Not Covered *****", erl_scan:tokens(Cont,Chars,Pos). ...... fwrite(Format,Args) -> "**** 4 ****", io_lib_format:fwrite(Format,Args). fread(Chars,Format) -> "**** Not Covered *****", io_lib_fread:fread(Chars,Format). format(Format,Args) -> "**** 138 ****", io_lib_format:fwrite(Format,Args). ....The listing shows how often each line of code in
io_lib.erl
was called.**** Not Covered **** indicates that this line of code was never evaluated. In an extensive test suite, it is desirable to force every line in a module to be evaluated at least once, as this is a valuable way of discovering programming errors.
The following functions are used to clear the coverage data in order to provide a fresh start for the collection of new coverage data.
cover:zero(Mod)
clears the coverage data of the moduleMod
cover:zero()
clears the coverage data for all modules
Coverage data is the statistical data accumulated by the coverage analyser as a result of executing code.
The following example illustrates the following interaction:
cover:zero()
clears all current coverage data- additional test cases are run
cover:most_called(io_lib)
produces a list which shows the most called functions inio-lib
for the new test cases.5> cover:zero(). ... do some work ... 6> cover:most_called(io_lib). 1 22272 63.45 63.45 io_lib:deep_char_list/2 2 3318 9.45 72.90 io_lib:quote_atom/1 3 2278 6.49 79.39 io_lib:write_string1/2 4 2154 6.14 85.52 io_lib:write_char/3 5 1182 3.37 88.89 io_lib:write_atom/1 6 1181 3.36 92.25 io_lib:quote_atom/2 7 827 2.36 94.61 io_lib:write/2 8 585 1.67 96.28 io_lib:format/2 9 422 1.20 97.48 io_lib:write/1 ...Refer to Printout from cover:most_called/1 again for an explanation of this printout.
1.3 Precautions
When files have been compiled for coverage analysis, you must ensure that the correct version of the object code is used for coverage analysis. The correct files are those which have been prepared for analysis with the function
cover:file(File)
.This situation can be illustrated with the example described in the previous section. Suppose that the Erlang system is stopped and restarted and the current directory is the same as when
io_lib
was prepared for analysis.
- Evaluate the function
cover:all_covered()
, using the current search path of the code server (refer Reference Manual, modulecode
for information about the search path of the code server). This function returns a list of all modules prepared for coverage analysis.- The function returns
[]
, which is an empty list.This result indicates that
io_lib
is not prepared for coverage analysis. The reason is that the system has been restarted andio_lib
is loaded from its standard location, where the original object code file resides. This code has not been prepared for coverage analysis.It is possible to load the version of
io_lib
which was prepared for coverage analysis by entering the following command from the shell:> code:load_file(io_lib). {module,io_lib} > cover:all_covered(). [io_lib].