/usr/share/doc/clang-5.0-doc/html/SourceBasedCodeCoverage.html is in clang-5.0-doc 1:5.0.1-4.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Source-based Code Coverage — Clang 5 documentation</title>
<link rel="stylesheet" href="_static/haiku.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '5',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Modules" href="Modules.html" />
<link rel="prev" title="SafeStack" href="SafeStack.html" />
</head>
<body>
<div class="header" role="banner"><h1 class="heading"><a href="index.html">
<span>Clang 5 documentation</span></a></h1>
<h2 class="heading"><span>Source-based Code Coverage</span></h2>
</div>
<div class="topnav" role="navigation" aria-label="top navigation">
<p>
«  <a href="SafeStack.html">SafeStack</a>
  ::  
<a class="uplink" href="index.html">Contents</a>
  ::  
<a href="Modules.html">Modules</a>  »
</p>
</div>
<div class="content">
<div class="section" id="source-based-code-coverage">
<h1>Source-based Code Coverage<a class="headerlink" href="#source-based-code-coverage" title="Permalink to this headline">¶</a></h1>
<div class="contents local topic" id="contents">
<ul class="simple">
<li><a class="reference internal" href="#introduction" id="id1">Introduction</a></li>
<li><a class="reference internal" href="#the-code-coverage-workflow" id="id2">The code coverage workflow</a></li>
<li><a class="reference internal" href="#compiling-with-coverage-enabled" id="id3">Compiling with coverage enabled</a></li>
<li><a class="reference internal" href="#running-the-instrumented-program" id="id4">Running the instrumented program</a></li>
<li><a class="reference internal" href="#creating-coverage-reports" id="id5">Creating coverage reports</a></li>
<li><a class="reference internal" href="#exporting-coverage-data" id="id6">Exporting coverage data</a></li>
<li><a class="reference internal" href="#interpreting-reports" id="id7">Interpreting reports</a></li>
<li><a class="reference internal" href="#format-compatibility-guarantees" id="id8">Format compatibility guarantees</a></li>
<li><a class="reference internal" href="#using-the-profiling-runtime-without-static-initializers" id="id9">Using the profiling runtime without static initializers</a></li>
<li><a class="reference internal" href="#collecting-coverage-reports-for-the-llvm-project" id="id10">Collecting coverage reports for the llvm project</a></li>
<li><a class="reference internal" href="#drawbacks-and-limitations" id="id11">Drawbacks and limitations</a></li>
</ul>
</div>
<div class="section" id="introduction">
<h2><a class="toc-backref" href="#id1">Introduction</a><a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h2>
<p>This document explains how to use clang’s source-based code coverage feature.
It’s called “source-based” because it operates on AST and preprocessor
information directly. This allows it to generate very precise coverage data.</p>
<p>Clang ships two other code coverage implementations:</p>
<ul class="simple">
<li><a class="reference internal" href="SanitizerCoverage.html"><span class="doc">SanitizerCoverage</span></a> - A low-overhead tool meant for use alongside the
various sanitizers. It can provide up to edge-level coverage.</li>
<li>gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
This is enabled by <code class="docutils literal"><span class="pre">-ftest-coverage</span></code> or <code class="docutils literal"><span class="pre">--coverage</span></code>.</li>
</ul>
<p>From this point onwards “code coverage” will refer to the source-based kind.</p>
</div>
<div class="section" id="the-code-coverage-workflow">
<h2><a class="toc-backref" href="#id2">The code coverage workflow</a><a class="headerlink" href="#the-code-coverage-workflow" title="Permalink to this headline">¶</a></h2>
<p>The code coverage workflow consists of three main steps:</p>
<ul class="simple">
<li>Compiling with coverage enabled.</li>
<li>Running the instrumented program.</li>
<li>Creating coverage reports.</li>
</ul>
<p>The next few sections work through a complete, copy-‘n-paste friendly example
based on this program:</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">cat</span> <span class="o"><<</span><span class="n">EOF</span> <span class="o">></span> <span class="n">foo</span><span class="p">.</span><span class="n">cc</span>
<span class="cp">#define BAR(x) ((x) || (x))</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="n">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">foo</span><span class="p">(</span><span class="n">T</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">unsigned</span> <span class="n">I</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">I</span> <span class="o"><</span> <span class="mi">10</span><span class="p">;</span> <span class="o">++</span><span class="n">I</span><span class="p">)</span> <span class="p">{</span> <span class="n">BAR</span><span class="p">(</span><span class="n">I</span><span class="p">);</span> <span class="p">}</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">foo</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="n">foo</span><span class="o"><</span><span class="kt">float</span><span class="o">></span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">EOF</span>
</pre></div>
</div>
</div>
<div class="section" id="compiling-with-coverage-enabled">
<h2><a class="toc-backref" href="#id3">Compiling with coverage enabled</a><a class="headerlink" href="#compiling-with-coverage-enabled" title="Permalink to this headline">¶</a></h2>
<p>To compile code with coverage enabled, pass <code class="docutils literal"><span class="pre">-fprofile-instr-generate</span>
<span class="pre">-fcoverage-mapping</span></code> to the compiler:</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">#</span> Step <span class="m">1</span>: Compile with coverage enabled.
<span class="gp">%</span> clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
</pre></div>
</div>
<p>Note that linking together code with and without coverage instrumentation is
supported. Uninstrumented code simply won’t be accounted for in reports.</p>
</div>
<div class="section" id="running-the-instrumented-program">
<h2><a class="toc-backref" href="#id4">Running the instrumented program</a><a class="headerlink" href="#running-the-instrumented-program" title="Permalink to this headline">¶</a></h2>
<p>The next step is to run the instrumented program. When the program exits it
will write a <strong>raw profile</strong> to the path specified by the <code class="docutils literal"><span class="pre">LLVM_PROFILE_FILE</span></code>
environment variable. If that variable does not exist, the profile is written
to <code class="docutils literal"><span class="pre">default.profraw</span></code> in the current directory of the program. If
<code class="docutils literal"><span class="pre">LLVM_PROFILE_FILE</span></code> contains a path to a non-existent directory, the missing
directory structure will be created. Additionally, the following special
<strong>pattern strings</strong> are rewritten:</p>
<ul class="simple">
<li>“%p” expands out to the process ID.</li>
<li>“%h” expands out to the hostname of the machine running the program.</li>
<li>“%Nm” expands out to the instrumented binary’s signature. When this pattern
is specified, the runtime creates a pool of N raw profiles which are used for
on-line profile merging. The runtime takes care of selecting a raw profile
from the pool, locking it, and updating it before the program exits. If N is
not specified (i.e the pattern is “%m”), it’s assumed that <code class="docutils literal"><span class="pre">N</span> <span class="pre">=</span> <span class="pre">1</span></code>. N must
be between 1 and 9. The merge pool specifier can only occur once per filename
pattern.</li>
</ul>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">#</span> Step <span class="m">2</span>: Run the program.
<span class="gp">%</span> <span class="nv">LLVM_PROFILE_FILE</span><span class="o">=</span><span class="s2">"foo.profraw"</span> ./foo
</pre></div>
</div>
</div>
<div class="section" id="creating-coverage-reports">
<h2><a class="toc-backref" href="#id5">Creating coverage reports</a><a class="headerlink" href="#creating-coverage-reports" title="Permalink to this headline">¶</a></h2>
<p>Raw profiles have to be <strong>indexed</strong> before they can be used to generate
coverage reports. This is done using the “merge” tool in <code class="docutils literal"><span class="pre">llvm-profdata</span></code>
(which can combine multiple raw profiles and index them at the same time):</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">#</span> Step <span class="m">3</span><span class="o">(</span>a<span class="o">)</span>: Index the raw profile.
<span class="gp">%</span> llvm-profdata merge -sparse foo.profraw -o foo.profdata
</pre></div>
</div>
<p>There are multiple different ways to render coverage reports. The simplest
option is to generate a line-oriented report:</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">#</span> Step <span class="m">3</span><span class="o">(</span>b<span class="o">)</span>: Create a line-oriented coverage report.
<span class="gp">%</span> llvm-cov show ./foo -instr-profile<span class="o">=</span>foo.profdata
</pre></div>
</div>
<p>This report includes a summary view as well as dedicated sub-views for
templated functions and their instantiations. For our example program, we get
distinct views for <code class="docutils literal"><span class="pre">foo<int>(...)</span></code> and <code class="docutils literal"><span class="pre">foo<float>(...)</span></code>. If
<code class="docutils literal"><span class="pre">-show-line-counts-or-regions</span></code> is enabled, <code class="docutils literal"><span class="pre">llvm-cov</span></code> displays sub-line
region counts (even in macro expansions):</p>
<div class="highlight-none"><div class="highlight"><pre><span></span> 1| 20|#define BAR(x) ((x) || (x))
^20 ^2
2| 2|template <typename T> void foo(T x) {
3| 22| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
^22 ^20 ^20^20
4| 2|}
------------------
| void foo<int>(int):
| 2| 1|template <typename T> void foo(T x) {
| 3| 11| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
| ^11 ^10 ^10^10
| 4| 1|}
------------------
| void foo<float>(int):
| 2| 1|template <typename T> void foo(T x) {
| 3| 11| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
| ^11 ^10 ^10^10
| 4| 1|}
------------------
</pre></div>
</div>
<p>To generate a file-level summary of coverage statistics instead of a
line-oriented report, try:</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">#</span> Step <span class="m">3</span><span class="o">(</span>c<span class="o">)</span>: Create a coverage summary.
<span class="gp">%</span> llvm-cov report ./foo -instr-profile<span class="o">=</span>foo.profdata
<span class="go">Filename Regions Missed Regions Cover Functions Missed Functions Executed Lines Missed Lines Cover</span>
<span class="go">--------------------------------------------------------------------------------------------------------------------------------------</span>
<span class="go">/tmp/foo.cc 13 0 100.00% 3 0 100.00% 13 0 100.00%</span>
<span class="go">--------------------------------------------------------------------------------------------------------------------------------------</span>
<span class="go">TOTAL 13 0 100.00% 3 0 100.00% 13 0 100.00%</span>
</pre></div>
</div>
<p>The <code class="docutils literal"><span class="pre">llvm-cov</span></code> tool supports specifying a custom demangler, writing out
reports in a directory structure, and generating html reports. For the full
list of options, please refer to the <a class="reference external" href="http://llvm.org/docs/CommandGuide/llvm-cov.html">command guide</a>.</p>
<p>A few final notes:</p>
<ul>
<li><p class="first">The <code class="docutils literal"><span class="pre">-sparse</span></code> flag is optional but can result in dramatically smaller
indexed profiles. This option should not be used if the indexed profile will
be reused for PGO.</p>
</li>
<li><p class="first">Raw profiles can be discarded after they are indexed. Advanced use of the
profile runtime library allows an instrumented program to merge profiling
information directly into an existing raw profile on disk. The details are
out of scope.</p>
</li>
<li><p class="first">The <code class="docutils literal"><span class="pre">llvm-profdata</span></code> tool can be used to merge together multiple raw or
indexed profiles. To combine profiling data from multiple runs of a program,
try e.g:</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">%</span> llvm-profdata merge -sparse foo1.profraw foo2.profdata -o foo3.profdata
</pre></div>
</div>
</li>
</ul>
</div>
<div class="section" id="exporting-coverage-data">
<h2><a class="toc-backref" href="#id6">Exporting coverage data</a><a class="headerlink" href="#exporting-coverage-data" title="Permalink to this headline">¶</a></h2>
<p>Coverage data can be exported into JSON using the <code class="docutils literal"><span class="pre">llvm-cov</span> <span class="pre">export</span></code>
sub-command. There is a comprehensive reference which defines the structure of
the exported data at a high level in the llvm-cov source code.</p>
</div>
<div class="section" id="interpreting-reports">
<h2><a class="toc-backref" href="#id7">Interpreting reports</a><a class="headerlink" href="#interpreting-reports" title="Permalink to this headline">¶</a></h2>
<p>There are four statistics tracked in a coverage summary:</p>
<ul class="simple">
<li>Function coverage is the percentage of functions which have been executed at
least once. A function is considered to be executed if any of its
instantiations are executed.</li>
<li>Instantiation coverage is the percentage of function instantiations which
have been executed at least once. Template functions and static inline
functions from headers are two kinds of functions which may have multiple
instantiations.</li>
<li>Line coverage is the percentage of code lines which have been executed at
least once. Only executable lines within function bodies are considered to be
code lines.</li>
<li>Region coverage is the percentage of code regions which have been executed at
least once. A code region may span multiple lines (e.g in a large function
body with no control flow). However, it’s also possible for a single line to
contain multiple code regions (e.g in “return x || y && z”).</li>
</ul>
<p>Of these four statistics, function coverage is usually the least granular while
region coverage is the most granular. The project-wide totals for each
statistic are listed in the summary.</p>
</div>
<div class="section" id="format-compatibility-guarantees">
<h2><a class="toc-backref" href="#id8">Format compatibility guarantees</a><a class="headerlink" href="#format-compatibility-guarantees" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li>There are no backwards or forwards compatibility guarantees for the raw
profile format. Raw profiles may be dependent on the specific compiler
revision used to generate them. It’s inadvisable to store raw profiles for
long periods of time.</li>
<li>Tools must retain <strong>backwards</strong> compatibility with indexed profile formats.
These formats are not forwards-compatible: i.e, a tool which uses format
version X will not be able to understand format version (X+k).</li>
<li>Tools must also retain <strong>backwards</strong> compatibility with the format of the
coverage mappings emitted into instrumented binaries. These formats are not
forwards-compatible.</li>
<li>The JSON coverage export format has a (major, minor, patch) version triple.
Only a major version increment indicates a backwards-incompatible change. A
minor version increment is for added functionality, and patch version
increments are for bugfixes.</li>
</ul>
</div>
<div class="section" id="using-the-profiling-runtime-without-static-initializers">
<h2><a class="toc-backref" href="#id9">Using the profiling runtime without static initializers</a><a class="headerlink" href="#using-the-profiling-runtime-without-static-initializers" title="Permalink to this headline">¶</a></h2>
<p>By default the compiler runtime uses a static initializer to determine the
profile output path and to register a writer function. To collect profiles
without using static initializers, do this manually:</p>
<ul class="simple">
<li>Export a <code class="docutils literal"><span class="pre">int</span> <span class="pre">__llvm_profile_runtime</span></code> symbol from each instrumented shared
library and executable. When the linker finds a definition of this symbol, it
knows to skip loading the object which contains the profiling runtime’s
static initializer.</li>
<li>Forward-declare <code class="docutils literal"><span class="pre">void</span> <span class="pre">__llvm_profile_initialize_file(void)</span></code> and call it
once from each instrumented executable. This function parses
<code class="docutils literal"><span class="pre">LLVM_PROFILE_FILE</span></code>, sets the output path, and truncates any existing files
at that path. To get the same behavior without truncating existing files,
pass a filename pattern string to <code class="docutils literal"><span class="pre">void</span> <span class="pre">__llvm_profile_set_filename(char</span>
<span class="pre">*)</span></code>. These calls can be placed anywhere so long as they precede all calls
to <code class="docutils literal"><span class="pre">__llvm_profile_write_file</span></code>.</li>
<li>Forward-declare <code class="docutils literal"><span class="pre">int</span> <span class="pre">__llvm_profile_write_file(void)</span></code> and call it to write
out a profile. This function returns 0 when it succeeds, and a non-zero value
otherwise. Calling this function multiple times appends profile data to an
existing on-disk raw profile.</li>
</ul>
<p>In C++ files, declare these as <code class="docutils literal"><span class="pre">extern</span> <span class="pre">"C"</span></code>.</p>
</div>
<div class="section" id="collecting-coverage-reports-for-the-llvm-project">
<h2><a class="toc-backref" href="#id10">Collecting coverage reports for the llvm project</a><a class="headerlink" href="#collecting-coverage-reports-for-the-llvm-project" title="Permalink to this headline">¶</a></h2>
<p>To prepare a coverage report for llvm (and any of its sub-projects), add
<code class="docutils literal"><span class="pre">-DLLVM_BUILD_INSTRUMENTED_COVERAGE=On</span></code> to the cmake configuration. Raw
profiles will be written to <code class="docutils literal"><span class="pre">$BUILD_DIR/profiles/</span></code>. To prepare an html
report, run <code class="docutils literal"><span class="pre">llvm/utils/prepare-code-coverage-artifact.py</span></code>.</p>
<p>To specify an alternate directory for raw profiles, use
<code class="docutils literal"><span class="pre">-DLLVM_PROFILE_DATA_DIR</span></code>. To change the size of the profile merge pool, use
<code class="docutils literal"><span class="pre">-DLLVM_PROFILE_MERGE_POOL_SIZE</span></code>.</p>
</div>
<div class="section" id="drawbacks-and-limitations">
<h2><a class="toc-backref" href="#id11">Drawbacks and limitations</a><a class="headerlink" href="#drawbacks-and-limitations" title="Permalink to this headline">¶</a></h2>
<ul>
<li><p class="first">Prior to version 2.26, the GNU binutils BFD linker is not able link programs
compiled with <code class="docutils literal"><span class="pre">-fcoverage-mapping</span></code> in its <code class="docutils literal"><span class="pre">--gc-sections</span></code> mode. Possible
workarounds include disabling <code class="docutils literal"><span class="pre">--gc-sections</span></code>, upgrading to a newer version
of BFD, or using the Gold linker.</p>
</li>
<li><p class="first">Code coverage does not handle unpredictable changes in control flow or stack
unwinding in the presence of exceptions precisely. Consider the following
function:</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="kt">int</span> <span class="nf">f</span><span class="p">()</span> <span class="p">{</span>
<span class="n">may_throw</span><span class="p">();</span>
<span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>If the call to <code class="docutils literal"><span class="pre">may_throw()</span></code> propagates an exception into <code class="docutils literal"><span class="pre">f</span></code>, the code
coverage tool may mark the <code class="docutils literal"><span class="pre">return</span></code> statement as executed even though it is
not. A call to <code class="docutils literal"><span class="pre">longjmp()</span></code> can have similar effects.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="bottomnav" role="navigation" aria-label="bottom navigation">
<p>
«  <a href="SafeStack.html">SafeStack</a>
  ::  
<a class="uplink" href="index.html">Contents</a>
  ::  
<a href="Modules.html">Modules</a>  »
</p>
</div>
<div class="footer" role="contentinfo">
© Copyright 2007-2018, The Clang Team.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.7.
</div>
</body>
</html>
|