/usr/share/doc/ganeti/html/design-multi-version-tests.html is in ganeti-doc 2.16.0~rc2-1build1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Multi-version tests — Ganeti 2.16.0~rc2 documentation</title>
<link rel="stylesheet" href="_static/style.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '2.16.0~rc2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Network management" href="design-network.html" />
<link rel="prev" title="Moving instances across node groups" href="design-multi-reloc.html" />
</head>
<body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="design-network.html" title="Network management"
accesskey="N">next</a></li>
<li class="right" >
<a href="design-multi-reloc.html" title="Moving instances across node groups"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Ganeti 2.16.0~rc2 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="multi-version-tests">
<h1><a class="toc-backref" href="#id1">Multi-version tests</a><a class="headerlink" href="#multi-version-tests" title="Permalink to this headline">¶</a></h1>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Created:</th><td class="field-body">2013-Oct-30</td>
</tr>
<tr class="field-even field"><th class="field-name">Status:</th><td class="field-body">Implemented</td>
</tr>
<tr class="field-odd field"><th class="field-name">Ganeti-Version:</th><td class="field-body">2.11.0</td>
</tr>
</tbody>
</table>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#multi-version-tests" id="id1">Multi-version tests</a><ul>
<li><a class="reference internal" href="#desired-improvements" id="id2">Desired improvements</a></li>
<li><a class="reference internal" href="#required-use-cases" id="id3">Required use cases</a><ul>
<li><a class="reference internal" href="#compatibility-tests" id="id4">Compatibility tests</a></li>
<li><a class="reference internal" href="#upgrade-downgrade-tests" id="id5">Upgrade / downgrade tests</a></li>
</ul>
</li>
<li><a class="reference internal" href="#design-and-implementation" id="id6">Design and implementation</a><ul>
<li><a class="reference internal" href="#current-state" id="id7">Current state</a></li>
<li><a class="reference internal" href="#planned-changes" id="id8">Planned changes</a></li>
</ul>
</li>
<li><a class="reference internal" href="#potential-issues" id="id9">Potential issues</a></li>
</ul>
</li>
</ul>
</div>
<p>This is a design document describing how tests which use multiple
versions of Ganeti can be introduced into the current build
infrastructure.</p>
<div class="section" id="desired-improvements">
<h2><a class="toc-backref" href="#id2">Desired improvements</a><a class="headerlink" href="#desired-improvements" title="Permalink to this headline">¶</a></h2>
<p>The testing of Ganeti is currently done by using two different
approaches - unit tests and QA. While the former are useful for ensuring
that the individual parts of the system work as expected, most errors
are discovered only when all the components of Ganeti interact during
QA.</p>
<p>However useful otherwise, until now the QA has failed to provide support
for testing upgrades and version compatibility as it was limited to
using only one version of Ganeti. While these can be tested for every
release manually, a systematic approach is preferred and none can exist
with this restriction in place. To lift it, the buildbot scripts and QA
utilities must be extended to allow a way of specifying and using
diverse multi-version checks.</p>
</div>
<div class="section" id="required-use-cases">
<h2><a class="toc-backref" href="#id3">Required use cases</a><a class="headerlink" href="#required-use-cases" title="Permalink to this headline">¶</a></h2>
<p>There are two classes of multi-version tests that are interesting in
Ganeti, and this chapter provides an example from each to highlight what
should be accounted for in the design.</p>
<div class="section" id="compatibility-tests">
<h3><a class="toc-backref" href="#id4">Compatibility tests</a><a class="headerlink" href="#compatibility-tests" title="Permalink to this headline">¶</a></h3>
<p>One interface Ganeti exposes to clients interested in interacting with
it is the RAPI. Its stability has always been a design principle
followed during implementation, but whether it held true in practice was
not asserted through tests.</p>
<p>An automatic test of RAPI compatibility would have to take a diverse set
of RAPI requests and perform them on two clusters of different versions,
one of which would be the reference version. If the clusters had been
identically configured, all of the commands successfully executed on the
reference version should succeed on the newer version as well.</p>
<p>To achieve this, two versions of Ganeti can be run separately on a
cleanly setup cluster. With no guarantee that the versions can coexist,
the deployment of these has to be separate. A proxy placed between the
client and Ganeti records all the requests and responses. Using this
data, a testing utility can decide if the newer version is compatible or
not, and provide additional information to assist with debugging.</p>
</div>
<div class="section" id="upgrade-downgrade-tests">
<h3><a class="toc-backref" href="#id5">Upgrade / downgrade tests</a><a class="headerlink" href="#upgrade-downgrade-tests" title="Permalink to this headline">¶</a></h3>
<p>An upgrade / downgrade test serves to examine whether the state of the
cluster is unchanged after its configuration has been upgraded or
downgraded to another version of Ganeti.</p>
<p>The test works with two consecutive versions of Ganeti, both installed
on the same machine. It examines whether the configuration data and
instances survive the downgrade and upgrade procedures. This is done by
creating a cluster with the newer version, downgrading it to the older
one, and upgrading it to the newer one again. After every step, the
integrity of the cluster is checked by running various operations and
ensuring everything still works.</p>
</div>
</div>
<div class="section" id="design-and-implementation">
<h2><a class="toc-backref" href="#id6">Design and implementation</a><a class="headerlink" href="#design-and-implementation" title="Permalink to this headline">¶</a></h2>
<p>Although the previous examples have not been selected to show use cases
as diverse as possible, they still show a number of dissimilarities:</p>
<ul class="simple">
<li>Parallel installation vs sequential deployments</li>
<li>Comparing with reference version vs comparing consecutive versions</li>
<li>Examining result dumps vs trying a sequence of operations</li>
</ul>
<p>With the first two real use cases demonstrating such diversity, it does
not make sense to design multi-version test classes. Instead, the
programmability of buildbot’s configuration files can be leveraged to
implement each test as a separate builder with a custom sequence of
steps. The individual steps such as checking out a given or previous
version, or installing and removing Ganeti, will be provided as utility
functions for any test writer to use.</p>
<div class="section" id="current-state">
<h3><a class="toc-backref" href="#id7">Current state</a><a class="headerlink" href="#current-state" title="Permalink to this headline">¶</a></h3>
<p>An upgrade / downgrade test is a part of the QA suite as of commit
aa104b5e. The test and the corresponding buildbot changes are a very
good first step, both by showing that multi-version tests can be done,
and by providing utilities needed for builds of multiple branches.
Previously, the same folder was used as the base directory of any build,
and now a directory structure more accommodating to multiple builds is
in place.</p>
<p>The builder running the test has one flaw - regardless of the branch
submitted, it compares versions 2.10 and 2.11 (current master). This
behaviour is different from any of the other builders, which may
restrict the branches a test can be performed on, but do not
differentiate between them otherwise. While additional builders for
different versions pairs may be added, this is not a good long-term
solution.</p>
<p>The test can be improved by making it compare the current and the
previous version. As the buildbot has no notion of what a previous
version is, additional utilities to handle this logic will have to be
introduced.</p>
</div>
<div class="section" id="planned-changes">
<h3><a class="toc-backref" href="#id8">Planned changes</a><a class="headerlink" href="#planned-changes" title="Permalink to this headline">¶</a></h3>
<p>The upgrade / downgrade test should be generalized to work for any
version which can be downgraded from and upgraded to automatically,
meaning versions from 2.11 onwards. This will be made challenging by
the fact that the previous version has to be checked out by reading the
version of the currently checked out code, identifying the previous
version, and then making yet another checkout.</p>
<p>The major and minor version can be read from a Ganeti repository in
multiple ways. The two are present as constants defined in source files,
but due to refactorings shifting constants from the Python to the
Haskell side, their position varies across versions. A more reliable way
of fetching them is by examining the news file, as it obeys strict
formatting restrictions.</p>
<p>With the version found, a script that acts as a previous version
lookup table can be invoked. This script can be constructed dynamically
upon buildbot startup, and specified as a build step. The checkout
following it proceeds as expected.</p>
<p>The RAPI compatibility test should be added as a separate builder
afterwards. As the test requires additional comparison and proxy logic
to be used, it will be enabled only on 2.11 onwards, comparing the
versions to 2.6 - the reference version for the RAPI. Details on the
design of this test will be added in a separate document.</p>
</div>
</div>
<div class="section" id="potential-issues">
<h2><a class="toc-backref" href="#id9">Potential issues</a><a class="headerlink" href="#potential-issues" title="Permalink to this headline">¶</a></h2>
<p>While there are many advantages to having a single builder representing
a multi-version test, working on every branch, there is at least one
disadvantage: the need to define a base or reference version, which is
the only version that can be used to trigger the test, and the only one
on which code changes can be tried.</p>
<p>If an error is detected while running a test, and the issue lies with
a version other than the one used to invoke the test, the fix would
have to make it into the repository before the test could be tried
again.</p>
<p>For simple tests, the issue might be mitigated by running them locally.
However, the multi-version tests are more likely to be complicated than
not, and it could be difficult to reproduce a test by hand.</p>
<p>The situation can be made simpler by requiring that any multi-version
test can use only versions lower than the reference version. As errors
are more likely to be found in new rather than old code, this would at
least reduce the number of troublesome cases.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Multi-version tests</a><ul>
<li><a class="reference internal" href="#desired-improvements">Desired improvements</a></li>
<li><a class="reference internal" href="#required-use-cases">Required use cases</a><ul>
<li><a class="reference internal" href="#compatibility-tests">Compatibility tests</a></li>
<li><a class="reference internal" href="#upgrade-downgrade-tests">Upgrade / downgrade tests</a></li>
</ul>
</li>
<li><a class="reference internal" href="#design-and-implementation">Design and implementation</a><ul>
<li><a class="reference internal" href="#current-state">Current state</a></li>
<li><a class="reference internal" href="#planned-changes">Planned changes</a></li>
</ul>
</li>
<li><a class="reference internal" href="#potential-issues">Potential issues</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="design-multi-reloc.html"
title="previous chapter">Moving instances across node groups</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="design-network.html"
title="next chapter">Network management</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/design-multi-version-tests.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="design-network.html" title="Network management"
>next</a></li>
<li class="right" >
<a href="design-multi-reloc.html" title="Moving instances across node groups"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Ganeti 2.16.0~rc2 documentation</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2018, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Google Inc..
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.7.
</div>
</body>
</html>
|