/usr/share/doc/ganeti/html/design-ifdown.html is in ganeti-doc 2.16.0~rc2-1build1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Design for adding ifdown script to KVM — Ganeti 2.16.0~rc2 documentation</title>
<link rel="stylesheet" href="_static/style.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '2.16.0~rc2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Instance Reservations" href="design-reservations.html" />
<link rel="prev" title="Huge Pages Support for Ganeti" href="design-hugepages-support.html" />
</head>
<body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="design-reservations.html" title="Instance Reservations"
accesskey="N">next</a></li>
<li class="right" >
<a href="design-hugepages-support.html" title="Huge Pages Support for Ganeti"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Ganeti 2.16.0~rc2 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="design-draft.html" accesskey="U">Design document drafts</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="design-for-adding-ifdown-script-to-kvm">
<h1><a class="toc-backref" href="#id1">Design for adding ifdown script to KVM</a><a class="headerlink" href="#design-for-adding-ifdown-script-to-kvm" title="Permalink to this headline">¶</a></h1>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Created:</th><td class="field-body">2014-Jun-23</td>
</tr>
<tr class="field-even field"><th class="field-name">Status:</th><td class="field-body">Draft</td>
</tr>
</tbody>
</table>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#design-for-adding-ifdown-script-to-kvm" id="id1">Design for adding ifdown script to KVM</a><ul>
<li><a class="reference internal" href="#current-state-and-shortcomings" id="id2">Current state and shortcomings</a></li>
<li><a class="reference internal" href="#proposed-changes" id="id3">Proposed Changes</a></li>
<li><a class="reference internal" href="#implementation-details" id="id4">Implementation Details</a><ul>
<li><a class="reference internal" href="#configuration-changes" id="id5">Configuration Changes</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<p>This is a design document about adding support for an ifdown script responsible
for deconfiguring network devices and cleanup changes made by the ifup script. The
first implementation will target KVM but it could be ported to Xen as well
especially when hotplug gets implemented.</p>
<div class="section" id="current-state-and-shortcomings">
<h2><a class="toc-backref" href="#id2">Current state and shortcomings</a><a class="headerlink" href="#current-state-and-shortcomings" title="Permalink to this headline">¶</a></h2>
<p>Currently, KVM before instance startup, instance migration and NIC hotplug, it
creates a tap and invokes explicitly the kvm-ifup script with the relevant
environment (INTERFACE, MAC, IP, MODE, LINK, TAGS, and all the network info if
any; NETWORK_SUBNET, NETWORK_TAGS, etc).</p>
<p>For Xen we have the <cite>vif-ganeti</cite> script (associated with vif-script hypervisor
parameter). The main difference is that Xen calls it by itself by passing it as
an extra option in the configuration file.</p>
<p>This ifup script can do several things; bridge a tap to a bridge, add ip rules,
update a external DNS or DHCP server, enable proxy ARP or proxy NDP, issue
openvswitch commands, etc. In general we can divide those actions in two
categories:</p>
<ol class="arabic simple">
<li>Commands that change the state of the host</li>
<li>Commands that change the state of external components.</li>
</ol>
<p>Currently those changes do not get cleaned up or modified upon instance
shutdown, remove, migrate, or NIC hot-unplug. Thus we have stale entries in
hosts and most important might have stale/invalid configuration on external
components like routers that could affect connectivity.</p>
<p>A workaround could be hooks but:</p>
<p>1) During migrate hooks the environment is the one held in config data
and not in runtime files. The NIC configuration might have changed on
master but not on the running KVM process (unless hotplug is used).
Plus the NIC order in config data might not be the same one on the KVM
process.</p>
<p>2) On instance modification, changes are not available on hooks. With
other words we do not know the configuration before and after modification.</p>
<p>Since Ganeti is the orchestrator and is the one who explicitly configures
host devices (tap, vif) it should be the one responsible for cleanup/
deconfiguration. Especially on a SDN approach this kind of script might
be useful to cleanup flows in the cluster in order to ensure correct paths
without ping pongs between hosts or connectivity loss for the instance.</p>
</div>
<div class="section" id="proposed-changes">
<h2><a class="toc-backref" href="#id3">Proposed Changes</a><a class="headerlink" href="#proposed-changes" title="Permalink to this headline">¶</a></h2>
<p>We add an new script, kvm-ifdown that is explicitly invoked after:</p>
<ol class="arabic simple">
<li>instance shutdown on primary node</li>
<li>successful instance migration on source node</li>
<li>failed instance migration on target node</li>
<li>successful NIC hot-remove on primary node</li>
</ol>
<p>If an administrator’s custom ifdown script exists (e.g. <cite>kvm-ifdown-custom</cite>),
the <cite>kvm-ifdown</cite> script executes that script, as happens with <cite>kvm-ifup</cite>.</p>
<p>Along with that change we should rename custom ifup script from
<cite>kvm-vif-bridge</cite> (which does not make any sense) to <cite>kvm-ifup-custom</cite>.</p>
<p>In contrary to <cite>kvm-ifup</cite>, one cannot rely on <cite>kvm-ifdown</cite> script to be
called. A node might die just after a successful migration or after an
instance shutdown. In that case, all “undo” operations will not be invoked.
Thus, this script should work “on a best effort basis” and the network
should not rely on the script being called or being successful. Additionally
it should modify <em>only</em> the node local dynamic configs (routes, arp entries,
SDN, firewalls, etc.), whereas static ones (DNS, DHCP, etc.) should be modified
via hooks.</p>
</div>
<div class="section" id="implementation-details">
<h2><a class="toc-backref" href="#id4">Implementation Details</a><a class="headerlink" href="#implementation-details" title="Permalink to this headline">¶</a></h2>
<ol class="arabic simple">
<li>Where to get the NIC info?</li>
</ol>
<p>We cannot account on config data since it might have changed. So the only
place we keep our valid data is inside the runtime file. During instance
modifications (NIC hot-remove, hot-modify) we have the NIC object from
the RPC. We take its UUID and search for the corresponding entry in the
runtime file to get further info. After instance shutdown and migration
we just take all NICs from the runtime file and invoke the ifdown script
for each one</p>
<ol class="arabic simple" start="2">
<li>Where to find the corresponding TAP?</li>
</ol>
<p>Currently TAP names are kept under
/var/run/ganeti/kvm-hypervisor/nics/<instance>/<nic_index>.
This is not enough. As told above a NIC’s index might change during instance’s
life. An example will make things clear:</p>
<ul class="simple">
<li>The admin starts an instance with three NICs.</li>
<li>The admin removes the second without hotplug.</li>
<li>The admin removes the first with hotplug.</li>
</ul>
<p>The index that will arrive with the RPC will be 1 and if we read the relevant
NIC file we will get the tap of the NIC that has been removed on second
step but is still existing in the KVM process.</p>
<p>So upon TAP creation we write another file with the same info but named
after the NIC’s UUID. The one named after its index can be left
for compatibility (Ganeti does not use it; external tools might)
Obviously this info will not be available for old instances in the cluster.
The ifdown script should be aware of this corner case.</p>
<ol class="arabic simple" start="3">
<li>What should we cleanup/deconfigure?</li>
</ol>
<p>Upon NIC hot-remove we obviously want to wipe everything. But on instance
migration we don’t want to reset external configuration like DNS. So we choose
to pass an extra positional argument to the ifdown script (it already has the
TAP name) that will reflect the context it was invoked with. Please note that
de-configuration of external components is not encouraged and should be
done via hooks. Still we could easily support it via this extra argument.</p>
<ol class="arabic simple" start="4">
<li>What will be the script environment?</li>
</ol>
<p>In general the same environment passed to ifup script. Except instance’s
tags. Those are the only info not kept in runtime file and it can
change between ifup and ifdown script execution. The ifdown
script must be aware of it and should cleanup everything that ifup script
might setup depending on instance tags (e.g. firewalls, etc)</p>
<div class="section" id="configuration-changes">
<h3><a class="toc-backref" href="#id5">Configuration Changes</a><a class="headerlink" href="#configuration-changes" title="Permalink to this headline">¶</a></h3>
<ol class="arabic simple">
<li>The <cite>kvm-ifdown</cite> script will be an extra file installed under the same dir
<cite>kvm-ifup</cite> resides. We could have a single script (and symbolic links to it)
that shares the same code, where a second positional argument or an extra
environment variable would define if we are bringing the interface up or
down. Still this is not the best practice since it is not equivalent
with how KVM uses <cite>script</cite> and <cite>downscript</cite> in the <cite>netdev</cite> option; scripts
are different files that get the tap name as positional argument. Of course
common code will go in <cite>net-common</cite> so that it can be sourced from either
Xen or KVM specific scripts.</li>
<li>An extra file written upon TAP creation named after the NIC’s UUID and
including the TAP’s name. Since this should be the correct file to keep
backwards compatibility we create a symbolic link named after the NIC’s
index and pointing to this new file.</li>
</ol>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Design for adding ifdown script to KVM</a><ul>
<li><a class="reference internal" href="#current-state-and-shortcomings">Current state and shortcomings</a></li>
<li><a class="reference internal" href="#proposed-changes">Proposed Changes</a></li>
<li><a class="reference internal" href="#implementation-details">Implementation Details</a><ul>
<li><a class="reference internal" href="#configuration-changes">Configuration Changes</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="design-hugepages-support.html"
title="previous chapter">Huge Pages Support for Ganeti</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="design-reservations.html"
title="next chapter">Instance Reservations</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/design-ifdown.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="design-reservations.html" title="Instance Reservations"
>next</a></li>
<li class="right" >
<a href="design-hugepages-support.html" title="Huge Pages Support for Ganeti"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Ganeti 2.16.0~rc2 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="design-draft.html" >Design document drafts</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2018, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Google Inc..
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.7.
</div>
</body>
</html>
|