/usr/share/doc/ganeti/html/design-internal-shutdown.html is in ganeti-doc 2.16.0~rc2-1build1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Detection of user-initiated shutdown from inside an instance — Ganeti 2.16.0~rc2 documentation</title>
<link rel="stylesheet" href="_static/style.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '2.16.0~rc2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="KVM daemon" href="design-kvmd.html" />
<link rel="prev" title="Hotplug" href="design-hotplug.html" />
</head>
<body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="design-kvmd.html" title="KVM daemon"
accesskey="N">next</a></li>
<li class="right" >
<a href="design-hotplug.html" title="Hotplug"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Ganeti 2.16.0~rc2 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="detection-of-user-initiated-shutdown-from-inside-an-instance">
<h1><a class="toc-backref" href="#id1">Detection of user-initiated shutdown from inside an instance</a><a class="headerlink" href="#detection-of-user-initiated-shutdown-from-inside-an-instance" title="Permalink to this headline">¶</a></h1>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Created:</th><td class="field-body">2011-Mar-23</td>
</tr>
<tr class="field-even field"><th class="field-name">Status:</th><td class="field-body">Partially Implemented</td>
</tr>
<tr class="field-odd field"><th class="field-name">Ganeti-Version:</th><td class="field-body">2.10.0, 2.11.0</td>
</tr>
</tbody>
</table>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#detection-of-user-initiated-shutdown-from-inside-an-instance" id="id1">Detection of user-initiated shutdown from inside an instance</a><ul>
<li><a class="reference internal" href="#current-state-and-shortcomings" id="id2">Current state and shortcomings</a></li>
<li><a class="reference internal" href="#proposed-changes" id="id3">Proposed changes</a></li>
<li><a class="reference internal" href="#implementation" id="id4">Implementation</a></li>
</ul>
</li>
</ul>
</div>
<p>This is a design document detailing the implementation of a way for Ganeti to
detect whether an instance marked as up but not running was shutdown gracefully
by the user from inside the instance itself.</p>
<div class="section" id="current-state-and-shortcomings">
<h2><a class="toc-backref" href="#id2">Current state and shortcomings</a><a class="headerlink" href="#current-state-and-shortcomings" title="Permalink to this headline">¶</a></h2>
<p>Ganeti keeps track of the desired status of instances in order to be able to
take proper action (e.g.: reboot) on the instances that happen to crash.
Currently, the only way to properly shut down an instance is through Ganeti’s
own commands, which can be used to mark an instance as <code class="docutils literal"><span class="pre">ADMIN_down</span></code>.</p>
<p>If a user shuts down an instance from inside, through the proper command of the
operating system it is running, the instance will be shutdown gracefully, but
Ganeti is not aware of that: the desired status of the instance will still be
marked as <code class="docutils literal"><span class="pre">running</span></code>, so when the watcher realises that the instance is down,
it will restart it. This behaviour is usually not what the user expects.</p>
</div>
<div class="section" id="proposed-changes">
<h2><a class="toc-backref" href="#id3">Proposed changes</a><a class="headerlink" href="#proposed-changes" title="Permalink to this headline">¶</a></h2>
<p>We propose to modify Ganeti in such a way that it will detect when an instance
was shutdown as a result of an explicit request from the user. When such a
situation is detected, instead of presenting an error as it happens now, either
the state of the instance will be set to <code class="docutils literal"><span class="pre">ADMIN_down</span></code>, or the instance will be
automatically rebooted, depending on an instance-specific configuration value.
The default behavior in case no such parameter is found will be to follow the
apparent will of the user, and setting to <code class="docutils literal"><span class="pre">ADMIN_down</span></code> an instance that was
shut down correctly from inside.</p>
<p>The rest of this design document details the implementation of instance shutdown
detection for Xen. The KVM implementation is detailed in <a class="reference internal" href="design-kvmd.html"><span class="doc">KVM daemon</span></a>.</p>
</div>
<div class="section" id="implementation">
<h2><a class="toc-backref" href="#id4">Implementation</a><a class="headerlink" href="#implementation" title="Permalink to this headline">¶</a></h2>
<p>Xen knows why a domain is being shut down (a crash or an explicit shutdown
or poweroff request), but such information is not usually readily available
externally, because all such cases lead to the virtual machine being destroyed
immediately after the event is detected.</p>
<p>Still, Xen allows the instance configuration file to define what action to be
taken in all those cases through the <code class="docutils literal"><span class="pre">on_poweroff</span></code>, <code class="docutils literal"><span class="pre">on_shutdown</span></code> and
<code class="docutils literal"><span class="pre">on_crash</span></code> variables. By setting them to <code class="docutils literal"><span class="pre">preserve</span></code>, Xen will avoid
destroying the domains automatically.</p>
<p>When the domain is not destroyed, it can be viewed by using <code class="docutils literal"><span class="pre">xm</span> <span class="pre">list</span></code> (or <code class="docutils literal"><span class="pre">xl</span>
<span class="pre">list</span></code> in newer Xen versions), and the <code class="docutils literal"><span class="pre">State</span></code> field of the output will
provide useful information.</p>
<p>If the state is <code class="docutils literal"><span class="pre">----c-</span></code> it means the instance has crashed.</p>
<p>If the state is <code class="docutils literal"><span class="pre">---s--</span></code> it means the instance was properly shutdown.</p>
<p>If the instance was properly shutdown and it is still marked as <code class="docutils literal"><span class="pre">running</span></code> by
Ganeti, it means that it was shutdown from inside by the user, and the Ganeti
status of the instance needs to be changed to <code class="docutils literal"><span class="pre">ADMIN_down</span></code>.</p>
<p>This will be done at regular intervals by the group watcher, just before
deciding which instances to reboot.</p>
<p>On top of that, at the same time, the watcher will also need to issue <code class="docutils literal"><span class="pre">xm</span>
<span class="pre">destroy</span></code> commands for all the domains that are in a crashed or shutdown state,
since this will not be done automatically by Xen anymore because of the
<code class="docutils literal"><span class="pre">preserve</span></code> setting in their config files.</p>
<p>This behavior will be limited to the domains shut down from inside, because it
will actually keep the resources of the domain busy until the watcher will do
the cleaning job (that, with the default setting, is up to every 5 minutes).
Still, this is considered acceptable, because it is not frequent for a domain to
be shut down this way. The cleanup function will be also run automatically just
before performing any job that requires resources to be available (such as when
creating a new instance), in order to ensure that the new resource allocation
happens starting from a clean state. Functionalities that only query the state
of instances will not run the cleanup function.</p>
<p>The cleanup operation includes both node-specific operations (the actual
destruction of the stopped domains) and configuration changes, to be performed
on the master node (marking as offline an instance that was shut down
internally). The watcher, on the master node, will fetch the list of instances
that have been shutdown from inside (recognizable by their <code class="docutils literal"><span class="pre">oper_state</span></code>
as described below). It will then submit a series of <code class="docutils literal"><span class="pre">InstanceShutdown</span></code> jobs
that will mark such instances as <code class="docutils literal"><span class="pre">ADMIN_down</span></code> and clean them up (after
the functionality of <code class="docutils literal"><span class="pre">InstanceShutdown</span></code> will have been extended as specified
in the rest of this design document).</p>
<p>LUs performing operations other than an explicit cleanup will have to be
modified to perform the cleanup as well, either by submitting a job to perform
the cleanup (to be completed before actually performing the task at hand) or by
explicitly performing the cleanup themselves through the RPC calls.</p>
<div class="section" id="other-required-changes">
<h3>Other required changes<a class="headerlink" href="#other-required-changes" title="Permalink to this headline">¶</a></h3>
<p>The implementation of this design document will require some commands to be
changed in order to cope with the new shutdown procedure.</p>
<p>With the default shutdown action in Xen set to <code class="docutils literal"><span class="pre">preserve</span></code>, the Ganeti
command for shutting down instances would leave them in a shutdown but
preserved state. Therefore, it will have to be changed in such a way to
immediately perform the cleanup of the instance after verifying its correct
shutdown. Also, it will correctly deal with instances that have been shutdown
from inside but are still active according to Ganeti, by detecting this
situation, destroying the instance and carrying out the rest of the Ganeti
shutdown procedure as usual.</p>
<p>The <code class="docutils literal"><span class="pre">gnt-instance</span> <span class="pre">list</span></code> command will need to be able to handle the situation
where an instance was shutdown internally but not yet cleaned up. The
<code class="docutils literal"><span class="pre">admin_state</span></code> field will maintain the current meaning unchanged. The
<code class="docutils literal"><span class="pre">oper_state</span></code> field will get a new possible state, <code class="docutils literal"><span class="pre">S</span></code>, meaning that the
instance was shutdown internally.</p>
<p>The <code class="docutils literal"><span class="pre">gnt-instance</span> <span class="pre">info</span></code> command <code class="docutils literal"><span class="pre">State</span></code> field, in such case, will show a
message stating that the instance was supposed to be run but was shut down
internally.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Detection of user-initiated shutdown from inside an instance</a><ul>
<li><a class="reference internal" href="#current-state-and-shortcomings">Current state and shortcomings</a></li>
<li><a class="reference internal" href="#proposed-changes">Proposed changes</a></li>
<li><a class="reference internal" href="#implementation">Implementation</a><ul>
<li><a class="reference internal" href="#other-required-changes">Other required changes</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="design-hotplug.html"
title="previous chapter">Hotplug</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="design-kvmd.html"
title="next chapter">KVM daemon</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/design-internal-shutdown.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="design-kvmd.html" title="KVM daemon"
>next</a></li>
<li class="right" >
<a href="design-hotplug.html" title="Hotplug"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Ganeti 2.16.0~rc2 documentation</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2018, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Google Inc..
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.7.
</div>
</body>
</html>
|