/usr/share/doc/pacemaker/crm

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
    "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
<meta name="generator" content="AsciiDoc 8.6.10" />
<title>Fencing and Stonith</title>
<style type="text/css">
/* Shared CSS for AsciiDoc xhtml11 and html5 backends */

/* Default font. */
body {
  font-family: Georgia,serif;
}

/* Title font. */
h1, h2, h3, h4, h5, h6,
div.title, caption.title,
thead, p.table.header,
#toctitle,
#author, #revnumber, #revdate, #revremark,
#footer {
  font-family: Arial,Helvetica,sans-serif;
}

body {
  margin: 1em 5% 1em 5%;
}

a {
  color: blue;
  text-decoration: underline;
}
a:visited {
  color: fuchsia;
}

em {
  font-style: italic;
  color: navy;
}

strong {
  font-weight: bold;
  color: #083194;
}

h1, h2, h3, h4, h5, h6 {
  color: #527bbd;
  margin-top: 1.2em;
  margin-bottom: 0.5em;
  line-height: 1.3;
}

h1, h2, h3 {
  border-bottom: 2px solid silver;
}
h2 {
  padding-top: 0.5em;
}
h3 {
  float: left;
}
h3 + * {
  clear: left;
}
h5 {
  font-size: 1.0em;
}

div.sectionbody {
  margin-left: 0;
}

hr {
  border: 1px solid silver;
}

p {
  margin-top: 0.5em;
  margin-bottom: 0.5em;
}

ul, ol, li > p {
  margin-top: 0;
}
ul > li     { color: #aaa; }
ul > li > * { color: black; }

.monospaced, code, pre {
  font-family: "Courier New", Courier, monospace;
  font-size: inherit;
  color: navy;
  padding: 0;
  margin: 0;
}
pre {
  white-space: pre-wrap;
}

#author {
  color: #527bbd;
  font-weight: bold;
  font-size: 1.1em;
}
#email {
}
#revnumber, #revdate, #revremark {
}

#footer {
  font-size: small;
  border-top: 2px solid silver;
  padding-top: 0.5em;
  margin-top: 4.0em;
}
#footer-text {
  float: left;
  padding-bottom: 0.5em;
}
#footer-badges {
  float: right;
  padding-bottom: 0.5em;
}

#preamble {
  margin-top: 1.5em;
  margin-bottom: 1.5em;
}
div.imageblock, div.exampleblock, div.verseblock,
div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
div.admonitionblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
div.admonitionblock {
  margin-top: 2.0em;
  margin-bottom: 2.0em;
  margin-right: 10%;
  color: #606060;
}

div.content { /* Block element content. */
  padding: 0;
}

/* Block element titles. */
div.title, caption.title {
  color: #527bbd;
  font-weight: bold;
  text-align: left;
  margin-top: 1.0em;
  margin-bottom: 0.5em;
}
div.title + * {
  margin-top: 0;
}

td div.title:first-child {
  margin-top: 0.0em;
}
div.content div.title:first-child {
  margin-top: 0.0em;
}
div.content + div.title {
  margin-top: 0.0em;
}

div.sidebarblock > div.content {
  background: #ffffee;
  border: 1px solid #dddddd;
  border-left: 4px solid #f0f0f0;
  padding: 0.5em;
}

div.listingblock > div.content {
  border: 1px solid #dddddd;
  border-left: 5px solid #f0f0f0;
  background: #f8f8f8;
  padding: 0.5em;
}

div.quoteblock, div.verseblock {
  padding-left: 1.0em;
  margin-left: 1.0em;
  margin-right: 10%;
  border-left: 5px solid #f0f0f0;
  color: #888;
}

div.quoteblock > div.attribution {
  padding-top: 0.5em;
  text-align: right;
}

div.verseblock > pre.content {
  font-family: inherit;
  font-size: inherit;
}
div.verseblock > div.attribution {
  padding-top: 0.75em;
  text-align: left;
}
/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
div.verseblock + div.attribution {
  text-align: left;
}

div.admonitionblock .icon {
  vertical-align: top;
  font-size: 1.1em;
  font-weight: bold;
  text-decoration: underline;
  color: #527bbd;
  padding-right: 0.5em;
}
div.admonitionblock td.content {
  padding-left: 0.5em;
  border-left: 3px solid #dddddd;
}

div.exampleblock > div.content {
  border-left: 3px solid #dddddd;
  padding-left: 0.5em;
}

div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; vertical-align: text-bottom; }
a.image:visited { color: white; }

dl {
  margin-top: 0.8em;
  margin-bottom: 0.8em;
}
dt {
  margin-top: 0.5em;
  margin-bottom: 0;
  font-style: normal;
  color: navy;
}
dd > *:first-child {
  margin-top: 0.1em;
}

ul, ol {
    list-style-position: outside;
}
ol.arabic {
  list-style-type: decimal;
}
ol.loweralpha {
  list-style-type: lower-alpha;
}
ol.upperalpha {
  list-style-type: upper-alpha;
}
ol.lowerroman {
  list-style-type: lower-roman;
}
ol.upperroman {
  list-style-type: upper-roman;
}

div.compact ul, div.compact ol,
div.compact p, div.compact p,
div.compact div, div.compact div {
  margin-top: 0.1em;
  margin-bottom: 0.1em;
}

tfoot {
  font-weight: bold;
}
td > div.verse {
  white-space: pre;
}

div.hdlist {
  margin-top: 0.8em;
  margin-bottom: 0.8em;
}
div.hdlist tr {
  padding-bottom: 15px;
}
dt.hdlist1.strong, td.hdlist1.strong {
  font-weight: bold;
}
td.hdlist1 {
  vertical-align: top;
  font-style: normal;
  padding-right: 0.8em;
  color: navy;
}
td.hdlist2 {
  vertical-align: top;
}
div.hdlist.compact tr {
  margin: 0;
  padding-bottom: 0;
}

.comment {
  background: yellow;
}

.footnote, .footnoteref {
  font-size: 0.8em;
}

span.footnote, span.footnoteref {
  vertical-align: super;
}

#footnotes {
  margin: 20px 0 20px 0;
  padding: 7px 0 0 0;
}

#footnotes div.footnote {
  margin: 0 0 5px 0;
}

#footnotes hr {
  border: none;
  border-top: 1px solid silver;
  height: 1px;
  text-align: left;
  margin-left: 0;
  width: 20%;
  min-width: 100px;
}

div.colist td {
  padding-right: 0.5em;
  padding-bottom: 0.3em;
  vertical-align: top;
}
div.colist td img {
  margin-top: 0.3em;
}

@media print {
  #footer-badges { display: none; }
}

#toc {
  margin-bottom: 2.5em;
}

#toctitle {
  color: #527bbd;
  font-size: 1.1em;
  font-weight: bold;
  margin-top: 1.0em;
  margin-bottom: 0.1em;
}

div.toclevel0, div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
  margin-top: 0;
  margin-bottom: 0;
}
div.toclevel2 {
  margin-left: 2em;
  font-size: 0.9em;
}
div.toclevel3 {
  margin-left: 4em;
  font-size: 0.9em;
}
div.toclevel4 {
  margin-left: 6em;
  font-size: 0.9em;
}

span.aqua { color: aqua; }
span.black { color: black; }
span.blue { color: blue; }
span.fuchsia { color: fuchsia; }
span.gray { color: gray; }
span.green { color: green; }
span.lime { color: lime; }
span.maroon { color: maroon; }
span.navy { color: navy; }
span.olive { color: olive; }
span.purple { color: purple; }
span.red { color: red; }
span.silver { color: silver; }
span.teal { color: teal; }
span.white { color: white; }
span.yellow { color: yellow; }

span.aqua-background { background: aqua; }
span.black-background { background: black; }
span.blue-background { background: blue; }
span.fuchsia-background { background: fuchsia; }
span.gray-background { background: gray; }
span.green-background { background: green; }
span.lime-background { background: lime; }
span.maroon-background { background: maroon; }
span.navy-background { background: navy; }
span.olive-background { background: olive; }
span.purple-background { background: purple; }
span.red-background { background: red; }
span.silver-background { background: silver; }
span.teal-background { background: teal; }
span.white-background { background: white; }
span.yellow-background { background: yellow; }

span.big { font-size: 2em; }
span.small { font-size: 0.6em; }

span.underline { text-decoration: underline; }
span.overline { text-decoration: overline; }
span.line-through { text-decoration: line-through; }

div.unbreakable { page-break-inside: avoid; }


/*
 * xhtml11 specific
 *
 * */

div.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
div.tableblock > table {
  border: 3px solid #527bbd;
}
thead, p.table.header {
  font-weight: bold;
  color: #527bbd;
}
p.table {
  margin-top: 0;
}
/* Because the table frame attribute is overriden by CSS in most browsers. */
div.tableblock > table[frame="void"] {
  border-style: none;
}
div.tableblock > table[frame="hsides"] {
  border-left-style: none;
  border-right-style: none;
}
div.tableblock > table[frame="vsides"] {
  border-top-style: none;
  border-bottom-style: none;
}


/*
 * html5 specific
 *
 * */

table.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
thead, p.tableblock.header {
  font-weight: bold;
  color: #527bbd;
}
p.tableblock {
  margin-top: 0;
}
table.tableblock {
  border-width: 3px;
  border-spacing: 0px;
  border-style: solid;
  border-color: #527bbd;
  border-collapse: collapse;
}
th.tableblock, td.tableblock {
  border-width: 1px;
  padding: 4px;
  border-style: solid;
  border-color: #527bbd;
}

table.tableblock.frame-topbot {
  border-left-style: hidden;
  border-right-style: hidden;
}
table.tableblock.frame-sides {
  border-top-style: hidden;
  border-bottom-style: hidden;
}
table.tableblock.frame-none {
  border-style: hidden;
}

th.tableblock.halign-left, td.tableblock.halign-left {
  text-align: left;
}
th.tableblock.halign-center, td.tableblock.halign-center {
  text-align: center;
}
th.tableblock.halign-right, td.tableblock.halign-right {
  text-align: right;
}

th.tableblock.valign-top, td.tableblock.valign-top {
  vertical-align: top;
}
th.tableblock.valign-middle, td.tableblock.valign-middle {
  vertical-align: middle;
}
th.tableblock.valign-bottom, td.tableblock.valign-bottom {
  vertical-align: bottom;
}


/*
 * manpage specific
 *
 * */

body.manpage h1 {
  padding-top: 0.5em;
  padding-bottom: 0.5em;
  border-top: 2px solid silver;
  border-bottom: 2px solid silver;
}
body.manpage h2 {
  border-style: none;
}
body.manpage div.sectionbody {
  margin-left: 3em;
}

@media print {
  body.manpage div#toc { display: none; }
}


</style>
<script type="text/javascript">
/*<![CDATA[*/
var asciidoc = {  // Namespace.

/////////////////////////////////////////////////////////////////////
// Table Of Contents generator
/////////////////////////////////////////////////////////////////////

/* Author: Mihai Bazon, September 2002
 * http://students.infoiasi.ro/~mishoo
 *
 * Table Of Content generator
 * Version: 0.4
 *
 * Feel free to use this script under the terms of the GNU General Public
 * License, as long as you do not remove or alter this notice.
 */

 /* modified by Troy D. Hanson, September 2006. License: GPL */
 /* modified by Stuart Rackham, 2006, 2009. License: GPL */

// toclevels = 1..4.
toc: function (toclevels) {

  function getText(el) {
    var text = "";
    for (var i = el.firstChild; i != null; i = i.nextSibling) {
      if (i.nodeType == 3 /* Node.TEXT_NODE */) // IE doesn't speak constants.
        text += i.data;
      else if (i.firstChild != null)
        text += getText(i);
    }
    return text;
  }

  function TocEntry(el, text, toclevel) {
    this.element = el;
    this.text = text;
    this.toclevel = toclevel;
  }

  function tocEntries(el, toclevels) {
    var result = new Array;
    var re = new RegExp('[hH]([1-'+(toclevels+1)+'])');
    // Function that scans the DOM tree for header elements (the DOM2
    // nodeIterator API would be a better technique but not supported by all
    // browsers).
    var iterate = function (el) {
      for (var i = el.firstChild; i != null; i = i.nextSibling) {
        if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
          var mo = re.exec(i.tagName);
          if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
            result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
          }
          iterate(i);
        }
      }
    }
    iterate(el);
    return result;
  }

  var toc = document.getElementById("toc");
  if (!toc) {
    return;
  }

  // Delete existing TOC entries in case we're reloading the TOC.
  var tocEntriesToRemove = [];
  var i;
  for (i = 0; i < toc.childNodes.length; i++) {
    var entry = toc.childNodes[i];
    if (entry.nodeName.toLowerCase() == 'div'
     && entry.getAttribute("class")
     && entry.getAttribute("class").match(/^toclevel/))
      tocEntriesToRemove.push(entry);
  }
  for (i = 0; i < tocEntriesToRemove.length; i++) {
    toc.removeChild(tocEntriesToRemove[i]);
  }

  // Rebuild TOC entries.
  var entries = tocEntries(document.getElementById("content"), toclevels);
  for (var i = 0; i < entries.length; ++i) {
    var entry = entries[i];
    if (entry.element.id == "")
      entry.element.id = "_toc_" + i;
    var a = document.createElement("a");
    a.href = "#" + entry.element.id;
    a.appendChild(document.createTextNode(entry.text));
    var div = document.createElement("div");
    div.appendChild(a);
    div.className = "toclevel" + entry.toclevel;
    toc.appendChild(div);
  }
  if (entries.length == 0)
    toc.parentNode.removeChild(toc);
},


/////////////////////////////////////////////////////////////////////
// Footnotes generator
/////////////////////////////////////////////////////////////////////

/* Based on footnote generation code from:
 * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
 */

footnotes: function () {
  // Delete existing footnote entries in case we're reloading the footnodes.
  var i;
  var noteholder = document.getElementById("footnotes");
  if (!noteholder) {
    return;
  }
  var entriesToRemove = [];
  for (i = 0; i < noteholder.childNodes.length; i++) {
    var entry = noteholder.childNodes[i];
    if (entry.nodeName.toLowerCase() == 'div' && entry.getAttribute("class") == "footnote")
      entriesToRemove.push(entry);
  }
  for (i = 0; i < entriesToRemove.length; i++) {
    noteholder.removeChild(entriesToRemove[i]);
  }

  // Rebuild footnote entries.
  var cont = document.getElementById("content");
  var spans = cont.getElementsByTagName("span");
  var refs = {};
  var n = 0;
  for (i=0; i<spans.length; i++) {
    if (spans[i].className == "footnote") {
      n++;
      var note = spans[i].getAttribute("data-note");
      if (!note) {
        // Use [\s\S] in place of . so multi-line matches work.
        // Because JavaScript has no s (dotall) regex flag.
        note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
        spans[i].innerHTML =
          "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
          "' title='View footnote' class='footnote'>" + n + "</a>]";
        spans[i].setAttribute("data-note", note);
      }
      noteholder.innerHTML +=
        "<div class='footnote' id='_footnote_" + n + "'>" +
        "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
        n + "</a>. " + note + "</div>";
      var id =spans[i].getAttribute("id");
      if (id != null) refs["#"+id] = n;
    }
  }
  if (n == 0)
    noteholder.parentNode.removeChild(noteholder);
  else {
    // Process footnoterefs.
    for (i=0; i<spans.length; i++) {
      if (spans[i].className == "footnoteref") {
        var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
        href = href.match(/#.*/)[0];  // Because IE return full URL.
        n = refs[href];
        spans[i].innerHTML =
          "[<a href='#_footnote_" + n +
          "' title='View footnote' class='footnote'>" + n + "</a>]";
      }
    }
  }
},

install: function(toclevels) {
  var timerId;

  function reinstall() {
    asciidoc.footnotes();
    if (toclevels) {
      asciidoc.toc(toclevels);
    }
  }

  function reinstallAndRemoveTimer() {
    clearInterval(timerId);
    reinstall();
  }

  timerId = setInterval(reinstall, 500);
  if (document.addEventListener)
    document.addEventListener("DOMContentLoaded", reinstallAndRemoveTimer, false);
  else
    window.onload = reinstallAndRemoveTimer;
}

}
asciidoc.install();
/*]]>*/
</script>
</head>
<body class="article">
<div id="header">
<h1>Fencing and Stonith</h1>
<span id="author">Dejan Muhamedagic</span><br />
<span id="email"><code>&lt;<a href="mailto:dejan@suse.de">dejan@suse.de</a>&gt;</code></span><br />
<span id="revdate">v0.9</span>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph"><p>Fencing is a very important concept in computer clusters for HA
(High Availability). Unfortunately, given that fencing does not
offer a visible service to users, it is often neglected.</p></div>
<div class="paragraph"><p>Fencing may be defined as a method to bring an HA cluster to a
known state. But, what is a "cluster state" after all? To answer
that question we have to see what is in the cluster.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_introduction_to_ha_clusters">Introduction to HA clusters</h2>
<div class="sectionbody">
<div class="paragraph"><p>Any computer cluster may be loosely defined as a collection of
cooperating computers or nodes. Nodes talk to each other over
communication channels, which are typically standard network
connections, such as Ethernet.</p></div>
<div class="paragraph"><p>The main purpose of an HA cluster is to manage user services.
Typical examples of user services are an Apache web server or,
say, a MySQL database. From the user&#8217;s point of view, the
services do some specific and hopefully useful work when ordered
to do so. To the cluster, however, they are just things which may
be started or stopped. This distinction is important, because the
nature of the service is irrelevant to the cluster. In the
cluster lingo, the user services are known as resources.</p></div>
<div class="paragraph"><p>Every resource has a state attached, for instance: "resource r1
is started on node1". In an HA cluster, such state implies that
"resource r1 is stopped on all nodes but node1", because an HA
cluster must make sure that every resource may be started on at
most one node.</p></div>
<div class="paragraph"><p>A collection of resource states and node states is the cluster
state.</p></div>
<div class="paragraph"><p>Every node must report every change that happens to resources.
This may happen only for the running resources, because a node
should not start resources unless told so by somebody. That
somebody is the Cluster Resource Manager (CRM) in our case.</p></div>
<div class="paragraph"><p>So far so good. But what if, for whatever reason, we cannot
establish with certainty a state of some node or resource? This
is where fencing comes in. With fencing, even when the cluster
doesn&#8217;t know what is happening on some node, we can make sure
that that node doesn&#8217;t run any or certain important resources.</p></div>
<div class="paragraph"><p>If you wonder how this can happen, there may be many risks
involved with computing: reckless people, power outages, natural
disasters, rodents, thieves, software bugs, just to name a few.
We are sure that at least a few times your computer failed
unpredictably.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_fencing">Fencing</h2>
<div class="sectionbody">
<div class="paragraph"><p>There are two kinds of fencing: resource level and node level.</p></div>
<div class="paragraph"><p>Using the resource level fencing the cluster can make sure that
a node cannot access one or more resources. One typical example
is a SAN, where a fencing operation changes rules on a SAN switch
to deny access from a node.</p></div>
<div class="paragraph"><p>The resource level fencing may be achieved using normal resources
on which the resource we want to protect would depend. Such a
resource would simply refuse to start on this node and therefore
resources which depend on it will be unrunnable on the same node
as well.</p></div>
<div class="paragraph"><p>The node level fencing makes sure that a node does not run any
resources at all. This is usually done in a very simple, yet
brutal way: the node is simply reset using a power switch. This
may ultimately be necessary because the node may not be
responsive at all.</p></div>
<div class="paragraph"><p>The node level fencing is our primary subject below.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_node_level_fencing_devices">Node level fencing devices</h2>
<div class="sectionbody">
<div class="paragraph"><p>Before we get into the configuration details, you need to pick a
fencing device for the node level fencing. There are quite a few
to choose from. If you want to see the list of stonith devices
which are supported just run:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>stonith -L</code></pre>
</div></div>
<div class="paragraph"><p>Stonith devices may be classified into five categories:</p></div>
<div class="ulist"><ul>
<li>
<p>
UPS (Uninterruptible Power Supply)
</p>
</li>
<li>
<p>
PDU (Power Distribution Unit)
</p>
</li>
<li>
<p>
Blade power control devices
</p>
</li>
<li>
<p>
Lights-out devices
</p>
</li>
<li>
<p>
Testing devices
</p>
</li>
</ul></div>
<div class="paragraph"><p>The choice depends mainly on your budget and the kind of
hardware. For instance, if you&#8217;re running a cluster on a set of
blades, then the power control device in the blade enclosure is
the only candidate for fencing. Of course, this device must be
capable of managing single blade computers.</p></div>
<div class="paragraph"><p>The lights-out devices (IBM RSA, HP iLO, Dell DRAC) are becoming
increasingly popular and in future they may even become standard
equipment of of-the-shelf computers. They are, however, inferior
to UPS devices, because they share a power supply with their host
(a cluster node). If a node stays without power, the device
supposed to control it would be just as useless. Even though this
is obvious to us, the cluster manager is not in the know and will
try to fence the node in vain. This will continue forever because
all other resource operations would wait for the fencing/stonith
operation to succeed.</p></div>
<div class="paragraph"><p>The testing devices are used exclusively for testing purposes.
They are usually more gentle on the hardware. Once the cluster
goes into production, they must be replaced with real fencing
devices.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_stonith_shoot_the_other_node_in_the_head">STONITH (Shoot The Other Node In The Head)</h2>
<div class="sectionbody">
<div class="paragraph"><p>Stonith is our fencing implementation. It provides the node level
fencing.</p></div>
<div class="paragraph"><div class="title">NB</div><p>The stonith and fencing terms are often used
interchangeably here as well as in other texts.</p></div>
<div class="paragraph"><p>The stonith subsystem consists of two components:</p></div>
<div class="ulist"><ul>
<li>
<p>
stonithd
</p>
</li>
<li>
<p>
stonith plugins
</p>
</li>
</ul></div>
<div class="sect2">
<h3 id="_stonithd">stonithd</h3>
<div class="paragraph"><p>stonithd is a daemon which may be accessed by the local processes
or over the network. It accepts commands which correspond to
fencing operations: reset, power-off, and power-on.  It may also
check the status of the fencing device.</p></div>
<div class="paragraph"><p>stonithd runs on every node in the CRM HA cluster. The
stonithd instance running on the DC node receives a fencing
request from the CRM. It is up to this and other stonithd
programs to carry out the desired fencing operation.</p></div>
</div>
<div class="sect2">
<h3 id="_stonith_plugins">Stonith plugins</h3>
<div class="paragraph"><p>For every supported fencing device there is a stonith plugin
which is capable of controlling that device. A stonith plugin is
the interface to the fencing device. All stonith plugins look the
same to stonithd, but are quite different on the other side
reflecting the nature of the fencing device.</p></div>
<div class="paragraph"><p>Some plugins support more than one device. A typical example is
ipmilan (or external/ipmi) which implements the IPMI protocol and
can control any device which supports this protocol.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_crm_stonith_configuration">CRM stonith configuration</h2>
<div class="sectionbody">
<div class="paragraph"><p>The fencing configuration consists of one or more stonith
resources.</p></div>
<div class="paragraph"><p>A stonith resource is a resource of class stonith and it is
configured just like any other resource. The list of parameters
(attributes) depend on and are specific to a stonith type. Use
the stonith(1) program to see the list:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>$ stonith -t ibmhmc -n
ipaddr
$ stonith -t ipmilan -n
hostname  ipaddr  port  auth  priv  login  password reset_method</code></pre>
</div></div>
<div class="paragraph"><div class="title">NB</div><p>It is easy to guess the class of a fencing device from
the set of attribute names.</p></div>
<div class="paragraph"><p>A short help text is also available:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>$ stonith -t ibmhmc -h
STONITH Device: ibmhmc - IBM Hardware Management Console (HMC)
Use for IBM i5, p5, pSeries and OpenPower systems managed by HMC
  Optional parameter name managedsyspat is white-space delimited
list of patterns used to match managed system names; if last
character is '*', all names that begin with the pattern are matched
  Optional parameter name password is password for hscroot if
passwordless ssh access to HMC has NOT been setup (to do so,
it is necessary to create a public/private key pair with
empty passphrase - see "Configure the OpenSSH client" in the
redbook for more details)
For more information see
http://publib-b.boulder.ibm.com/redbooks.nsf/RedbookAbstracts/SG247038.html</code></pre>
</div></div>
<div class="sidebarblock">
<div class="content">
<div class="title">You just said that there is stonithd and stonith plugins. What&#8217;s with these resources now?</div>
<div class="paragraph"><p>Resources of class stonith are just a representation of stonith
plugins in the CIB. Well, a bit more: apart from the fencing
operations, the stonith resources, just like any other, may be
started and stopped and monitored. The start and stop operations
are a bit of a misnomer: enable and disable would serve better,
but it&#8217;s too late to change that. So, these two are actually
administrative operations and do not translate to any operation
on the fencing device itself. Monitor, however, does translate to
device status.</p></div>
</div></div>
<div class="paragraph"><p>A dummy stonith resource configuration, which may be used in some
testing scenarios is very simple:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>configure
primitive st-null stonith:null \
        params hostlist="node1 node2"
clone fencing st-null
commit</code></pre>
</div></div>
<div class="sidebarblock">
<div class="content">
<div class="title">NB</div>
<div class="paragraph"><p>All configuration examples are in the crm configuration tool
syntax. To apply them, put the sample in a text file, say
sample.txt and run:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>crm &lt; sample.txt</code></pre>
</div></div>
<div class="paragraph"><p>The configure and commit lines are omitted from further examples.</p></div>
</div></div>
<div class="paragraph"><p>An alternative configuration:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>primitive st-node1 stonith:null \
        params hostlist="node1"
primitive st-node2 stonith:null \
        params hostlist="node2"
location l-st-node1 st-node1 -inf: node1
location l-st-node2 st-node2 -inf: node2</code></pre>
</div></div>
<div class="paragraph"><p>This configuration is perfectly alright as far as the cluster
software is concerned. The only difference to a real world
configuration is that no fencing operation takes place.</p></div>
<div class="paragraph"><p>A more realistic, but still only for testing, is the following
external/ssh configuration:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>primitive st-ssh stonith:external/ssh \
        params hostlist="node1 node2"
clone fencing st-ssh</code></pre>
</div></div>
<div class="paragraph"><p>This one can also reset nodes. As you can see, this configuration
is remarkably similar to the first one which features the null
stonith device.</p></div>
<div class="sidebarblock">
<div class="content">
<div class="title">What is this clone thing?</div>
<div class="paragraph"><p>Clones are a CRM/Pacemaker feature. A clone is basically a
shortcut: instead of defining <em>n</em> identical, yet differently named
resources, a single cloned resource suffices. By far the most
common use of clones is with stonith resources if the stonith
device is accessible from all nodes.</p></div>
</div></div>
<div class="paragraph"><p>The real device configuration is not much different, though some
devices may require more attributes. For instance, an IBM RSA
lights-out device might be configured like this:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>primitive st-ibmrsa-1 stonith:external/ibmrsa-telnet \
        params nodename=node1 ipaddr=192.168.0.101 \
        userid=USERID passwd=PASSW0RD
primitive st-ibmrsa-2 stonith:external/ibmrsa-telnet \
        params nodename=node2 ipaddr=192.168.0.102 \
        userid=USERID passwd=PASSW0RD
# st-ibmrsa-1 can run anywhere but on node1
location l-st-node1 st-ibmrsa-1 -inf: node1
# st-ibmrsa-2 can run anywhere but on node2
location l-st-node2 st-ibmrsa-2 -inf: node2</code></pre>
</div></div>
<div class="sidebarblock">
<div class="content">
<div class="title">Why those strange location constraints?</div>
<div class="paragraph"><p>There is always certain probability that the stonith operation is
going to fail. Hence, a stonith operation on the node which is
the executioner too is not reliable. If the node is reset, then
it cannot send the notification about the fencing operation
outcome. The only way to do that is to assume that the operation
is going to succeed and send the notification beforehand. Then,
if the operation fails, we are in trouble.</p></div>
<div class="paragraph"><p>Given all this, we decided that, by convention, stonithd refuses
to kill its host.</p></div>
</div></div>
<div class="paragraph"><p>If you haven&#8217;t already guessed, configuration of a UPS kind of
fencing device is remarkably similar to all we have already
shown.</p></div>
<div class="paragraph"><p>All UPS devices employ the same mechanics for fencing. What is,
however, different is how the device itself is accessed. Old UPS
devices, those that were considered professional, used to have
just a serial port, typically connected at 1200baud using a
special serial cable. Many new ones still come equipped with a
serial port, but often they also sport a USB interface or an
Ethernet interface. The kind of connection we may make use of
depends on what the plugin supports. Let&#8217;s see a few examples for
the APC UPS equipment:</p></div>
<div class="literalblock">
<div class="content">
<pre><code>$ stonith -t apcmaster -h</code></pre>
</div></div>
<div class="literalblock">
<div class="content">
<pre><code>STONITH Device: apcmaster - APC MasterSwitch (via telnet)
NOTE: The APC MasterSwitch accepts only one (telnet)
connection/session a time. When one session is active,
subsequent attempts to connect to the MasterSwitch will fail.
For more information see http://www.apc.com/
List of valid parameter names for apcmaster STONITH device:
        ipaddr
                login
                password</code></pre>
</div></div>
<div class="literalblock">
<div class="content">
<pre><code>$ stonith -t apcsmart -h</code></pre>
</div></div>
<div class="literalblock">
<div class="content">
<pre><code>STONITH Device: apcsmart - APC Smart UPS
 (via serial port - NOT USB!).
 Works with higher-end APC UPSes, like
 Back-UPS Pro, Smart-UPS, Matrix-UPS, etc.
 (Smart-UPS may have to be &gt;= Smart-UPS 700?).
 See http://www.networkupstools.org/protocols/apcsmart.html
 for protocol compatibility details.
For more information see http://www.apc.com/
List of valid parameter names for apcsmart STONITH device:
                ttydev
                hostlist</code></pre>
</div></div>
<div class="paragraph"><p>The former plugin supports APC UPS with a network port and telnet
protocol. The latter plugin uses the APC SMART protocol over the
serial line which is supported by many different APC UPS product
lines.</p></div>
<div class="sidebarblock">
<div class="content">
<div class="title">So, what do I use: clones, constraints, both?</div>
<div class="paragraph"><p>It depends. Depends on the nature of the fencing device. For
example, if the device cannot serve more than one connection at
the time, then clones won&#8217;t do. Depends on how many hosts can the
device manage. If it&#8217;s only one, and that is always the case with
lights-out devices, then again clones are right out. Depends
also on the number of nodes in your cluster: the more nodes the
more desirable to use clones. Finally, it is also a matter of
personal preference.</p></div>
<div class="paragraph"><p>In short: if clones are safe to use with your configuration and
if they reduce the configuration, then make cloned stonith
resources.</p></div>
</div></div>
<div class="paragraph"><p>The CRM configuration is left as an exercise to the reader.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_monitoring_the_fencing_devices">Monitoring the fencing devices</h2>
<div class="sectionbody">
<div class="paragraph"><p>Just like any other resource, the stonith class agents also
support the monitor operation. Given that we have often seen
monitor either not configured or configured in a wrong way, we
have decided to devote a section to the matter.</p></div>
<div class="paragraph"><p>Monitoring stonith resources, which is actually checking status
of the corresponding fencing devices, is strongly recommended. So
strongly, that we should consider a configuration without it
invalid.</p></div>
<div class="paragraph"><p>On the one hand, though an indispensable part of an HA cluster, a
fencing device, being the last line of defense, is used seldom.
Very seldom and preferably never. On the other, for whatever
reason, the power management equipment is known to be rather
fragile on the communication side. Some devices were known to
give up if there was too much broadcast traffic on the wire. Some
cannot handle more than ten or so connections per minute. Some
get confused or depressed if two clients try to connect at the
same time. Most cannot handle more than one session at the time.
The bottom line: try not to exercise your fencing device too
often. It may not like it. Use monitoring regularly, yet
sparingly, say once every couple of hours. The probability that
within those few hours there will be a need for a fencing
operation and that the power switch would fail is usually low.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_odd_plugins">Odd plugins</h2>
<div class="sectionbody">
<div class="paragraph"><p>Apart from plugins which handle real devices, some stonith
plugins are a bit out of line and deserve special attention.</p></div>
<div class="sect2">
<h3 id="_external_kdumpcheck">external/kdumpcheck</h3>
<div class="paragraph"><p>Sometimes, it may be important to get a kernel core dump. This
plugin may be used to check if the dump is in progress. If
that is the case, then it will return true, as if the node has
been fenced, which is actually true given that it cannot run
any resources at the time. kdumpcheck is typically used in
concert with another, real, fencing device. See
README_kdumpcheck.txt for more details.</p></div>
</div>
<div class="sect2">
<h3 id="_external_sbd">external/sbd</h3>
<div class="paragraph"><p>This is a self-fencing device. It reacts to a so-called "poison
pill" which may be inserted into a shared disk. On shared storage
connection loss, it also makes the node commit suicide. See
<a href="http://www.linux-ha.org/wiki/SBD_Fencing">http://www.linux-ha.org/wiki/SBD_Fencing</a> for more details.</p></div>
</div>
<div class="sect2">
<h3 id="_meatware">meatware</h3>
<div class="paragraph"><p>Strange name and a simple concept. <code>meatware</code> requires help from a
human to operate. Whenever invoked, <code>meatware</code> logs a CRIT severity
message which should show up on the node&#8217;s console. The operator
should then make sure that the node is down and issue a
<code>meatclient(8)</code> command to tell <code>meatware</code> that it&#8217;s OK to tell the
cluster that it may consider the node dead. See <code>README.meatware</code>
for more information.</p></div>
</div>
<div class="sect2">
<h3 id="_null">null</h3>
<div class="paragraph"><p>This one is probably not of much importance to the general
public. It is used in various testing scenarios. <code>null</code> is an
imaginary device which always behaves and always claims that it
has shot a node, but never does anything. Sort of a
happy-go-lucky. Do not use it unless you know what you are doing.</p></div>
</div>
<div class="sect2">
<h3 id="_suicide">suicide</h3>
<div class="paragraph"><p><code>suicide</code> is a software-only device, which can reboot a node it is
running on. It depends on the operating system, so it should be
avoided whenever possible. But it is OK on one-node clusters.
<code>suicide</code> and <code>null</code> are the only exceptions to the "don&#8217;t shoot my
host" rule.</p></div>
<div class="sidebarblock">
<div class="content">
<div class="title">What about that stonithd? You forgot about it, eh?</div>
<div class="paragraph"><p>The stonithd daemon, though it is really the master of ceremony,
requires no configuration itself. All configuration is stored in
the CIB.</p></div>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_resources">Resources</h2>
<div class="sectionbody">
<div class="paragraph"><p><a href="http://www.linux-ha.org/wiki/STONITH">http://www.linux-ha.org/wiki/STONITH</a></p></div>
<div class="paragraph"><p><a href="http://www.clusterlabs.org/doc/crm_fencing.html">http://www.clusterlabs.org/doc/crm_fencing.html</a></p></div>
<div class="paragraph"><p><a href="http://www.clusterlabs.org/doc/en-US/Pacemaker/1.0/html/Pacemaker_Explained">http://www.clusterlabs.org/doc/en-US/Pacemaker/1.0/html/Pacemaker_Explained</a></p></div>
<div class="paragraph"><p><a href="http://techthoughts.typepad.com/managing_computers/2007/10/split-brain-quo.html">http://techthoughts.typepad.com/managing_computers/2007/10/split-brain-quo.html</a></p></div>
</div>
</div>
</div>
<div id="footnotes"><hr /></div>
<div id="footer">
</div>
</body>
</html>
pacemaker-doc 1.1.18-0ubuntu1 / usr / share / doc / pacemaker / crm_fencing.html
