/usr/share/doc/python-fedmsg-doc/html/deployment.html is in python-fedmsg-doc 0.9.3-2.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Deploying fedmsg for yourself — Fedora Messaging documentation</title>
<link rel="stylesheet" href="_static/site.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Noticia+Text|Open+Sans|Droid+Sans+Mono" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/cloud.js"></script>
<link rel="top" title="Fedora Messaging documentation" href="index.html" />
<link rel="next" title="Commands" href="commands.html" />
<link rel="prev" title="Development" href="development.html" />
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<div class="relbar-top">
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> </li>
<li class="right" >
<a href="commands.html" title="Commands"
accesskey="N">next</a> </li>
<li class="right" >
<a href="development.html" title="Development"
accesskey="P">previous</a> </li>
<li><a href="index.html">Fedora Messaging documentation</a> »</li>
</ul>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="deploying-fedmsg-for-yourself">
<h1>Deploying fedmsg for yourself<a class="headerlink" href="#deploying-fedmsg-for-yourself" title="Permalink to this headline">¶</a></h1>
<p>Elsewhere, the emphasis in fedmsg docs is on how to subscribe to an existing fedmsg
deployment; how do I listen for koji builds from Fedora Infrastructure? This
document, on the other hand, is directed at those who want to deploy fedmsg for
their own systems.</p>
<p>This document also only goes as far as setting things up for a single machine.
You typically deploy fedmsg across an <em>infrastructure</em> but if you just want to
try it out for “proof-of-concept”, these are the docs for you.</p>
<p>Lastly, the emphasis here is on the practical – there will be lots of
examples. There is plenty of long-winded explanation over at <a class="reference internal" href="overview.html"><em>Overview</em></a>.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>Caveat: fedmsg is deployed at a couple different sites:</p>
<ul class="simple">
<li>Fedora Infrastructure</li>
<li><a class="reference external" href="http://data.gouv.fr">data.gouv.fr</a></li>
<li>to some extent, Debian Infrastructure</li>
</ul>
<p class="last">We wrote this document much later afterwards, so, if you come across errors,
or things that don’t work right. Please <a class="reference external" href="https://github.com/fedora-infra/fedmsg/issues/new">report it</a>.</p>
</div>
<div class="section" id="the-basics">
<h2>The basics<a class="headerlink" href="#the-basics" title="Permalink to this headline">¶</a></h2>
<p>First install fedmsg:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo yum install fedmsg
</pre></div>
</div>
<p>Now you have some <tt class="docutils literal"><span class="pre">fedmsg-*</span></tt> cli tools like <tt class="docutils literal"><span class="pre">fedmsg-tail</span></tt> and
<tt class="docutils literal"><span class="pre">fedmsg-logger</span></tt>.</p>
<p>On Fedora systems, fedmsg is configured by default to subscribe to Fedora
Infrastructure’s bus. Since you are deploying for your own site, you don’t
want that. So edit <tt class="docutils literal"><span class="pre">/etc/fedmsg.d/endpoints.py</span></tt> and <em>comment out the whole
“fedora-infrastructure” section</em>, like this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c">#"fedora-infrastructure": [</span>
<span class="c"># "tcp://hub.fedoraproject.org:9940",</span>
<span class="c"># #"tcp://stg.fedoraproject.org:9940",</span>
<span class="c">#],</span>
</pre></div>
</div>
</div>
<div class="section" id="starting-fedmsg-relay">
<h2>Starting fedmsg-relay<a class="headerlink" href="#starting-fedmsg-relay" title="Permalink to this headline">¶</a></h2>
<p>Not all fedmsg interactions require the relay, but publishing messages from a
terminal does.</p>
<p>Install fedmsg-relay and start it:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo yum install fedmsg-relay
$ sudo systemctl restart fedmsg-relay
$ sudo systemctl enable fedmsg-relay
</pre></div>
</div>
<p>It has a pid file in <tt class="docutils literal"><span class="pre">/var/run/fedmsg/fedmsg-relay.pid</span></tt> and you can view the
logs in <tt class="docutils literal"><span class="pre">journalctl</span> <span class="pre">--follow</span></tt>. On other systems you can find the logs in
<tt class="docutils literal"><span class="pre">/var/log/fedmsg/fedmsg-relay.log</span></tt>.</p>
<p>Out of the box, it should be listening for incoming messages on
<tt class="docutils literal"><span class="pre">tcp://127.0.0.1:2003</span></tt> and re-publishing them indiscriminately at
<tt class="docutils literal"><span class="pre">tcp://127.0.0.1:4001</span></tt>. It is fine to keep these defaults.</p>
</div>
<div class="section" id="test-it-out">
<h2>Test it out<a class="headerlink" href="#test-it-out" title="Permalink to this headline">¶</a></h2>
<p>Try a test! Open two terminals:</p>
<ul class="simple">
<li>In the first, type <tt class="docutils literal"><span class="pre">fedmsg-tail</span> <span class="pre">--really-pretty</span></tt></li>
<li>In the second, type <tt class="docutils literal"><span class="pre">echo</span> <span class="pre">"Hello</span> <span class="pre">world"</span> <span class="pre">|</span> <span class="pre">fedmsg-logger</span></tt></li>
</ul>
<p>You should see the JSON representation of your message show up in the first
terminal. It should look something like this.</p>
<div class="highlight-javascript"><div class="highlight"><pre><span class="p">{</span>
<span class="s2">"username"</span><span class="o">:</span> <span class="s2">"root"</span><span class="p">,</span>
<span class="s2">"i"</span><span class="o">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s2">"timestamp"</span><span class="o">:</span> <span class="mi">1393878837</span><span class="p">,</span>
<span class="s2">"msg_id"</span><span class="o">:</span> <span class="s2">"2014-f1c49f0b-5caf-49e6-b79a-cc54bcfac602"</span><span class="p">,</span>
<span class="s2">"topic"</span><span class="o">:</span> <span class="s2">"org.fedoraproject.dev.logger.log"</span><span class="p">,</span>
<span class="s2">"msg"</span><span class="o">:</span> <span class="p">{</span>
<span class="s2">"log"</span><span class="o">:</span> <span class="s2">"Hello world"</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>These are two handy tools for debugging the configuration of your
bus.</p>
</div>
<div class="section" id="store-all-messages">
<h2>Store all messages<a class="headerlink" href="#store-all-messages" title="Permalink to this headline">¶</a></h2>
<p>We use a tool called <a class="reference external" href="https://github.com/fedora-infra/datanommer">datanommer</a>
to store all the messages that come across the bus in a postgres database.
Using whatever relational database you like should be possible just by
modifying the config.</p>
<div class="section" id="setting-up-postgres">
<h3>Setting up postgres<a class="headerlink" href="#setting-up-postgres" title="Permalink to this headline">¶</a></h3>
<p>Here, set up a postgres database:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo yum install postgresql-server python-psycopg2
$ postgresql-setup initdb
</pre></div>
</div>
<p>Edit the <tt class="docutils literal"><span class="pre">/var/lib/pgsql/data/pg_hba.conf</span></tt> as the user postgres. You might
find a line like this:</p>
<div class="highlight-python"><div class="highlight"><pre>host all all 127.0.0.1/32 ident sameuser
host all all ::1/128 ident sameuser
</pre></div>
</div>
<p>Instead of that line, change it to this:</p>
<div class="highlight-python"><div class="highlight"><pre>host all all 127.0.0.1/32 trust
host all all ::1/128 trust
</pre></div>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Using <tt class="docutils literal"><span class="pre">trust</span></tt> is super unsafe long term. That means that anyone
with any password will be able to connect locally. That’s fine for our
little one-box test here, but you’ll want to use md5 or kerberos or
something long term.</p>
</div>
<p>Start up postgres:</p>
<div class="highlight-python"><div class="highlight"><pre>$ systemctl start postgresql
$ systemctl enable postgresql
</pre></div>
</div>
<p>Create a database user and the db itself for datanommer and friends:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo -u postgres createuser -SDRPE datanommer
$ sudo -u postgres createdb -E utf8 datanommer -O datanommer
</pre></div>
</div>
</div>
<div class="section" id="setting-up-datanommer">
<h3>Setting up datanommer<a class="headerlink" href="#setting-up-datanommer" title="Permalink to this headline">¶</a></h3>
<p>Install it:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo yum install fedmsg-hub python-datanommer-consumer datanommer-commands
</pre></div>
</div>
<p>Edit the configuration to 1) be enabled, 2) point at your newly created
postgres db. Edit <tt class="docutils literal"><span class="pre">/etc/fedmsg.d/datanommer.py</span></tt> and change the whole thing
to look like this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">config</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">'datanommer.enabled'</span><span class="p">:</span> <span class="bp">True</span><span class="p">,</span>
<span class="s">'datanommer.sqlalchemy.url'</span><span class="p">:</span> <span class="s">'postgresql://datanommer:password@localhost/datanommer'</span><span class="p">,</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Run the following command from the <tt class="docutils literal"><span class="pre">datanommer-commands</span></tt> package to set up
the tables. It will read in that connection url from
<tt class="docutils literal"><span class="pre">/etc/fedmsg.d/datanommer.py</span></tt>:</p>
<div class="highlight-python"><div class="highlight"><pre>$ datanommer-create-db
</pre></div>
</div>
<p>Start the <tt class="docutils literal"><span class="pre">fedmsg-hub</span></tt> daemon, which will pick up the datanommer plugin,
which will in turn read in that connection string, start listening for
messages, and store them all in the db.</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo systemctl start fedmsg-hub
$ sudo systemctl enable fedmsg-hub
</pre></div>
</div>
<p>You can check <tt class="docutils literal"><span class="pre">journalctl</span> <span class="pre">--follow</span></tt> for logs.</p>
<p>Try testing again with <tt class="docutils literal"><span class="pre">fedmsg-logger</span></tt>. After publishing a message, you
should see it in the datanommer stats if you run <tt class="docutils literal"><span class="pre">datanommer-stats</span></tt>:</p>
<div class="highlight-python"><div class="highlight"><pre>$ datanommer-stats
[2014-03-03 20:34:43][ fedmsg INFO] logger has 2 entries
</pre></div>
</div>
</div>
</div>
<div class="section" id="querying-datanommer-with-datagrepper">
<h2>Querying datanommer with datagrepper<a class="headerlink" href="#querying-datanommer-with-datagrepper" title="Permalink to this headline">¶</a></h2>
<p>You can, of course, query datanommer with SQL yourself (and there’s a python
API for directly querying in the <tt class="docutils literal"><span class="pre">datanommer.models</span></tt> module). For the rest
here is the HTTP API we have called “datagrepper”. Let’s set it up:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo yum install datagrepper mod_wsgi
</pre></div>
</div>
<p>Add a config file for it in <tt class="docutils literal"><span class="pre">/etc/httpd/conf.d/datagrepper.conf</span></tt> with these contents:</p>
<div class="highlight-python"><div class="highlight"><pre>LoadModule wsgi_module modules/mod_wsgi.so
# Static resources for the datagrepper app.
Alias /datagrepper/css /usr/lib/python2.7/site-packages/datagrepper/static/css
WSGIDaemonProcess datagrepper user=fedmsg group=fedmsg maximum-requests=50000 display-name=datagrepper processes=8 threads=4 inactivity-timeout=300
WSGISocketPrefix run/wsgi
WSGIRestrictStdout Off
WSGIRestrictSignal Off
WSGIPythonOptimize 1
WSGIScriptAlias /datagrepper /usr/share/datagrepper/apache/datagrepper.wsgi
<Directory /usr/share/datagrepper/>
WSGIProcessGroup datagrepper
# XXX - The syntax for this is different for different versions of apache
Require all granted
</Directory>
</pre></div>
</div>
<p>Finally, start up httpd with:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo systemctl restart httpd
$ sudo systemctl enable httpd
</pre></div>
</div>
<p>And it should just work. Open a web browser and try to visit
<tt class="docutils literal"><span class="pre">http://localhost/datagrepper/</span></tt>.</p>
<p>The whole point of datagrepper is its API, which you might experiment with
using the httpie tool:</p>
<div class="highlight-python"><div class="highlight"><pre>$ sudo yum install httpie
$ http get http://localhost/datagrepper/raw/ order==desc
</pre></div>
</div>
</div>
<div class="section" id="outro">
<h2>Outro<a class="headerlink" href="#outro" title="Permalink to this headline">¶</a></h2>
<p>This document is a work in progress. Future topics may include selinux and <a class="reference internal" href="crypto.html"><em>Cryptography and Message Signing</em></a>.</p>
<p>Let us know what you’d like to know if it is missing.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper"><h1 class='fedmsg-logo'><span></span>fedmsg</h1><div class="sphinxlocaltoc">
<h3><a href="index.html">Page contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Deploying fedmsg for yourself</a><ul>
<li><a class="reference internal" href="#the-basics">The basics</a></li>
<li><a class="reference internal" href="#starting-fedmsg-relay">Starting fedmsg-relay</a></li>
<li><a class="reference internal" href="#test-it-out">Test it out</a></li>
<li><a class="reference internal" href="#store-all-messages">Store all messages</a><ul>
<li><a class="reference internal" href="#setting-up-postgres">Setting up postgres</a></li>
<li><a class="reference internal" href="#setting-up-datanommer">Setting up datanommer</a></li>
</ul>
</li>
<li><a class="reference internal" href="#querying-datanommer-with-datagrepper">Querying datanommer with datagrepper</a></li>
<li><a class="reference internal" href="#outro">Outro</a></li>
</ul>
</li>
</ul>
</div>
<div class="sphinxprev">
<h4>Previous page</h4>
<p class="topless"><a href="development.html"
title="Previous page">← Development</a></p>
</div>
<div class="sphinxnext">
<h4>Next page</h4>
<p class="topless"><a href="commands.html"
title="Next page">→ Commands</a></p>
</div><h3>Edit this document</h3>
<p>Go to <a href="https://github.com/fedora-infra/fedmsg/blob/develop/doc/deployment.rst">Deploying fedmsg for yourself</a>
on GitHub.
</p>
<p>Use the web interface to <b>fork</b> the repo, <b>edit</b> the file, and
<b>send a pull request</b>.</p>
<p>Your changes will be queued for review under project's
<a href="https://github.com/fedora-infra/fedmsg/pulls">Pull requests</a> tab on
Github.</p>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="relbar-bottom">
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> </li>
<li class="right" >
<a href="commands.html" title="Commands"
>next</a> </li>
<li class="right" >
<a href="development.html" title="Development"
>previous</a> </li>
<li><a href="index.html">Fedora Messaging documentation</a> »</li>
</ul>
</div>
</div>
<div class="footer">
© Copyright 2012 - 2014 Red Hat, Inc. and others..
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.2.3.
</div>
<!-- cloud_sptheme 1.4 -->
</body>
</html>
|