/usr/share/doc/simgrid/html/introduction.html is in simgrid-doc 3.10-7.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<title>SimGrid: Introduction to SimGrid</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
</script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
$(document).ready(function() { searchBox.OnSelectItem(0); });
</script>
<link href="stylesheet.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">SimGrid
 <span id="projectnumber">3.10</span>
</div>
<div id="projectbrief">Versatile Simulation of Distributed Systems</div>
</td>
</tr>
</tbody>
</table>
</div>
<div id="navrow1" class="tabs">
<ul class="tablist">
<li><a href="http://simgrid.gforge.inria.fr/"><span>Home page</span></a></li>
<li><a href="http://simgrid.gforge.inria.fr/documentation.html"><span>Online documentation</span></a></li>
<li><a href="https://gforge.inria.fr/projects/simgrid"><span>Dev's Corner</span></a></li>
<li> <div id="MSearchBox" class="MSearchBoxInactive">
<span class="left">
<img id="MSearchSelect" src="search/mag_sel.png"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
alt=""/>
<input type="text" id="MSearchField" value="Search" accesskey="S"
onfocus="searchBox.OnSearchFieldFocus(true)"
onblur="searchBox.OnSearchFieldFocus(false)"
onkeyup="searchBox.OnSearchFieldChange(event)"/>
</span><span class="right">
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
</span>
</div>
</li>
</ul>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.1.2 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "search",false,'Search');
</script>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('introduction.html','');});
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
<a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(0)"><span class="SelectionMark"> </span>All</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(1)"><span class="SelectionMark"> </span>Data Structures</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(2)"><span class="SelectionMark"> </span>Functions</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(3)"><span class="SelectionMark"> </span>Variables</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(4)"><span class="SelectionMark"> </span>Typedefs</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(5)"><span class="SelectionMark"> </span>Enumerations</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(6)"><span class="SelectionMark"> </span>Enumerator</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(7)"><span class="SelectionMark"> </span>Groups</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(8)"><span class="SelectionMark"> </span>Pages</a></div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
<div class="header">
<div class="headertitle">
<div class="title">Introduction to SimGrid </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p><a href="http://simgrid.gforge.inria.fr/">SimGrid</a> is a toolkit that provides core functionalities for the simulation of distributed applications in heterogeneous distributed environments.</p>
<p>The specific goal of the project is to facilitate research in the area of distributed and parallel application scheduling on distributed computing platforms ranging from simple network of workstations to Computational Grids.</p>
<h1>Scenario</h1>
<p>The goal of this practical session is to illustrate various usage of the MSG interface. To this end we will use the following simple setting:</p>
<blockquote class="doxtable">
<p>Assume we have a (possibly large) bunch of (possibly large) data to process and which originally reside on a server (a.k.a. master). For sake of simplicity, we assume all input file require the same amount of computation. We assume the server can be helped by a (possibly large) set of worker machines. What is the best way to organize the computations ?</p>
<p></p>
</blockquote>
<p>Although this looks like a very simple setting it raises several interesting questions:</p>
<ul>
<li><p class="startli">Which algorithm should the master use to send workload?</p>
<p class="startli">The most obvious algorithm would be to send tasks to workers in a round-robin fashion. This is the initial code we provide you.</p>
<p class="startli">A less obvious one but probably more efficient would be to set up a request mechanism where client first ask for tasks, which allows the server to decide which request to answer and possibly to send the tasks to the fastest machines. Maybe you can think of a smarter mechanism...</p>
</li>
</ul>
<ul>
<li><p class="startli">How much tasks should the client ask for?</p>
<p class="startli">Indeed, if we set up a request mechanism and that workers only send request whenever they have no more task to process, they are likely to be poorly exploited since they will have to wait for the master to consider their request and for the input data to be transferred. A client should thus probably request a pool of tasks but if it requests too much task, it is likely to lead to a poor load-balancing...</p>
</li>
</ul>
<ul>
<li><p class="startli">How is the quality of such algorithm dependent on the platform characteristics? on the task characteristics?</p>
<p class="startli">Whenever the input communication time is very small compared to processing time and workers are homogeneous, it is likely that the round-robin algorithm performs very well. Would it still hold true when transfer time is not negligible and the platform is, say, a volunteer computing system ?</p>
</li>
</ul>
<ul>
<li><p class="startli">The network topology interconnecting the master and the workers may be quite complicated. How does such topology impact the previous result?</p>
<p class="startli">When data transfers are the bottleneck, it is likely that a good modeling of the platform becomes essential, in which case, you may want to be able to account for complex platform topologies.</p>
</li>
</ul>
<ul>
<li><p class="startli">Do the algorithms depend on a perfect knowledge of this topology?</p>
<p class="startli">Should we still use a flat master worker deployment or should we use a</p>
</li>
</ul>
<ul>
<li><p class="startli">How is such algorithm sensitive to external workload variation?</p>
<p class="startli">What if bandwidth, latency and power can vary with no warning? Shouldn't you study whether your algorithm is sensitive to such load variations?</p>
</li>
</ul>
<ul>
<li><p class="startli">Although an algorithm may be more efficient than another, how does it interfere with other applications?</p>
<p class="startli">As you can see, this very simple setting may need to evolve way beyond what you initially imagined.</p>
<blockquote class="doxtable">
<p>Premature optimization is the root of all evil. – D.E.Knuth</p>
</blockquote>
<p class="startli">Furthermore, writing your own simulator is much harder that what you may imagine. This is why should rely on an established and flexible one.</p>
</li>
</ul>
<p>The following figure is a screenshot of <a href="http://triva.gforge.inria.fr/index.html">triva</a> visualizing a <a href="http://hal.inria.fr/inria-00529569">SimGrid simulation</a> of two master worker applications (one in light gray and the other in dark gray) running in concurrence and showing resource usage over a long period of time.</p>
<div class="image">
<img src="sc3-description.png" alt="sc3-description.png"/>
<div class="caption">
Test</div></div>
<h1>Prerequisites</h1>
<h2>Tutorials</h2>
<p>A lot of information on how to install and use Simgrid are available on the <a href="http://simgrid.gforge.inria.fr/documentation.html">online documentation</a> and in the tutorials:</p>
<ul>
<li><a href="http://simgrid.gforge.inria.fr/tutorials/simgrid-use-101.pdf">http://simgrid.gforge.inria.fr/tutorials/simgrid-use-101.pdf</a></li>
<li><a href="http://simgrid.gforge.inria.fr/tutorials/simgrid-tracing-101.pdf">http://simgrid.gforge.inria.fr/tutorials/simgrid-tracing-101.pdf</a></li>
<li><a href="http://simgrid.gforge.inria.fr/tutorials/simgrid-platf-101.pdf">http://simgrid.gforge.inria.fr/tutorials/simgrid-platf-101.pdf</a></li>
</ul>
<h2>Installing SimGrid</h2>
<pre class="fragment">sudo apt-get install simgrid
</pre><p>This tutorial requires simgrid 3.8 at last so you may need to get the <a href="http://packages.debian.org/unstable/main/simgrid">debian package</a>. Here is a shortcut:</p>
<ul>
<li>AMD64: <a href="http://ftp.de.debian.org/debian/pool/main/s/simgrid/simgrid_3.8.1-2_amd64.deb">http://ftp.de.debian.org/debian/pool/main/s/simgrid/simgrid_3.8.1-2_amd64.deb</a></li>
<li>i386: <a href="http://ftp.de.debian.org/debian/pool/main/s/simgrid/simgrid_3.8.1-2_i386.deb">http://ftp.de.debian.org/debian/pool/main/s/simgrid/simgrid_3.8.1-2_i386.deb</a></li>
</ul>
<p>Then</p>
<div class="fragment"><div class="line">sudo dpkg -i simgrid_3.8*.deb</div>
</div><!-- fragment --><h1>Recommended Steps</h1>
<h2>Installing Viva</h2>
<p>This <a href="http://triva.gforge.inria.fr/index.html">software</a> will be useful to make fancy graph or treemap visualizations and get a better understanding of simulations. You will first need to install pajeng:</p>
<div class="fragment"><div class="line">sudo apt-<span class="keyword">get</span> install git cmake build-essential libqt4-dev libboost-dev freeglut3-dev ;</div>
<div class="line">git clone https:<span class="comment">//github.com/schnorr/pajeng.git</span></div>
<div class="line">cd pajeng && mkdir -p build && cd build && cmake ../ -DCMAKE_INSTALL_PREFIX=$HOME && make -j install </div>
<div class="line">cd ../../</div>
</div><!-- fragment --><p>Then you can install viva.</p>
<div class="fragment"><div class="line">sudo apt-<span class="keyword">get</span> install libboost-dev libconfig++-dev libconfig8-dev libgtk2.0-dev freeglut3-dev</div>
<div class="line">git clone https:<span class="comment">//github.com/schnorr/viva.git</span></div>
<div class="line">cd viva && mkdir -p build_graph && cd build_graph && cmake ../ -DTUPI_LIBRARY=ON -DVIVA=ON -DCMAKE_INSTALL_PREFIX=$HOME && make -j install </div>
<div class="line">cd ../../</div>
</div><!-- fragment --><h2>Installing Paje</h2>
<p>This <a href="http://paje.sourceforge.net/">software</a> provides a Gantt-chart visualization.</p>
<div class="fragment"><div class="line">sudo apt-<span class="keyword">get</span> install paje.app</div>
</div><!-- fragment --><h2>Installing Vite</h2>
<p>This software provides a <a href="http://vite.gforge.inria.fr/">Gantt-chart visualization</a>.</p>
<div class="fragment"><div class="line">sudo apt-<span class="keyword">get</span> install vite</div>
</div><!-- fragment --><h1>Let's get Started</h1>
<h2>Setting up and Compiling</h2>
<p>The corresponding archive with all source files and platform files can be obtained <a href="http://simgrid.gforge.inria.fr/tutorials/msg-tuto/msg-tuto.tgz">here</a>.</p>
<div class="fragment"><div class="line">tar zxf msg-tuto.tgz</div>
<div class="line">cd msg-tuto/src</div>
<div class="line">make</div>
</div><!-- fragment --><p>As you can see, there is already a nice Makefile that compiles everything for you. Now the tiny example has been compiled and it can be easily run as follows:</p>
<div class="fragment"><div class="line">./masterworker0 platforms/platform.xml deployment0.xml 2>&1</div>
</div><!-- fragment --><p>If you create a single self-content C-file named foo.c, the corresponding program will be simply compiled and linked with SimGrid by typing:</p>
<div class="fragment"><div class="line">make foo</div>
</div><!-- fragment --><p>For a more "fancy" output, you can try:</p>
<div class="fragment"><div class="line">./masterworker0 platforms/platform.xml deployment0.xml 2>&1 | simgrid-colorizer</div>
</div><!-- fragment --><p>For a really fancy output, you should use <a href="http://triva.gforge.inria.fr/index.html">viva/triva</a>:</p>
<div class="fragment"><div class="line">./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:yes \</div>
<div class="line"> --cfg=tracing/uncategorized:yes --cfg=viva/uncategorized:uncat.plist</div>
<div class="line">LANG=C ; viva simgrid.trace uncat.plist</div>
</div><!-- fragment --><p>For a more classical Gantt-Chart visualization, you can produce a <a href="http://paje.sourceforge.net/">Paje</a> trace:</p>
<div class="fragment"><div class="line">./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:yes \</div>
<div class="line"> --cfg=tracing/msg/process:yes</div>
<div class="line">LANG=C ; Paje simgrid.trace</div>
</div><!-- fragment --><p>Alternatively, you can use <a href="http://vite.gforge.inria.fr/">vite</a>.</p>
<div class="fragment"><div class="line">./masterworker0 platforms/platform.xml deployment0.xml --cfg=tracing:yes \</div>
<div class="line"> --cfg=tracing/msg/process:yes --cfg=tracing/basic:yes</div>
<div class="line">vite simgrid.trace</div>
</div><!-- fragment --><h2>Getting Rid of Workers in the Deployment File</h2>
<p>In the previous example, the deployment file <code>deployment0.xml</code> is tightly connected to the platform file <code>platform.xml</code> and a worker process is launched on each host:</p>
<div class="fragment"><div class="line"><?xml version=<span class="stringliteral">'1.0'</span>?></div>
<div class="line"><!DOCTYPE platform SYSTEM <span class="stringliteral">"http://simgrid.gforge.inria.fr/simgrid.dtd"</span>></div>
<div class="line"><platform version=<span class="stringliteral">"3"</span>></div>
<div class="line"> </div>
<div class="line"> <process host=<span class="stringliteral">"Tremblay"</span> <span class="keyword">function</span>=<span class="stringliteral">"master"</span>></div>
<div class="line"> <argument value=<span class="stringliteral">"20"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"50000000"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"1000000"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"Jupiter"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"Fafard"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"Ginette"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"Bourassa"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"Tremblay"</span>/> </div>
<div class="line"> </process></div>
<div class="line"> </div>
<div class="line"> <process host=<span class="stringliteral">"Tremblay"</span> <span class="keyword">function</span>=<span class="stringliteral">"worker"</span> on_failure=<span class="stringliteral">"RESTART"</span>/></div>
<div class="line"> <process host=<span class="stringliteral">"Jupiter"</span> <span class="keyword">function</span>=<span class="stringliteral">"worker"</span> on_failure=<span class="stringliteral">"RESTART"</span>/></div>
<div class="line"> <process host=<span class="stringliteral">"Fafard"</span> <span class="keyword">function</span>=<span class="stringliteral">"worker"</span> on_failure=<span class="stringliteral">"RESTART"</span>/></div>
<div class="line"> <process host=<span class="stringliteral">"Ginette"</span> <span class="keyword">function</span>=<span class="stringliteral">"worker"</span> on_failure=<span class="stringliteral">"RESTART"</span>/></div>
<div class="line"> <process host=<span class="stringliteral">"Bourassa"</span> <span class="keyword">function</span>=<span class="stringliteral">"worker"</span> on_failure=<span class="stringliteral">"RESTART"</span>/></div>
<div class="line"></platform></div>
</div><!-- fragment --><p>This is ok as the platform is rather small but will be painful when using larger platforms. Instead, modify the simulator <code>masterworker0.c</code> into <code>masterworker1.c</code> so that the master launches a worker process on all the other machines at startup. The new deployment file <code>deployment1.xml</code> should thus now simply be:</p>
<div class="fragment"><div class="line"><?xml version=<span class="stringliteral">'1.0'</span>?></div>
<div class="line"><!DOCTYPE platform SYSTEM <span class="stringliteral">"http://simgrid.gforge.inria.fr/simgrid.dtd"</span>></div>
<div class="line"><platform version=<span class="stringliteral">"3"</span>></div>
<div class="line"> </div>
<div class="line"> <process host=<span class="stringliteral">"Tremblay"</span> <span class="keyword">function</span>=<span class="stringliteral">"master"</span>></div>
<div class="line"> <argument value=<span class="stringliteral">"20"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"50000000"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"1000000"</span>/> </div>
<div class="line"> </process></div>
<div class="line"></platform></div>
</div><!-- fragment --><p>To this end you may need the following MSG functions, whose behavior is described in the <a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">online documentation</a> (hint: use the search field to access directly the function you are looking for):</p>
<div class="fragment"><div class="line"><span class="keywordtype">int</span> <a class="code" href="group__m__host__management.html#gaafceb2773bb9e39878592ff9a9a81a01" title="Return the current number MSG hosts.">MSG_get_host_number</a> (<span class="keywordtype">void</span>)</div>
<div class="line"><a class="code" href="group__XBT__dynar__cons.html#gac826571988d2b63ae225e5c62ecdbc79" title="Dynar data type (opaque type)">xbt_dynar_t</a> <a class="code" href="group__m__host__management.html#gaa886b8367e635efe11211b2aca5a1d43" title="Return a dynar containing all the hosts declared at a given point of time.">MSG_hosts_as_dynar</a>(<span class="keywordtype">void</span>);</div>
<div class="line"><span class="keywordtype">void</span> * <a class="code" href="group__XBT__dynar__array.html#ga8fb5745b688cae53556f69b8e728ef70" title="Transform a dynar into a NULL terminated array. The dynar won't be usable afterwards.">xbt_dynar_to_array</a> (<a class="code" href="group__XBT__dynar__cons.html#gac826571988d2b63ae225e5c62ecdbc79" title="Dynar data type (opaque type)">xbt_dynar_t</a> dynar);</div>
<div class="line"><a class="code" href="structs__smx__process.html" title="Process datatype.">msg_process_t</a> <a class="code" href="group__m__process__management.html#ga943d27b38754b2af9f78e80cc29db306" title="Creates and runs a new msg_process_t.">MSG_process_create</a>(const <span class="keywordtype">char</span> *name, xbt_main_func_t code,</div>
<div class="line"> <span class="keywordtype">void</span> *data, <a class="code" href="group__m__host__management.html#ga0cc2134715cceaeea82df1c1305fbecf" title="Host datatype.A location (or host) is any possible place where a process may run. Thus it is represen...">msg_host_t</a> host);</div>
</div><!-- fragment --><p>Note that it may avoid bugs later to avoid launching a worker on the master host so you probably want to remove it from the host list.</p>
<p>The <code>data</code> field of the <code>MSG_process_create</code> can be used to pass a channel name that will be private between master and workers (e.g., <code>master_name:worker_name</code>). Adding the <code>master_name</code> in the channel name will allow to easily have several masters and a worker per master on each machine. To this end, you may need to use the following functions:</p>
<div class="fragment"><div class="line"><a class="code" href="group__m__host__management.html#ga0cc2134715cceaeea82df1c1305fbecf" title="Host datatype.A location (or host) is any possible place where a process may run. Thus it is represen...">msg_host_t</a> <a class="code" href="group__m__host__management.html#ga8552d95b844f896978eca735cbd73e9f" title="Return the location on which the current process is executed.">MSG_host_self</a>(<span class="keywordtype">void</span>);</div>
<div class="line"><span class="keyword">const</span> <span class="keywordtype">char</span> * <a class="code" href="group__m__host__management.html#gad4daf43406f4dce8caae5310cdf40bde" title="Return the name of the msg_host_t.">MSG_host_get_name</a>(<a class="code" href="group__m__host__management.html#ga0cc2134715cceaeea82df1c1305fbecf" title="Host datatype.A location (or host) is any possible place where a process may run. Thus it is represen...">msg_host_t</a> host);</div>
<div class="line"><a class="code" href="structs__smx__process.html" title="Process datatype.">msg_process_t</a> <a class="code" href="group__m__process__management.html#gaf76911391b5116a7d1775e74185e64d5" title="Return the current process.">MSG_process_self</a>(<span class="keywordtype">void</span>);</div>
<div class="line"><span class="keywordtype">void</span> * <a class="code" href="group__m__process__management.html#ga2a510fa3957f963a04eaf6b483bfc6a9" title="Returns the user data of a process.">MSG_process_get_data</a>(<a class="code" href="structs__smx__process.html" title="Process datatype.">msg_process_t</a> process);</div>
</div><!-- fragment --><p>Again, you should check the <a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">online documentation</a> for more information. If you are not too much familiar with string manipulation in C, you may want to use the following functions</p>
<div class="fragment"><div class="line"><span class="keywordtype">char</span> *strcpy(<span class="keywordtype">char</span> *dest, <span class="keyword">const</span> <span class="keywordtype">char</span> *src);</div>
<div class="line"><span class="keywordtype">char</span> *strcat(<span class="keywordtype">char</span> *dest, <span class="keyword">const</span> <span class="keywordtype">char</span> *src);</div>
</div><!-- fragment --><h2>Setting up a Time Limit Mechanism</h2>
<p>In the current version, the number of tasks is defined in the worker arguments. Hence, tasks are created at the very beginning of the simulation. Instead, create tasks as needed and provide a time limit indicating when it stops sending tasks. To this end, you will obviously need to know what time it is (<a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">reference manual</a>):</p>
<div class="fragment"><div class="line"><span class="keywordtype">double</span> <a class="code" href="group__msg__simulation.html#gacfeb7fa281abd0ff74b0591937cbb574" title="A clock (in second).">MSG_get_clock</a>(<span class="keywordtype">void</span>);</div>
</div><!-- fragment --><p>Otherwise, a quite effective way of terminating the simulation would be to use some of the following <a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">function</a>:</p>
<div class="fragment"><div class="line"><span class="keywordtype">void</span> <a class="code" href="group__m__process__management.html#gab7ed321f5bf4776d42e1d91158d67b8d">MSG_process_kill</a>(<a class="code" href="structs__smx__process.html" title="Process datatype.">msg_process_t</a> process);</div>
<div class="line"><span class="keywordtype">int</span> <a class="code" href="group__msg__simulation.html#ga0dedd7962cd7d916406ef45c471f6134" title="Kill all running process.">MSG_process_killall</a>(<span class="keywordtype">int</span> reset_PIDs);</div>
</div><!-- fragment --><p>Anyway, the new deployment <code>deployment2.xml</code> file should thus look like this:</p>
<div class="fragment"><div class="line"><?xml version=<span class="stringliteral">'1.0'</span>?></div>
<div class="line"><!DOCTYPE platform SYSTEM <span class="stringliteral">"http://simgrid.gforge.inria.fr/simgrid.dtd"</span>></div>
<div class="line"><platform version=<span class="stringliteral">"3"</span>></div>
<div class="line"> <process host=<span class="stringliteral">"Tremblay"</span> <span class="keyword">function</span>=<span class="stringliteral">"master"</span>></div>
<div class="line"> <argument value=<span class="stringliteral">"3600"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"50000000"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"1000000"</span>/> </div>
<div class="line"> </process></div>
<div class="line"></platform></div>
</div><!-- fragment --><p>It may also be a good idea to transform most of the <code>XBT_INFO</code> into <code>XBT_DEBUG</code> (e.g., keep the information on the total number of tasks processed). These debug messages can be activated as follows:</p>
<div class="fragment"><div class="line">./masterworker2 platforms/platform.xml deployment2.xml --log=msg_test.thres:debug</div>
</div><!-- fragment --><h2>Using the Tracing Mechanism</h2>
<p>SimGrid can trace all resource consumption and the outcome can be displayed with viva as illustrated in the section "Setting up and Compiling". However, when several masters are deployed, it is hard to understand what happens.</p>
<div class="fragment"><div class="line"><?xml version=<span class="stringliteral">'1.0'</span>?></div>
<div class="line"><!DOCTYPE platform SYSTEM <span class="stringliteral">"http://simgrid.gforge.inria.fr/simgrid.dtd"</span>></div>
<div class="line"><platform version=<span class="stringliteral">"3"</span>></div>
<div class="line"> <process host=<span class="stringliteral">"Tremblay"</span> <span class="keyword">function</span>=<span class="stringliteral">"master"</span>></div>
<div class="line"> <argument value=<span class="stringliteral">"3600"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"50000000"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"10"</span>/> </div>
<div class="line"> </process></div>
<div class="line"> <process host=<span class="stringliteral">"Fafard"</span> <span class="keyword">function</span>=<span class="stringliteral">"master"</span>></div>
<div class="line"> <argument value=<span class="stringliteral">"3600"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"50000000"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"10"</span>/> </div>
<div class="line"> </process></div>
<div class="line"> <process host=<span class="stringliteral">"Jupiter"</span> <span class="keyword">function</span>=<span class="stringliteral">"master"</span>></div>
<div class="line"> <argument value=<span class="stringliteral">"3600"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"50000000"</span>/> </div>
<div class="line"> <argument value=<span class="stringliteral">"10"</span>/> </div>
<div class="line"> </process></div>
<div class="line"></platform></div>
</div><!-- fragment --><p>So let's use categories to track more precisely who does what and <a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">when</a>.</p>
<div class="fragment"><div class="line"><span class="keywordtype">void</span> <a class="code" href="group__TRACE__category.html#ga2c341df9553b432c507d4ab2a87e8685" title="Declare a new category with a random color.">TRACE_category</a>(<span class="keyword">const</span> <span class="keywordtype">char</span> *category);</div>
<div class="line"><span class="keywordtype">void</span> <a class="code" href="group__msg__task__usage.html#gaff6a660d1f3a37ff7ee3daee0e3ae3b7" title="Sets the tracing category of a task.">MSG_task_set_category</a> (<a class="code" href="group__m__task__management.html#gaa7cd89c7f542d19fd817a614d1c1cc95" title="Task datatype.A task may then be defined by a computing amount, a message size and some private data...">msg_task_t</a> task, <span class="keyword">const</span> <span class="keywordtype">char</span> *category);</div>
</div><!-- fragment --><p>The outcome can then be visualized as follows:</p>
<div class="fragment"><div class="line">./masterworker3 platforms/platform.xml deployment3.xml --cfg=tracing:yes\</div>
<div class="line"> --cfg=tracing/categorized:yes --cfg=viva/categorized:viva_cat.plist</div>
<div class="line">LANG=C; viva simgrid.trace viva_cat.plist</div>
</div><!-- fragment --><p>Right now, you should realize that nothing is behaving like you expect. Most workers are idle even though input data are ridiculous and there are several masters deployed on the platform. Using a Gantt-chart visualization may help:</p>
<div class="fragment"><div class="line">./masterworker3 platforms/platform.xml deployment3.xml --cfg=tracing:yes \</div>
<div class="line"> --cfg=tracing/msg/process:yes</div>
<div class="line">LANG=C; Paje simgrid.trace</div>
</div><!-- fragment --><p>OK, so it should now be obvious that round robin is actually very bad.</p>
<h2>Improving the Scheduling</h2>
<p>Instead of a round-robin scheduling, let's implement a first-come first-served mechanism. To this end, workers need to send a tiny request first. A possible way to implement such a request with MSG is to send on a specific channel (e.g., the name of the master name) a task with payload 0 and whose attached data is the worker name. This way, the master can keep track of which workers are idle and willing to work.</p>
<p>To know whether it has pending requests, the master can use the following <a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">function</a>:</p>
<div class="fragment"><div class="line"><span class="keywordtype">int</span> <a class="code" href="group__msg__task__usage.html#ga297e41ae363db652f2075d7e9d164bb5" title="Check if there is a communication going on in a mailbox.">MSG_task_listen</a>(<span class="keyword">const</span> <span class="keywordtype">char</span> *alias);</div>
</div><!-- fragment --><p>If so, it should get the request and push the corresponding host into a dynar so that they can later be retrieved when sending a real <a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">task</a>.</p>
<div class="fragment"><div class="line"><a class="code" href="group__XBT__dynar__cons.html#gac826571988d2b63ae225e5c62ecdbc79" title="Dynar data type (opaque type)">xbt_dynar_t</a> <a class="code" href="group__XBT__dynar__cons.html#ga69ab371f667dfe5aa51fb612b9afaca5" title="Constructor.">xbt_dynar_new</a>(<span class="keyword">const</span> <span class="keywordtype">unsigned</span> <span class="keywordtype">long</span> elm_size,</div>
<div class="line"> void_f_pvoid_t <span class="keyword">const</span> free_f);</div>
<div class="line"><span class="keywordtype">void</span> <a class="code" href="group__XBT__dynar__perl.html#gac89da4c2c302218759c7fe477d8eacc0" title="Add an element at the end of the dynar.">xbt_dynar_push</a>(<a class="code" href="group__XBT__dynar__cons.html#gac826571988d2b63ae225e5c62ecdbc79" title="Dynar data type (opaque type)">xbt_dynar_t</a> <span class="keyword">const</span> dynar, <span class="keyword">const</span> <span class="keywordtype">void</span> *src);</div>
<div class="line"><span class="keywordtype">void</span> <a class="code" href="group__XBT__dynar__perl.html#ga2966d02b350ef985447c4053c258a1a0" title="Get and remove the first element of the dynar.">xbt_dynar_shift</a>(<a class="code" href="group__XBT__dynar__cons.html#gac826571988d2b63ae225e5c62ecdbc79" title="Dynar data type (opaque type)">xbt_dynar_t</a> <span class="keyword">const</span> dynar, <span class="keywordtype">void</span> *<span class="keyword">const</span> dst);</div>
<div class="line"><span class="keywordtype">unsigned</span> <span class="keywordtype">long</span> <a class="code" href="group__XBT__dynar__misc.html#ga3beec123eb77ceda5b750f6f2ca48e18" title="Count of dynar's elements.">xbt_dynar_length</a>(<span class="keyword">const</span> <a class="code" href="group__XBT__dynar__cons.html#gac826571988d2b63ae225e5c62ecdbc79" title="Dynar data type (opaque type)">xbt_dynar_t</a> dynar);</div>
</div><!-- fragment --><p>As you will soon realize, with such simple mechanisms, simple deadlocks will soon appear. They can easily be removed with a simple polling mechanism, hence the need for the following <a href="http://simgrid.gforge.inria.fr/simgrid/3.8.1/ref_guide/html/index.html">function</a>:</p>
<div class="fragment"><div class="line"><a class="code" href="group__msg__simulation.html#gaf79b56c0bd3b78b539b0cb4c12e56425" title="Return code of most MSG functions.">msg_error_t</a> <a class="code" href="group__msg__task__usage.html#gab7c18061b167e38f28886512ea73be6b" title="Sleep for the specified number of seconds.">MSG_process_sleep</a>(<span class="keywordtype">double</span> nb_sec);</div>
</div><!-- fragment --><p>As you should quickly realize, on the simple previous example, it will double the throughput of the platform but will be quite ineffective when input size of the tasks is not negligible anymore.</p>
<p>From this, many things can easily be added. For example, you could:</p>
<ul>
<li>add a performance measurement mechanism;</li>
<li>enable the master to make smart scheduling choices using measurement information;</li>
<li>allow workers to have several pending requests so as to overlap communication and computations as much as possible;</li>
<li>...</li>
</ul>
<h2>Using More Elaborate Platforms</h2>
<p>SimGrid offers a rather powerful platform modeling mechanism. The <code>src/platform/</code> repository comprises a variety of platform ranging from simple ones to quite elaborated ones. Associated to a good visualization tool to ensure your simulation is meaningful, they can allow you to study to which extent your algorithm scales...</p>
<p>What is the largest number of tasks requiring 50e6 flops and 1e5 bytes that you manage to distribute and process in one hour on <code>g5k.xml</code> (you should use <code>deployment_general.xml</code>)?</p>
<h1>Points to improve for the next time</h1>
<ul>
<li>Propose equivalent exercises and skeleton in java.</li>
<li>Propose a virtualbox image with everything (simgrid, paje, viva, ...) already set up.</li>
<li>Ease the installation on mac OS X (binary installer) and windows.</li>
<li>Explain that programming in C or java and having a working development environment is a prerequisite. </li>
</ul>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="footer">Generated on Sun Nov 17 2013 21:34:46 for SimGrid by
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.1.2 </li>
</ul>
</div>
</body>
</html>
|