/usr/share/doc/twisted-doc/howto/gendefer.html is in twisted-doc 11.1.0-1ubuntu2.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 | <?xml version="1.0" encoding="utf-8"?><!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Twisted Documentation: Generating Deferreds</title>
<link href="stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body bgcolor="white">
<h1 class="title">Generating Deferreds</h1>
<div class="toc"><ol><li><a href="#auto0">Class overview</a></li><ul><li><a href="#auto1">Basic Callback Functions</a></li></ul><li><a href="#auto2">What Deferreds don't do: make your code asynchronous</a></li><li><a href="#auto3">Advanced Processing Chain Control</a></li><li><a href="#auto4">Returning Deferreds from synchronous functions</a></li><li><a href="#auto5">Integrating blocking code with Twisted</a></li><li><a href="#auto6">Possible sources of error</a></li><ul><li><a href="#auto7">Firing Deferreds more than once is impossible</a></li><li><a href="#auto8">Synchronous callback execution</a></li></ul></ol></div>
<div class="content">
<span/>
<p><code class="API"><a href="http://twistedmatrix.com/documents/11.1.0/api/twisted.internet.defer.Deferred.html" title="twisted.internet.defer.Deferred">Deferred</a></code> objects are
signals that a function you have called does not yet have the data you want
available. When a function returns a Deferred object, your calling function
attaches callbacks to it to handle the data when available.</p>
<p>This document addresses the other half of the question: writing functions
that return Deferreds, that is, constructing Deferred objects, arranging for
them to be returned immediately without blocking until data is available, and
firing their callbacks when the data is available.</p>
<p>This document assumes that you are familiar with the asynchronous model used
by Twisted, and with <a href="defer.html" shape="rect">using deferreds returned by functions</a>
.</p>
<a name="class" shape="rect"/>
<h2>Class overview<a name="auto0"/></h2>
<p>This is an overview API reference for Deferred from the point of creating a
Deferred and firing its callbacks and errbacks. It is not meant to be a
substitute for the docstrings in the Deferred class, but can provide
guidelines for its use.</p>
<p>There is a parallel overview of functions used by calling function which
the Deferred is returned to at <a href="defer.html#class" shape="rect">Using Deferreds</a>.</p>
<h3>Basic Callback Functions<a name="auto1"/></h3>
<ul>
<li>
<code class="py-prototype">callback(result)</code>
<p>Run success callbacks with the given result. <em>This
can only be run once.</em> Later calls to this or
<code>errback</code> will raise <code class="API"><a href="http://twistedmatrix.com/documents/11.1.0/api/twisted.internet.defer.AlreadyCalledError.html" title="twisted.internet.defer.AlreadyCalledError">twisted.internet.defer.AlreadyCalledError</a></code>.
If further callbacks or errbacks are added after this
point, addCallbacks will run the callbacks immediately.</p>
</li>
<li>
<code class="py-prototype">errback(failure)</code>
<p>Run error callbacks with the given failure. <em>This can
only be run once.</em> Later calls to this or
<code>callback</code> will raise <code class="API"><a href="http://twistedmatrix.com/documents/11.1.0/api/twisted.internet.defer.AlreadyCalledError.html" title="twisted.internet.defer.AlreadyCalledError">twisted.internet.defer.AlreadyCalledError</a></code>.
If further callbacks or errbacks are added after this
point, addCallbacks will run the callbacks immediately.</p>
</li>
</ul>
<h2>What Deferreds don't do: make your code asynchronous<a name="auto2"/></h2>
<p><em>Deferreds do not make the code magically not block.</em></p>
<p>Let's take this function as an example:</p>
<pre class="python"><p class="py-linenumber"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
</p><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">defer</span>
<span class="py-src-variable">TARGET</span> = <span class="py-src-number">10000</span>
<span class="py-src-keyword">def</span> <span class="py-src-identifier">largeFibonnaciNumber</span>():
<span class="py-src-comment"># create a Deferred object to return:</span>
<span class="py-src-variable">d</span> = <span class="py-src-variable">defer</span>.<span class="py-src-variable">Deferred</span>()
<span class="py-src-comment"># calculate the ten thousandth Fibonnaci number</span>
<span class="py-src-variable">first</span> = <span class="py-src-number">0</span>
<span class="py-src-variable">second</span> = <span class="py-src-number">1</span>
<span class="py-src-keyword">for</span> <span class="py-src-variable">i</span> <span class="py-src-keyword">in</span> <span class="py-src-variable">xrange</span>(<span class="py-src-variable">TARGET</span> - <span class="py-src-number">1</span>):
<span class="py-src-variable">new</span> = <span class="py-src-variable">first</span> + <span class="py-src-variable">second</span>
<span class="py-src-variable">first</span> = <span class="py-src-variable">second</span>
<span class="py-src-variable">second</span> = <span class="py-src-variable">new</span>
<span class="py-src-keyword">if</span> <span class="py-src-variable">i</span> % <span class="py-src-number">100</span> == <span class="py-src-number">0</span>:
<span class="py-src-keyword">print</span> <span class="py-src-string">"Progress: calculating the %dth Fibonnaci number"</span> % <span class="py-src-variable">i</span>
<span class="py-src-comment"># give the Deferred the answer to pass to the callbacks:</span>
<span class="py-src-variable">d</span>.<span class="py-src-variable">callback</span>(<span class="py-src-variable">second</span>)
<span class="py-src-comment"># return the Deferred with the answer:</span>
<span class="py-src-keyword">return</span> <span class="py-src-variable">d</span>
<span class="py-src-keyword">import</span> <span class="py-src-variable">time</span>
<span class="py-src-variable">timeBefore</span> = <span class="py-src-variable">time</span>.<span class="py-src-variable">time</span>()
<span class="py-src-comment"># call the function and get our Deferred</span>
<span class="py-src-variable">d</span> = <span class="py-src-variable">largeFibonnaciNumber</span>()
<span class="py-src-variable">timeAfter</span> = <span class="py-src-variable">time</span>.<span class="py-src-variable">time</span>()
<span class="py-src-keyword">print</span> <span class="py-src-string">"Total time taken for largeFibonnaciNumber call: %0.3f seconds"</span> %
(<span class="py-src-variable">timeAfter</span> - <span class="py-src-variable">timeBefore</span>)
<span class="py-src-comment"># add a callback to it to print the number</span>
<span class="py-src-keyword">def</span> <span class="py-src-identifier">printNumber</span>(<span class="py-src-parameter">number</span>):
<span class="py-src-keyword">print</span> <span class="py-src-string">"The %dth Fibonacci number is %d"</span> % (<span class="py-src-variable">TARGET</span>, <span class="py-src-variable">number</span>)
<span class="py-src-keyword">print</span> <span class="py-src-string">"Adding the callback now."</span>
<span class="py-src-variable">d</span>.<span class="py-src-variable">addCallback</span>(<span class="py-src-variable">printNumber</span>)
</pre>
<p>You will notice that despite creating a Deferred in the <code>largeFibonnaciNumber</code> function, these things happened:</p>
<ul>
<li>the "Total time taken for largeFibonnaciNumber call" output
shows that the function did not return immediately as asynchronous functions
are expected to do; and</li>
<li>rather than the callback being added before the result was available and
called after the result is available, it isn't even added until after the
calculation has been completed.</li>
</ul>
<p> The function completed its calculation before returning, blocking the
process until it had finished, which is exactly what asynchronous functions
are not meant to do. Deferreds are not a non-blocking talisman: they are a
signal for asynchronous functions to <em>use</em> to pass results onto
callbacks, but using them does not guarantee that you have an asynchronous
function.</p>
<h2>Advanced Processing Chain Control<a name="auto3"/></h2>
<ul>
<li>
<code class="py-prototype">pause()</code>
<p>Cease calling any methods as they are added, and do not
respond to <code>callback</code>, until
<code>self.unpause()</code> is called.</p>
</li>
<li>
<code class="py-prototype">unpause()</code>
<p>If <code>callback</code> has been called on this
Deferred already, call all the callbacks that have been
added to this Deferred since <code>pause</code> was
called.</p>
<p>Whether it was called or not, this will put this
Deferred in a state where further calls to
<code>addCallbacks</code> or <code>callback</code> will
work as normal.</p>
</li>
</ul>
<h2>Returning Deferreds from synchronous functions<a name="auto4"/></h2>
<p>Sometimes you might wish to return a Deferred from a synchronous function.
There are several reasons why, the major two are maintaining API compatibility
with another version of your function which returns a Deferred, or allowing
for the possiblity that in the future your function might need to be
asynchronous.</p>
<p>In the <a href="defer.html" shape="rect">Using Deferreds</a> reference, we gave the
following example of a synchronous function:</p>
<div class="py-listing"><pre><p class="py-linenumber">1
2
3
4
5
</p><span class="py-src-keyword">def</span> <span class="py-src-identifier">synchronousIsValidUser</span>(<span class="py-src-parameter">user</span>):
<span class="py-src-string">'''
Return true if user is a valid user, false otherwise
'''</span>
<span class="py-src-keyword">return</span> <span class="py-src-variable">user</span> <span class="py-src-keyword">in</span> [<span class="py-src-string">"Alice"</span>, <span class="py-src-string">"Angus"</span>, <span class="py-src-string">"Agnes"</span>]
</pre><div class="caption">Source listing - <a href="listings/deferred/synch-validation.py"><span class="filename">listings/deferred/synch-validation.py</span></a></div></div>
<p>While we can require that callers of our function wrap our synchronous
result in a Deferred using <code class="API"><a href="http://twistedmatrix.com/documents/11.1.0/api/twisted.internet.defer.maybeDeferred.html" title="twisted.internet.defer.maybeDeferred">maybeDeferred</a></code>, for the sake of API
compatibility it is better to return a Deferred ourselves using <code class="API"><a href="http://twistedmatrix.com/documents/11.1.0/api/twisted.internet.defer.succeed.html" title="twisted.internet.defer.succeed">defer.succeed</a></code>:</p>
<pre class="python"><p class="py-linenumber"> 1
2
3
4
5
6
7
8
9
10
11
12
</p><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">defer</span>
<span class="py-src-keyword">def</span> <span class="py-src-identifier">immediateIsValidUser</span>(<span class="py-src-parameter">user</span>):
<span class="py-src-string">'''
Returns a Deferred resulting in true if user is a valid user, false
otherwise
'''</span>
<span class="py-src-variable">result</span> = <span class="py-src-variable">user</span> <span class="py-src-keyword">in</span> [<span class="py-src-string">"Alice"</span>, <span class="py-src-string">"Angus"</span>, <span class="py-src-string">"Agnes"</span>]
<span class="py-src-comment"># return a Deferred object already called back with the value of result</span>
<span class="py-src-keyword">return</span> <span class="py-src-variable">defer</span>.<span class="py-src-variable">succeed</span>(<span class="py-src-variable">result</span>)
</pre>
<p>There is an equivalent <code class="API"><a href="http://twistedmatrix.com/documents/11.1.0/api/twisted.internet.defer.fail.html" title="twisted.internet.defer.fail">defer.fail</a></code> method to return a Deferred with the
errback chain already fired.</p>
<h2>Integrating blocking code with Twisted<a name="auto5"/></h2>
<p>At some point, you are likely to need to call a blocking function: many
functions in third party libraries will have long running blocking functions.
There is no way to 'force' a function to be asynchronous: it must be written
that way specifically. When using Twisted, your own code should be
asynchronous, but there is no way to make third party functions asynchronous
other than rewriting them.</p>
<p>In this case, Twisted provides the ability to run the blocking code in a
separate thread rather than letting it block your application. The <code class="API"><a href="http://twistedmatrix.com/documents/11.1.0/api/twisted.internet.threads.deferToThread.html" title="twisted.internet.threads.deferToThread">twisted.internet.threads.deferToThread</a></code> function will set up
a thread to run your blocking function, return a Deferred and later fire that
Deferred when the thread completes.</p>
<p>Let's assume our <code class="python">largeFibonnaciNumber</code> function
from above is in a third party library (returning the result of the
calculation, not a Deferred) and is not easily modifiable to be finished in
discrete blocks. This example shows it being called in a thread, unlike in the
earlier section we'll see that the operation does not block our entire
program:</p>
<pre class="python"><p class="py-linenumber"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
</p><span class="py-src-keyword">def</span> <span class="py-src-identifier">largeFibonnaciNumber</span>():
<span class="py-src-string">"""
Represent a long running blocking function by calculating
the TARGETth Fibonnaci number
"""</span>
<span class="py-src-variable">TARGET</span> = <span class="py-src-number">10000</span>
<span class="py-src-variable">first</span> = <span class="py-src-number">0</span>
<span class="py-src-variable">second</span> = <span class="py-src-number">1</span>
<span class="py-src-keyword">for</span> <span class="py-src-variable">i</span> <span class="py-src-keyword">in</span> <span class="py-src-variable">xrange</span>(<span class="py-src-variable">TARGET</span> - <span class="py-src-number">1</span>):
<span class="py-src-variable">new</span> = <span class="py-src-variable">first</span> + <span class="py-src-variable">second</span>
<span class="py-src-variable">first</span> = <span class="py-src-variable">second</span>
<span class="py-src-variable">second</span> = <span class="py-src-variable">new</span>
<span class="py-src-keyword">return</span> <span class="py-src-variable">second</span>
<span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">internet</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">threads</span>, <span class="py-src-variable">reactor</span>
<span class="py-src-keyword">def</span> <span class="py-src-identifier">fibonacciCallback</span>(<span class="py-src-parameter">result</span>):
<span class="py-src-string">"""
Callback which manages the largeFibonnaciNumber result by
printing it out
"""</span>
<span class="py-src-keyword">print</span> <span class="py-src-string">"largeFibonnaciNumber result ="</span>, <span class="py-src-variable">result</span>
<span class="py-src-comment"># make sure the reactor stops after the callback chain finishes,</span>
<span class="py-src-comment"># just so that this example terminates</span>
<span class="py-src-variable">reactor</span>.<span class="py-src-variable">stop</span>()
<span class="py-src-keyword">def</span> <span class="py-src-identifier">run</span>():
<span class="py-src-string">"""
Run a series of operations, deferring the largeFibonnaciNumber
operation to a thread and performing some other operations after
adding the callback
"""</span>
<span class="py-src-comment"># get our Deferred which will be called with the largeFibonnaciNumber result</span>
<span class="py-src-variable">d</span> = <span class="py-src-variable">threads</span>.<span class="py-src-variable">deferToThread</span>(<span class="py-src-variable">largeFibonnaciNumber</span>)
<span class="py-src-comment"># add our callback to print it out</span>
<span class="py-src-variable">d</span>.<span class="py-src-variable">addCallback</span>(<span class="py-src-variable">fibonacciCallback</span>)
<span class="py-src-keyword">print</span> <span class="py-src-string">"1st line after the addition of the callback"</span>
<span class="py-src-keyword">print</span> <span class="py-src-string">"2nd line after the addition of the callback"</span>
<span class="py-src-keyword">if</span> <span class="py-src-variable">__name__</span> == <span class="py-src-string">'__main__'</span>:
<span class="py-src-variable">run</span>()
<span class="py-src-variable">reactor</span>.<span class="py-src-variable">run</span>()
</pre>
<h2>Possible sources of error<a name="auto6"/></h2>
<p>Deferreds greatly simplify the process of writing asynchronous code by
providing a standard for registering callbacks, but there are some subtle and
sometimes confusing rules that you need to follow if you are going to use
them. This mostly applies to people who are writing new systems that use
Deferreds internally, and not writers of applications that just add callbacks
to Deferreds produced and processed by other systems. Nevertheless, it is good
to know.</p>
<h3>Firing Deferreds more than once is impossible<a name="auto7"/></h3>
<p>Deferreds are one-shot. You can only call <code>Deferred.callback</code> or <code>Deferred.errback</code> once. The processing chain continues each time
you add new callbacks to an already-called-back-to Deferred.</p>
<h3>Synchronous callback execution<a name="auto8"/></h3>
<p>If a Deferred already has a result available, <code>addCallback</code> <strong>may</strong> call the callback synchronously: that is, immediately
after it's been added. In situations where callbacks modify state, it is
might be desirable for the chain of processing to halt until all callbacks are
added. For this, it is possible to <code>pause</code> and <code>unpause</code>
a Deferred's processing chain while you are adding lots of callbacks.</p>
<p>Be careful when you use these methods! If you <code>pause</code> a
Deferred, it is <em>your</em> responsibility to make sure that you unpause it.
The function adding the callbacks must unpause a paused Deferred, it should <em>never</em> be the responsibility of the code that actually fires the
callback chain by calling <code>callback</code> or <code>errback</code> as
this would negate its usefulness!</p>
</div>
<p><a href="index.html">Index</a></p>
<span class="version">Version: 11.1.0</span>
</body>
</html>
|