写在之前
基础依旧很弱,欠缺理论知识(实践就更不要说了),很多东西都只知道个大概,哎。
Future
Future对象实际是coroutine函数装饰器和IOLoop的沟通使者,有着非常重要的作用。
tornado.concurrent.Future
与 concurrent.futures.Future
类似,但感觉比其简易。以tornado.concurrent.Future
为例介绍该对象的重要方法:
-
_set_done()
和done()
: 用于设置和获取该future的完成状态,并且在_set_done()
时会顺序执行该对象的callback函数。
def _set_done(self):
self._done = True
for cb in self._callbacks:
try:
cb(self)
except Exception:
app_log.exception('Exception in callback %r for %r',
cb, self)
self._callbacks = None
-
set_result()
和result()
: 用于设置和获取该future的结果。与concurrent.futures.Future
不同,如果该tornado.concurrent.Future
没有完成时,调用result()
会抛出异常(而concurrent.futures.Future
是通过自身的condition条件来控制)。
def _check_done(self):
if not self._done:
raise Exception("DummyFuture does not support blocking for results")
-
add_done_callback
: 为该对象添加回调函数,若该future已完成则直接执行,否则添加到_callbacks
列表中在“`_set_done()“时顺序执行。
def add_done_callback(self, fn):
"""Attaches the given callback to the `Future`.
It will be invoked with the `Future` as its argument when the Future
has finished running and its result is available. In Tornado
consider using `.IOLoop.add_future` instead of calling
`add_done_callback` directly.
"""
if self._done:
fn(self)
else:
self._callbacks.append(fn)
为什么别人博客说Future
实际是@tornado.gen.coroutine
和IOLoop的沟通使者呢?再看一下这两者的相关内容。
IOLoop
简单的理解IOLoop实际上就是一个事件循环,调度处理I/O事件和callback, timeout等事件。主要介绍add_future()
和add_callback
方法(删除了部分注释,可以在源码中查看tornado.ioloop
模块)。
def add_future(self, future, callback):
"""Schedules a callback on the ``IOLoop`` when the given
`.Future` is finished.
The callback is invoked with one argument, the
`.Future`.
"""
assert is_future(future)
callback = stack_context.wrap(callback)
future.add_done_callback(
lambda future: self.add_callback(callback, future))
def add_callback(self, callback, *args, **kwargs):
if thread.get_ident() != self._thread_ident:
with self._callback_lock:
if self._closing:
return
list_empty = not self._callbacks
self._callbacks.append(functools.partial(
stack_context.wrap(callback), *args, **kwargs))
if list_empty:
self._waker.wake()
else:
if self._closing:
return
self._callbacks.append(functools.partial(
stack_context.wrap(callback), *args, **kwargs))
查看了以上的源码可得,这两个方法实际上都是(在future完成后)通过add_callback()
往IOLoop的_callbacks
里一系列回调函数,使得IOLoop在下一次循环的时候得以顺序调用这些回调函数,即把控制权重新交还给了主线程。
@tornado.gen.coroutine
和yield
一般来说@tornado.gen.coroutine
和yield对应,被该装饰器装饰的方法需要存在yield(即是一个生成器)。
第一次看到以下的示例代码时,当时感觉与yield的使用相悖的。
class GenRequestHandler(BaseHandler):
@tornado.gen.coroutine
def get(self):
http = httpclient.AsyncHTTPClient()
res = yield http.fetch('http://www.baidu.com')
self.write(res.body)
为什么yield http.fetch('http://www.baidu.com')
直接返回了fetch得到的response?对于一般的生成器不应该是先调用next()
或者send(None)
然后遇到yield
后交出控制权然后再通过send(result)
来返回结果吗(如下例)?
def gen():
print('start')
first_send = yield 1
print('first_send: %s.' % first_send)
second_send = yield 2
print('second_send: %s.' % second_send)
print('end')
if __name__ == '__main__':
try:
g = gen()
first_yielded = next(g)
print('first_yielded: %s.' % first_yielded)
second_yielded = g.send(1)
print('second_yielded: %s.' % second_yielded)
g.send(2)
except StopIteration:
print('stop')
实际上@tornado.gen.coroutine
对原函数装饰了很多额外的操作,使得异步代码写起来和同步代码一样。上代码:
def coroutine(func, replace_callback=True):
return _make_coroutine_wrapper(func, replace_callback=True)
def _make_coroutine_wrapper(func, replace_callback):
func = types.coroutine(func)
@functools.wraps(func)
def wrapper(*args, **kwargs):
future = TracebackFuture()
if replace_callback and 'callback' in kwargs:
callback = kwargs.pop('callback')
IOLoop.current().add_future(
future, lambda future: callback(future.result()))
try:
result = func(*args, **kwargs)
except (Return, StopIteration) as e:
result = _value_from_stopiteration(e)
except Exception:
future.set_exc_info(sys.exc_info())
return future
else:
if isinstance(result, GeneratorType):
try:
orig_stack_contexts = stack_context._state.contexts
yielded = next(result)
if stack_context._state.contexts is not orig_stack_contexts:
yielded = TracebackFuture()
yielded.set_exception(
stack_context.StackContextInconsistentError(
'stack_context inconsistency (probably caused '
'by yield within a "with StackContext" block)'))
except (StopIteration, Return) as e:
future.set_result(_value_from_stopiteration(e))
except Exception:
future.set_exc_info(sys.exc_info())
else:
Runner(result, future, yielded)
try:
return future
finally:
future = None
future.set_result(result)
return future
return wrapper