Python multiprocessing.Manager和os.fork产生奇怪的行为

2019年7月21日 467次阅读

我的同事用他正在研究的守护程序脚本问了我的问题.他遇到了一个涉及multiprocessing.Manager的奇怪错误,我设法用以下五行重现：

import multiprocessing, os, sys
mgr = multiprocessing.Manager()
pid = os.fork()
if pid > 0:
    sys.exit(0)

在CentOS 6 Linux和Python 2.6上运行时,我收到以下错误：

Traceback (most recent call last):
  File "/usr/lib64/python2.6/multiprocessing/util.py", line 235, in _run_finalizers
    finalizer()
  File "/usr/lib64/python2.6/multiprocessing/util.py", line 174, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 576, in _finalize_manager
    if process.is_alive():
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 129, in is_alive
    assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib64/python2.6/multiprocessing/util.py", line 269, in _exit_function
    p.join()
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 117, in join
    assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib64/python2.6/multiprocessing/util.py", line 269, in _exit_function
    p.join()
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 117, in join
    assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process

我怀疑错误是由于os.fork和multiprocessing.Manager之间的某些交互,并且他应该使用多处理模块来创建新进程而不是os.fork.谁能证实这一点和/或解释发生了什么？如果我的预感是正确的,为什么这是使用os.fork的错误地方？

最佳答案 问题是Manager创建了一个进程并尝试在sys.exit上停止它.由于在fork期间(懒惰)复制进程的内存,父进程和子进程都会尝试停止进程并等待它停止.但是,作为例外,只提到父进程可以做到这一点.如果不使用os.fork,则使用multiprocessing.Process,它将生成一个新进程,该进程不会尝试在sys.exit上关闭Manager.

点赞