python多线程,多进程优选方案及实例

Python的GIL(全局解释器锁)会组织Python代码同时在多个处理器核心上运行。于是有人误认为Python不支持多线程,或无法发挥多喝硬件的优势。对于“计算密集型”(CPU-bound)程序来说,可以用multiprocessing模块实现并发,该模块不受GIL限制,可以完全利用每个核心。这样一来,处理速度就很容易提高了(大致同CPU的核心数成正比)。对于“I/O密集型”程序来说,我们既可以用multiprocessing模块来做,也可以用threading模块或concurrent.futures模块来做。实际上,使用threading模块来编写I/O密集型程序时,并不用担心由GIL所带来的开销,因为网络延迟的影响更大。

# -*- coding: utf-8 -*-
#!/usr/bin/env python 
# @Time    : 2018/6/29 11:26
# @Software: PyCharm
import time
from concurrent.futures import ThreadPoolExecutor
import threading
import multiprocessing
''' 
创建一个最大可容纳max_workers个的线程池
'''
pool1 = ThreadPoolExecutor(max_workers=300)
def return_future_result(message):
    time.sleep(3)
    return message

def go():
    for num in range(1, 10):
        # 往线程池里面加入一个task
        pool1.submit(return_future_result, ("hello"+str(num)))






if __name__ == '__main__':
    # 第一种
    pool = multiprocessing.Pool(processes=10)
    result = []
    for i in xrange(60000):
        msg = "hello %d" % (i)
        print i
        if int(i) % int(2000) == 0:
            time.sleep(2)
        result.append(pool.apply_async(return_future_result, (msg,)))

    # 第二种
    try:
        threads = []
        t1 = threading.Thread(target=return_future_result, args=(9101,))
        threads.append(t1)
        for port in (1,10):
            threads.append(threading.Thread(target=return_future_result, args=(int(port[0]),)))
        # 启动
        for n in range(len(threads)):
            threads[n].start()
    except Exception, e:
        print str(e)
    # 第三种
    for num in range(1, 1000):
        print num
        pool1.submit(return_future_result, num)
        if int(num) % int(1000) == 0:
            time.sleep(2)

 

    原文作者:代码帮
    原文地址: https://blog.csdn.net/ITLearnHall/article/details/86310469
    本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系博主进行删除。
点赞