Master

pyspider00

data_db(存储抓取数据)

• phantomjs
• scheduler
• processor
• –phantomjs-proxy=’localhost:25555′ fetcher
• result_worker
• webui

Slave_01

pyspider01

taskdb,projectdb,resultdb(存储主要任务和脚本数据,比较重要)

• phantomjs
• processor
• –phantomjs-proxy=”localhost:25555″ fetcher

Slave_02

pyspider02

• phantomjs
• processor
• –phantomjs-proxy=”localhost:25555″ fetcher