基本情况：
一个控制节点，尾号20
两个计算节点：尾号分别为39，40
[root@osnode066020 ~]# nova service-list
+----+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| 1 | nova-consoleauth | osnode066020.test.com | internal | enabled | up | 2017-09-30T03:54:18.000000 | - |
| 2 | nova-conductor | osnode066020.test.com | internal | enabled | up | 2017-09-30T03:54:21.000000 | - |
| 4 | nova-scheduler | osnode066020.test.com | internal | enabled | up | 2017-09-30T03:54:21.000000 | - |
| 7 | nova-compute | osnode066039.test.com | nova | enabled | up | 2017-09-30T03:54:19.000000 | - |
| 8 | nova-compute | osnode066040.test.com | nova | enabled | up | 2017-09-30T03:54:14.000000 | - |
+----+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
指定在 39 节点上创建KVM，创建成功：
$ nova boot --min-count 1 --flavor m1.nano --image fa26bdb4-5bd3-403c-978a-b8c0cd3aa2a3 --availability-zone nova:osnode066039.test.com --nic net-id=3de9eb8a-a8b0-4923-8c33-1fba17b429ea jia-1
指定在 40 节点上创建KVM，创建失败：
$ nova boot --min-count 1 --flavor m1.nano --image fa26bdb4-5bd3-403c-978a-b8c0cd3aa2a3 --availability-zone nova:osnode066040.test.com --nic net-id=3de9eb8a-a8b0-4923-8c33-1fba17b429ea jia
问题：
$nova show 51a69e00-18b0-4f71-b035-d6ec114d092f
...
| fault | {"message": "Binding failed for port 4569e0d2-8958-414f-b59e-5bd83ed23bea, please check neutron logs for more information.", "code": 500, "details": " File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1781, in _do_build_and_run_instance |
| | filter_properties) |
| | File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2017, in _build_and_run_instance |
| | instance_uuid=instance.uuid, reason=six.text_type(e)) |
| | ", "created": "2017-09-30T05:50:29Z"}
$ nova list
+--------------------------------------+----------------------+---------+------------+-------------+--------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+----------------------+---------+------------+-------------+--------------------------------------+
| 51a69e00-18b0-4f71-b035-d6ec114d092f | jia | ERROR | - | NOSTATE | |
查看日志：
$ vim /var/log/neutron/server.log + 27191
$ virsh list --all
setlocale: No such file or directory
Id Name State
----------------------------------------------------
18 instance-0000001c running
26 instance-0000001b running
28 instance-00000021 running
31 instance-0000002e running
35 instance-00000017 running
- instance-00000012 shut off
- instance-00000013 shut off
$ nova boot --min-count 1 --flavor m1.nano --image fa26bdb4-5bd3-403c-978a-b8c0cd3aa2a3 --availability-zone nova:osnode066040.test.com --nic net-id=3de9eb8a-a8b0-4923-8c33-1fba17b429ea jia-40-1
| fault | {"message": "Binding failed for port dac93414-49d9-4c13-bdfe-7e0aa987d8d4, please check neutron logs for more info
rmation.", "code": 500, "details": " File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1781, in _do_build_and_run_instance |
| | filter_properties) |
| | File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2017, in _build_and_run_instance |
| | instance_uuid=instance.uuid, reason=six.text_type(e))
$ nova boot --min-count 1 --flavor m1.nano --image fa26bdb4-5bd3-403c-978a-b8c0cd3aa2a3 --availability-zone nova:osnode066040.test.com --nic net-id=3de9eb8a-a8b0-4923-8c33-1fba17b429ea jia-40-2
| fault | {"message": "Binding failed for port 14a8a0a4-0a14-4fb8-9270-b89ca82cd33e, please check neutron logs for more info
rmation.", "code": 500, "details": " File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1781, in _do_build_and_run_instance |
| | filter_properties) |
| | File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2017, in _build_and_run_instance |
| | instance_uuid=instance.uuid, reason=six.text_type(e)) |
| | ", "created": "2017-09-30T07:40:03Z"}
查看 agent, 发现40节点 linux-bridge-agent 工作异常。
[root@osnode066020 ~]# neutron agent-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+----------------------------+--------------------+-----------------------+-------------------+-------+----------------+---------------------------+
| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
+----------------------------+--------------------+-----------------------+-------------------+-------+----------------+---------------------------+
| 0b9de34b-6384-4df6-9258-65 | Linux bridge agent | osnode066039.test.com | | :-) | True | neutron-linuxbridge-agent |
| 1c7563ae58 | | | | | | |
| 3ce0cffc-a04a-4239-9432-84 | DHCP agent | osnode066020.test.com | nova | :-) | True | neutron-dhcp-agent |
| 7686adc831 | | | | | | |
| 95982d8e-ab5b- | Linux bridge agent | osnode066040.test.com | | xxx | True | neutron-linuxbridge-agent |
| 4cfd-a808-8bf757bbdfac | | | | | | |
| aa234158-fec3-4e57-93e7-97 | Metadata agent | osnode066020.test.com | | :-) | True | neutron-metadata-agent |
| 1cc49e6394 | | | | | | |
| bb3889a0-eff8-4427-b5e8-3b | L3 agent | osnode066020.test.com | nova | :-) | True | neutron-l3-agent |
| e8fc8f32b5 | | | | | | |
| dfefb2b8-2b0c- | Linux bridge agent | osnode066020.test.com | | :-) | True | neutron-linuxbridge-agent |
| 410f-b927-f1850bb42ea9 | | | | | | |
+----------------------------+--------------------+-----------------------+-------------------+-------+----------------+---------------------------+
[root@osnode066020 ~]# nova availability-zone-list
+--------------------------+----------------------------------------+
| Name | Status |
+--------------------------+----------------------------------------+
| internal | available |
| |- osnode066020.test.com | |
| | |- nova-conductor | enabled :-) 2017-09-30T07:37:15.000000 |
| | |- nova-consoleauth | enabled :-) 2017-09-30T07:37:14.000000 |
| | |- nova-scheduler | enabled :-) 2017-09-30T07:37:15.000000 |
| nova | available |
| |- osnode066039.test.com | |
| | |- nova-compute | enabled :-) 2017-09-30T07:37:11.000000 |
| |- osnode066040.test.com | |
| | |- nova-compute | enabled :-) 2017-09-30T07:37:15.000000 |
+--------------------------+----------------------------------------+
[root@osnode066020 ~]# nova service-list
+----+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| 1 | nova-consoleauth | osnode066020.test.com | internal | enabled | up | 2017-09-30T07:41:44.000000 | - |
| 2 | nova-conductor | osnode066020.test.com | internal | enabled | up | 2017-09-30T07:41:53.000000 | - |
| 4 | nova-scheduler | osnode066020.test.com | internal | enabled | up | 2017-09-30T07:41:45.000000 | - |
| 7 | nova-compute | osnode066039.test.com | nova | enabled | up | 2017-09-30T07:41:51.000000 | - |
| 8 | nova-compute | osnode066040.test.com | nova | enabled | up | 2017-09-30T07:41:45.000000 | - |
+----+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
最后发现是40节点linux-bridge 工作异常。
通过linux-bridge 的日志可以看到，通过 neutron agent-list 也可以看出来。
但是通 systemctl 看不出来。
linux-bridge 不断尝试重启，
日志如下：
2017-09-30 15:50:03.645 30737 ERROR neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 474, in set_rpc_timeout
2017-09-30 15:50:03.645 30737 ERROR neutron self.state_rpc):
2017-09-30 15:50:03.645 30737 ERROR neutron AttributeError: 'CommonAgentLoop' object has no attribute 'state_rpc'
2017-09-30 15:50:03.645 30737 ERROR neutron
2017-09-30 15:50:03.648 30737 INFO oslo_rootwrap.client [-] Stopping rootwrap daemon process with pid=30754
2017-09-30 15:50:06.407 30791 INFO neutron.common.config [-] Logging enabled!
2017-09-30 15:50:06.408 30791 INFO neutron.common.config [-] /usr/bin/neutron-linuxbridge-agent version 10.0.3
2017-09-30 15:50:06.408 30791 DEBUG neutron.common.config [-] command line: /usr/bin/neutron-linuxbridge-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/linuxbridge_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-linuxbridge-agent --log-file /var/log/neutron/linuxbridge-agent.log setup_logging /usr/lib/python2.7/site-packages/neutron/common/config.py:115
2017-09-30 15:50:06.408 30791 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Interface mappings: {'provider': 'eth1'}
2017-09-30 15:50:06.409 30791 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Bridge mappings: {}
2017-09-30 15:50:06.409 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'eth1'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.418 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:06.420 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', 'addr', 'show', 'to', '10.253.66.40'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.428 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:06.429 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['bridge', 'fdb', 'help'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.436 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'vxlan-1'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.444 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-d', 'link', 'list'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.436 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'vxlan-1'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.444 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-d', 'link', 'list'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.451 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:06.452 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'vxlan-1'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:06.459 30791 DEBUG neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Creating vxlan interface vxlan-1 for VNI 1 ensure_vxlan /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py:288
2017-09-30 15:50:06.459 30791 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'link', 'add', 'vxlan-1', 'type', 'vxlan', 'id', '1', 'dev', 'brqb7589910-ee'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:113
2017-09-30 15:50:06.466 30791 DEBUG oslo_rootwrap.client [-] Popen for ['sudo', 'neutron-rootwrap-daemon', '/etc/neutron/rootwrap.conf'] command has been instantiated _initialize /usr/lib/python2.7/site-packages/oslo_rootwrap/client.py:68
2017-09-30 15:50:06.815 30791 INFO oslo_rootwrap.client [-] Spawned new rootwrap daemon process with pid=30809
2017-09-30 15:50:06.833 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:06.834 30791 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['sysctl', '-w', 'net.ipv6.conf.vxlan-1.disable_ipv6=1'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:113
2017-09-30 15:50:06.984 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:06.985 30791 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'link', 'set', 'vxlan-1', 'up'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:113
2017-09-30 15:50:06.991 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:06.991 30791 DEBUG neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Done creating vxlan interface vxlan-1 ensure_vxlan /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py:313
2017-09-30 15:50:06.991 30791 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['bridge', 'fdb', 'append', '00:00:00:00:00:00', 'dev', 'vxlan-1', 'dst', '1.1.1.1'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:113
2017-09-30 15:50:06.997 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:06.998 30791 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'vxlan-1'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:97
2017-09-30 15:50:07.007 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:07.008 30791 DEBUG neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Deleting interface vxlan-1 delete_interface /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py:596
2017-09-30 15:50:07.008 30791 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'link', 'set', 'vxlan-1', 'down'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:113
2017-09-30 15:50:07.014 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:07.015 30791 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'link', 'delete', 'vxlan-1'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:113
2017-09-30 15:50:07.031 30791 DEBUG neutron.agent.linux.utils [-] Exit code: 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:158
2017-09-30 15:50:07.032 30791 DEBUG neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Done deleting interface vxlan-1 delete_interface /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py:599
2017-09-30 15:50:07.032 30791 DEBUG neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Using unicast_flooding VXLAN mode check_vxlan_support /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py:675
2017-09-30 15:50:07.032 30791 DEBUG neutron.callbacks.manager [-] Subscribe: <function init_handler at 0x5361410> Linux bridge agent after_init subscribe /usr/lib/python2.7/site-packages/neutron/callbacks/manager.py:42
2017-09-30 15:50:07.034 30791 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Agent initialized successfully, now running...
2017-09-30 15:50:07.034 30791 DEBUG oslo_concurrency.lockutils [-] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
2017-09-30 15:50:07.035 30791 DEBUG oslo_concurrency.lockutils [-] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
2017-09-30 15:50:07.036 30791 DEBUG oslo_service.service [-] Full set of CONF: _wait_for_exit_or_signal /usr/lib/python2.7/site-packages/oslo_service/service.py:303
2017-09-30 15:50:07.036 30791 DEBUG oslo_service.service [-] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2726
2017-09-30 15:50:07.036 30791 DEBUG oslo_service.service [-] Configuration options gathered from: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2727
2017-09-30 15:50:07.037 30791 DEBUG oslo_service.service [-] command line args: ['--config-file', '/usr/share/neutron/neutron-dist.conf', '--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/linuxbridge_agent.ini', '--config-dir', '/etc/neutron/conf.d/common', '--config-dir', '/etc/neutron/conf.d/neutron-linuxbridge-agent', '--log-file', '/var/log/neutron/linuxbridge-agent.log'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2728
2017-09-30 15:50:07.037 30791 DEBUG oslo_service.service [-] config files: ['/usr/share/neutron/neutron-dist.conf', '/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/linuxbridge_agent.ini'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2730
2017-09-30 15:50:07.038 30791 DEBUG oslo_service.service [-] ================================================================================ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2731
2017-09-30 15:50:07.038 30791 DEBUG oslo_service.service [-] agent_down_time = 75 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.038 30791 DEBUG oslo_service.service [-] allow_bulk = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.039 30791 DEBUG oslo_service.service [-] allow_overlapping_ips = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.039 30791 DEBUG oslo_service.service [-] api_extensions_path = log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.040 30791 DEBUG oslo_service.service [-] api_paste_config = /usr/share/neutron/api-paste.ini log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.040 30791 DEBUG oslo_service.service [-] auth_strategy = keystone log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.040 30791 DEBUG oslo_service.service [-] backdoor_port = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.041 30791 DEBUG oslo_service.service [-] backdoor_socket = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.041 30791 DEBUG oslo_service.service [-] backlog = 4096 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2740
2017-09-30 15:50:07.108 30791 DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /var/lib/neutron/tmp log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2748
2017-09-30 15:50:07.108 30791 DEBUG oslo_service.service [-] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2750
2017-09-30 15:50:07.109 30791 DEBUG neutron.agent.securitygroups_rpc [req-41e7285e-4ca4-4470-bedb-f06d11651ab4 - - - - -] Init firewall settings (driver=neutron.agent.linux.iptables_firewall.IptablesFirewallDriver) init_firewall /usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py:82
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service [req-41e7285e-4ca4-4470-bedb-f06d11651ab4 - - - - -] Error starting thread.
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service Traceback (most recent call last):
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 722, in run_service
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service service.start()
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service return f(*args, **kwargs)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 88, in start
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service self.setup_rpc()
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service return f(*args, **kwargs)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 155, in setup_rpc
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service self.context, self.sg_plugin_rpc, defer_refresh_firewall=True)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py", line 77, in __init__
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service self.init_firewall(defer_refresh_firewall, integration_bridge)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py", line 86, in init_firewall
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service firewall_class = firewall.load_firewall_driver_class(firewall_driver)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/neutron/agent/firewall.py", line 48, in load_firewall_driver_class
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service 'neutron.agent.firewall_drivers', driver)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 425, in load_class_by_alias_or_classname
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service class_to_load = importutils.import_class(name)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_utils/importutils.py", line 30, in import_class
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service __import__(mod_str)
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_firewall.py", line 49
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service deef get_hybrid_port_name(port_name):
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service ^
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service SyntaxError: invalid syntax
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service SyntaxError: invalid syntax
2017-09-30 15:50:07.112 30791 ERROR oslo_service.service
2017-09-30 15:50:07.114 30791 INFO neutron.plugins.ml2.drivers.agent._common_agent [-] Stopping Linux bridge agent agent.
2017-09-30 15:50:07.115 30791 CRITICAL neutron [-] AttributeError: 'CommonAgentLoop' object has no attribute 'state_rpc'
2017-09-30 15:50:07.115 30791 ERROR neutron Traceback (most recent call last):
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/bin/neutron-linuxbridge-agent", line 10, in <module>
2017-09-30 15:50:07.115 30791 ERROR neutron sys.exit(main())
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/neutron/cmd/eventlet/plugins/linuxbridge_neutron_agent.py", line 21, in main
2017-09-30 15:50:07.115 30791 ERROR neutron agent_main.main()
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 944, in main
2017-09-30 15:50:07.115 30791 ERROR neutron launcher.wait()
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 329, in wait
2017-09-30 15:50:07.115 30791 ERROR neutron status, signo = self._wait_for_exit_or_signal()
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 314, in _wait_for_exit_or_signal
2017-09-30 15:50:07.115 30791 ERROR neutron self.stop()
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 228, in stop
2017-09-30 15:50:07.115 30791 ERROR neutron self.services.stop()
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 688, in stop
2017-09-30 15:50:07.115 30791 ERROR neutron service.stop()
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2017-09-30 15:50:07.115 30791 ERROR neutron return f(*args, **kwargs)
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 119, in stop
2017-09-30 15:50:07.115 30791 ERROR neutron self.set_rpc_timeout(self.quitting_rpc_timeout)
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2017-09-30 15:50:07.115 30791 ERROR neutron return f(*args, **kwargs)
2017-09-30 15:50:07.115 30791 ERROR neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 474, in set_rpc_timeout
2017-09-30 15:50:07.115 30791 ERROR neutron self.state_rpc):
2017-09-30 15:50:07.115 30791 ERROR neutron AttributeError: 'CommonAgentLoop' object has no attribute 'state_rpc'
2017-09-30 15:50:07.115 30791 ERROR neutron
2017-09-30 15:50:07.118 30791 INFO oslo_rootwrap.client [-] Stopping rootwrap daemon process with pid=30809
2017-09-30 15:50:09.663 30847 INFO neutron.common.config [-] Logging enabled!
2017-09-30 15:50:09.686 30847 INFO neutron.common.config [-] /usr/bin/neutron-linuxbridge-agent version 10.0.3
2017-09-30 15:50:09.687 30847 DEBUG neutron.common.config [-] command line: /usr/bin/neutron-linuxbridge-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/linuxbridge_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-linuxbridge-agent --log-file /var/log/neutron/linuxbridge-agent.log setup_logging /usr/lib/python2.7/site-packages/neutron/common/config.py:115
用 systemctl 查看，发现 linuxbridge-agent 进程id， 一直变换：
[root@osnode066040 ~]# systemctl status neutron-linuxbridge-agent.service
● neutron-linuxbridge-agent.service - OpenStack Neutron Linux Bridge Agent
Loaded: loaded (/usr/lib/systemd/system/neutron-linuxbridge-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2017-09-30 15:49:29 CST; 978ms ago
Process: 30219 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
Main PID: 30226 (neutron-linuxbr)
CGroup: /system.slice/neutron-linuxbridge-agent.service
└─30226 /usr/bin/python2 /usr/bin/neutron-linuxbridge-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neu...
Sep 30 15:49:29 osnode066040.test.com systemd[1]: Starting OpenStack Neutron Linux Bridge Agent...
Sep 30 15:49:29 osnode066040.test.com neutron-enable-bridge-firewall.sh[30219]: net.bridge.bridge-nf-call-arptables = 1
Sep 30 15:49:29 osnode066040.test.com neutron-enable-bridge-firewall.sh[30219]: net.bridge.bridge-nf-call-iptables = 1
Sep 30 15:49:29 osnode066040.test.com neutron-enable-bridge-firewall.sh[30219]: net.bridge.bridge-nf-call-ip6tables = 1
Sep 30 15:49:29 osnode066040.test.com systemd[1]: Started OpenStack Neutron Linux Bridge Agent.
Sep 30 15:49:29 osnode066040.test.com neutron-linuxbridge-agent[30226]: Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward ...ports.
Hint: Some lines were ellipsized, use -l to show in full.
[root@osnode066040 ~]# systemctl status neutron-linuxbridge-agent.service
● neutron-linuxbridge-agent.service - OpenStack Neutron Linux Bridge Agent
Loaded: loaded (/usr/lib/systemd/system/neutron-linuxbridge-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2017-09-30 15:49:32 CST; 1s ago
Process: 30274 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
Main PID: 30281 (neutron-linuxbr)
CGroup: /system.slice/neutron-linuxbridge-agent.service
└─30281 /usr/bin/python2 /usr/bin/neutron-linuxbridge-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neu...
Sep 30 15:49:32 osnode066040.test.com systemd[1]: Starting OpenStack Neutron Linux Bridge Agent...
Sep 30 15:49:32 osnode066040.test.com neutron-enable-bridge-firewall.sh[30274]: net.bridge.bridge-nf-call-arptables = 1
Sep 30 15:49:32 osnode066040.test.com neutron-enable-bridge-firewall.sh[30274]: net.bridge.bridge-nf-call-iptables = 1
Sep 30 15:49:32 osnode066040.test.com neutron-enable-bridge-firewall.sh[30274]: net.bridge.bridge-nf-call-ip6tables = 1
Sep 30 15:49:32 osnode066040.test.com systemd[1]: Started OpenStack Neutron Linux Bridge Agent.
Sep 30 15:49:33 osnode066040.test.com neutron-linuxbridge-agent[30281]: Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward ...ports.
Sep 30 15:49:34 osnode066040.test.com neutron-linuxbridge-agent[30281]: Option "verbose" from group "DEFAULT" is deprecated for removal. Its valu...uture.
Hint: Some lines were ellipsized, use -l to show in full.
[root@osnode066040 ~]# systemctl status neutron-linuxbridge-agent.service
● neutron-linuxbridge-agent.service - OpenStack Neutron Linux Bridge Agent
Loaded: loaded (/usr/lib/systemd/system/neutron-linuxbridge-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2017-09-30 15:50:38 CST; 10s ago
Process: 31348 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
Main PID: 31356 (neutron-linuxbr)
CGroup: /system.slice/neutron-linuxbridge-agent.service
├─31356 /usr/bin/python2 /usr/bin/neutron-linuxbridge-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neu...
├─31372 sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
└─31373 /usr/bin/python2 /usr/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
Sep 30 15:50:38 osnode066040.test.com neutron-enable-bridge-firewall.sh[31348]: net.bridge.bridge-nf-call-arptables = 1
Sep 30 15:50:38 osnode066040.test.com neutron-enable-bridge-firewall.sh[31348]: net.bridge.bridge-nf-call-iptables = 1
Sep 30 15:50:38 osnode066040.test.com neutron-enable-bridge-firewall.sh[31348]: net.bridge.bridge-nf-call-ip6tables = 1
Sep 30 15:50:38 osnode066040.test.com systemd[1]: Started OpenStack Neutron Linux Bridge Agent.
Sep 30 15:50:39 osnode066040.test.com neutron-linuxbridge-agent[31356]: Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward ...ports.
Sep 30 15:50:40 osnode066040.test.com neutron-linuxbridge-agent[31356]: Option "verbose" from group "DEFAULT" is deprecated for removal. Its valu...uture.
Sep 30 15:50:40 osnode066040.test.com neutron-linuxbridge-agent[31356]: Option "notification_driver" from group "DEFAULT" is deprecated. Use optio...ions".
Sep 30 15:50:41 osnode066040.test.com sudo[31372]: neutron : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/bin/neutron-rootwrap-daemon /etc/neutro...rap.conf
Sep 30 15:50:41 osnode066040.test.com neutron-linuxbridge-agent[31356]: /usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py:200: FutureWarning...
Sep 30 15:50:41 osnode066040.test.com neutron-linuxbridge-agent[31356]: access_policy)
Hint: Some lines were ellipsized, use -l to show in full.
定位到语法错误
File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_firewall.py", line 49
deef get_hybrid_port_name(port_name):
修改如下：
49 def get_hybrid_port_name(port_name):
总结
- 同一个节点，可以既有nova的服务，也可以有neutron的服务；
- 看日志可以看出一些问题；
- neutron-linuxbrideg-agent 有自动重启机制。如果程序有问题(比如代码中出现语法错误)，程序会尝试自动多次启动
- 遇到程序反复重启的情况，只看进程号，或者通过 systemctl 来查看，不容易发现问题。
扩展阅读
http://happylab.blog.51cto.com/1730296/1737842
openstack运维实战系列(十)之nova指定compute节点和IP地址
简介：
3. 在指定的compute节点启动虚拟机，并指定固定ip地址
[root@controller ~]# nova boot --flavor 10 \ #flavor名字
--image 37aaedc7-6fe6-4fc8-b110-408d166b8e51 \ #镜像id
--key-name KEY \ #KEY名字
--availability-zone ChuangYiYuanZone1:ChuangYiYuan_10_16_2_11 \ #在制定的区域和主机名启动instance
--security-groups 663468d9-73b1-4b04-8d4c-dac1bf21a94d \ #安全组
--nic net-id=99c68a93-336a-4605-aa78-343d41ca1206,v4-fixed-ip=192.168.100.200 happyblog_blog_51cto_com #指定网络和IP
Failed to bind port on host
https://ask.openstack.org/en/question/92893/failed-to-bind-port-on-host/
简介：
题主描述了自己的问题，也提出了排查手段，还是挺有用的。
neutron agent-list
neutron net-list
nova service-list
brctl show
https://bugs.launchpad.net/neutron/+bug/1527061
Nova should not throw exception when port binding fails for Ironic
简介：
neutron 的bug 管理平台。
Neutron server says Failed on bind port,while launching VM
https://bugs.launchpad.net/neutron/+bug/1374814
简介：
neutron 的bug 管理平台。
[Neutron Error] CreateVM, Failed to bind port
https://ask.openstack.org/en/question/101636/neutron-error-createvm-failed-to-bind-port/
简介：
可以参考一下。
