{"body":"RROR neutron.agent.linux.utils [-] Rootwrap error running command: ['ip', 'netns', 'exec', 'qdhcp-0e5dfd3d-7a4a-4be4-97cd-8aa3acb9a959', 'dhcp_release', 'tap34bc5e2b-55', '10.1.0.52', '52:54:00:73:b2:81']: multiprocessing.managers.RemoteError: \nDec 09 20:08:20.599306 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: DEBUG neutron.agent.dhcp.agent [None req-5707c7d9-87c8-4a6e-aa0b-9c7198f071ee None None] Resync event has been scheduled {{(pid=20208) _periodic_resync_helper /opt/stack/neutron/neutron/agent/dhcp/agent.py:277}}\nDec 09 20:08:20.600051 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: DEBUG neutron.common.utils [None req-5707c7d9-87c8-4a6e-aa0b-9c7198f071ee None None] Calling throttled function clear {{(pid=20208) wrapper /opt/stack/neutron/neutron/common/utils.py:110}}\nDec 09 20:08:20.600393 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: DEBUG neutron.agent.dhcp.agent [None req-5707c7d9-87c8-4a6e-aa0b-9c7198f071ee None None] resync (0e5dfd3d-7a4a-4be4-97cd-8aa3acb9a959): [RemoteError('Unserializable message: Traceback (most recent call last):\\n File \"/usr/lib/python3.6/multiprocessing/managers.py\", line 283, in serve_client\\n send(msg)\\n File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 128, in send\\n s = self.dumps(obj)\\n File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 170, in dumps\\n return json.dumps(obj, cls=RpcJSONEncoder).encode(\\'utf-8\\')\\n File \"/usr/lib/python3.6/json/__init__.py\", line 238, in dumps\\n **kw).encode(obj)\\n File \"/usr/lib/python3.6/json/encoder.py\", line 199, in encode\\n chunks = self.iterencode(o, _one_shot=True)\\n File \"/usr/lib/python3.6/json/encoder.py\", line 257, in iterencode\\n return _iterencode(o, 0)\\n File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 43, in default\\n return super(RpcJSONEncoder, self).default(o)\\n File \"/usr/lib/python3.6/json/encoder.py\", line 180, in default\\n o.__class__.__name__)\\nTypeError: Object of type \\'ValueError\\' is not JSON serializable\\n\\n',)] {{(pid=20208) _periodic_resync_helper /opt/stack/neutron/neutron/agent/dhcp/agent.py:294}}\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent [-] Unable to reload_allocations dhcp for 0e5dfd3d-7a4a-4be4-97cd-8aa3acb9a959.: multiprocessing.managers.RemoteError:\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ---------------------------------------------------------------------------\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: Unserializable message: Traceback (most recent call last):\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/lib/python3.6/multiprocessing/managers.py\", line 283, in serve_client\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: send(msg)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 128, in send\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: s = self.dumps(obj)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 170, in dumps\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: return json.dumps(obj, cls=RpcJSONEncoder).encode('utf-8')\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/lib/python3.6/json/__init__.py\", line 238, in dumps\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: **kw).encode(obj)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/lib/python3.6/json/encoder.py\", line 199, in encode\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: chunks = self.iterencode(o, _one_shot=True)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/lib/python3.6/json/encoder.py\", line 257, in iterencode\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: return _iterencode(o, 0)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 43, in default\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: return super(RpcJSONEncoder, self).default(o)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: File \"/usr/lib/python3.6/json/encoder.py\", line 180, in default\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: o.__class__.__name__)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: TypeError: Object of type 'ValueError' is not JSON serializable\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ---------------------------------------------------------------------------\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent Traceback (most recent call last):\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/dhcp/agent.py\", line 160, in call_driver\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent getattr(driver, action)(**action_kwargs)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/linux/dhcp.py\", line 566, in reload_allocations\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent self._release_unused_leases()\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/linux/dhcp.py\", line 941, in _release_unused_leases\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent entry['server_id'], entry['iaid'])\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/linux/dhcp.py\", line 538, in _release_lease\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent ip_wrapper.netns.execute(cmd, run_as_root=True)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/linux/ip_lib.py\", line 714, in execute\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent run_as_root=run_as_root)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/linux/utils.py\", line 122, in execute\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent execute_rootwrap_daemon(cmd, process_input, addl_env))\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/linux/utils.py\", line 109, in execute_rootwrap_daemon\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent LOG.error(\"Rootwrap error running command: %s\", cmd)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py\", line 220, in __exit__\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent self.force_reraise()\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py\", line 196, in force_reraise\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent six.reraise(self.type_, self.value, self.tb)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/six.py\", line 696, in reraise\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent raise value\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/opt/stack/neutron/neutron/agent/linux/utils.py\", line 106, in execute_rootwrap_daemon\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent return client.execute(cmd, process_input)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/client.py\", line 154, in execute\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent res = self._run_one_command(proxy, cmd, stdin)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/client.py\", line 139, in _run_one_command\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent res = proxy.run_one_command(cmd, stdin)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"\", line 2, in run_one_command\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/lib/python3.6/multiprocessing/managers.py\", line 772, in _callmethod\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent raise convert_to_error(kind, result)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent multiprocessing.managers.RemoteError:\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent ---------------------------------------------------------------------------\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent Unserializable message: Traceback (most recent call last):\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/lib/python3.6/multiprocessing/managers.py\", line 283, in serve_client\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent send(msg)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 128, in send\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent s = self.dumps(obj)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 170, in dumps\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent return json.dumps(obj, cls=RpcJSONEncoder).encode('utf-8')\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/lib/python3.6/json/__init__.py\", line 238, in dumps\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent **kw).encode(obj)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/lib/python3.6/json/encoder.py\", line 199, in encode\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent chunks = self.iterencode(o, _one_shot=True)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/lib/python3.6/json/encoder.py\", line 257, in iterencode\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent return _iterencode(o, 0)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/local/lib/python3.6/dist-packages/oslo_rootwrap/jsonrpc.py\", line 43, in default\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent return super(RpcJSONEncoder, self).default(o)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent File \"/usr/lib/python3.6/json/encoder.py\", line 180, in default\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent o.__class__.__name__)\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent TypeError: Object of type 'ValueError' is not JSON serializable\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent \nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent ---------------------------------------------------------------------------\nDec 09 20:08:20.601927 ubuntu-bionic-rax-dfw-0013298071 neutron-dhcp-agent[20208]: ERROR neutron.agent.dhcp.agent ","name":"","extension":"txt","url":"https://www.irccloud.com/pastebin/LUhfa1KS","modified":1575995028,"id":"LUhfa1KS","size":14436,"lines":78,"own_paste":false,"theme":"","date":1575995028}