{"body":"十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.435 16199 INFO kuryr_kubernetes.config [-] Logging enabled!\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.435 16199 INFO kuryr_kubernetes.config [-] /opt/stack/cni/bin/kuryr-cni version 0.5.1\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.527 16199 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.527 16199 DEBUG os_vif [-] Loaded VIF plugin class '<class 'kuryr_kubernetes.os_vif_plug_noop.SriovPlugin'>' with name 'sriov' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.527 16199 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.528 16199 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.528 16199 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.528 16199 INFO os_vif [-] Loaded VIF plugins: noop, sriov, ovs, linux_bridge, noop\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.529 16199 INFO kuryr_kubernetes.cni.main [-] Using 'CNIDaemonizedRunner' \n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.529 16199 DEBUG kuryr_kubernetes.cni.api [-] Making request to CNI Daemon. POST http://127.0.0.1:50036/addNetwork\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: {'CNI_IFNAME': 'eth0', 'CNI_NETNS': '/proc/16057/ns/net', 'CNI_PATH': '/opt/stack/cni/bin', 'CNI_COMMAND': 'ADD', 'CNI_ARGS': 'IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=nginx;K8S_POD_INFRA_CONTAINER_ID=1c6d46152c3b892d2d056bd768c549a379188d978c455b39064f3437ddf4468c', 'config_kuryr': {u'debug': True, u'cniVersion': u'0.3.1', u'type': u'kuryr-cni', u'kuryr_conf': u'/etc/kuryr/kuryr.conf', u'name': u'kuryr'}, 'CNI_CONTAINERID': '1c6d46152c3b892d2d056bd768c549a379188d978c455b39064f3437ddf4468c'} _make_request /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:179\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.984 15957 DEBUG kuryr_kubernetes.cni.api [-] CNI Daemon returned \"204 NO CONTENT\". _make_request /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:187\n十一 30 22:29:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:47.985 15957 DEBUG kuryr_kubernetes.cni.main [-] Exiting with status 0 run /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/main.py:78\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]: E1130 22:29:48.393473   22161 remote_runtime.go:96] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to set up sandbox container \"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\" network for pod \"svc-vdu1-e493cd-b54fd7f95-6whrz\": NetworkPlugin cni failed to set up pod \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system\" network: Got invalid status code from CNI daemon.; Traceback (most recent call last):\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 81, in run\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     vif = self._add(params)\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 154, in _add\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     resp = self._make_request('addNetwork', params, httplib.ACCEPTED)\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 191, in _make_request\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     raise k_exc.CNIError('Got invalid status code from CNI daemon.')\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]: CNIError: Got invalid status code from CNI daemon.\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]: E1130 22:29:48.394286   22161 kuberuntime_sandbox.go:65] CreatePodSandbox for pod \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\" failed: rpc error: code = Unknown desc = failed to set up sandbox container \"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\" network for pod \"svc-vdu1-e493cd-b54fd7f95-6whrz\": NetworkPlugin cni failed to set up pod \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system\" network: Got invalid status code from CNI daemon.; Traceback (most recent call last):\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 81, in run\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     vif = self._add(params)\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 154, in _add\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     resp = self._make_request('addNetwork', params, httplib.ACCEPTED)\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 191, in _make_request\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     raise k_exc.CNIError('Got invalid status code from CNI daemon.')\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]: CNIError: Got invalid status code from CNI daemon.\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]: E1130 22:29:48.394303   22161 kuberuntime_manager.go:657] createPodSandbox for pod \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\" failed: rpc error: code = Unknown desc = failed to set up sandbox container \"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\" network for pod \"svc-vdu1-e493cd-b54fd7f95-6whrz\": NetworkPlugin cni failed to set up pod \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system\" network: Got invalid status code from CNI daemon.; Traceback (most recent call last):\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 81, in run\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     vif = self._add(params)\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 154, in _add\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     resp = self._make_request('addNetwork', params, httplib.ACCEPTED)\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 191, in _make_request\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]:     raise k_exc.CNIError('Got invalid status code from CNI daemon.')\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]: CNIError: Got invalid status code from CNI daemon.\n十一 30 22:29:48 kuryr-demo-KVM hyperkube[22161]: E1130 22:29:48.394356   22161 pod_workers.go:186] Error syncing pod dd7207c9-f467-11e8-bf16-525400755526 (\"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\"), skipping: failed to \"CreatePodSandbox\" for \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\" with CreatePodSandboxError: \"CreatePodSandbox for pod \\\"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\\\" failed: rpc error: code = Unknown desc = failed to set up sandbox container \\\"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\\\" network for pod \\\"svc-vdu1-e493cd-b54fd7f95-6whrz\\\": NetworkPlugin cni failed to set up pod \\\"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system\\\" network: Got invalid status code from CNI daemon.; Traceback (most recent call last):\\n  File \\\"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\\\", line 81, in run\\n    vif = self._add(params)\\n  File \\\"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\\\", line 154, in _add\\n    resp = self._make_request('addNetwork', params, httplib.ACCEPTED)\\n  File \\\"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\\\", line 191, in _make_request\\n    raise k_exc.CNIError('Got invalid status code from CNI daemon.')\\nCNIError: Got invalid status code from CNI daemon.\\n\"\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: W1130 22:29:49.304202   22161 docker_sandbox.go:375] failed to read pod IP from plugin/docker: NetworkPlugin cni failed on the status hook for pod \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system\": CNI failed to retrieve network namespace path: cannot find network namespace for the terminated container \"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\"\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: I1130 22:29:49.304953   22161 kubelet.go:1928] SyncLoop (PLEG): \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\", event: &pleg.PodLifecycleEvent{ID:\"dd7207c9-f467-11e8-bf16-525400755526\", Type:\"ContainerDied\", Data:\"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\"}\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: W1130 22:29:49.305126   22161 pod_container_deletor.go:75] Container \"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\" not found in pod's containers\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: I1130 22:29:49.305074   22161 kuberuntime_manager.go:414] No ready sandbox for pod \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\" can be found. Need to start a new one\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: W1130 22:29:49.306671   22161 cni.go:293] CNI failed to retrieve network namespace path: cannot find network namespace for the terminated container \"a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6\"\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: /usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]:   \"\"\")\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.870 16250 INFO kuryr_kubernetes.config [-] Logging enabled!\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.871 16250 INFO kuryr_kubernetes.config [-] /opt/stack/cni/bin/kuryr-cni version 0.5.1\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.972 16250 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.973 16250 DEBUG os_vif [-] Loaded VIF plugin class '<class 'kuryr_kubernetes.os_vif_plug_noop.SriovPlugin'>' with name 'sriov' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.973 16250 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.974 16250 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.975 16250 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.975 16250 INFO os_vif [-] Loaded VIF plugins: noop, sriov, ovs, linux_bridge, noop\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.975 16250 INFO kuryr_kubernetes.cni.main [-] Using 'CNIDaemonizedRunner' \n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:29:49.976 16250 DEBUG kuryr_kubernetes.cni.api [-] Making request to CNI Daemon. POST http://127.0.0.1:50036/delNetwork\n十一 30 22:29:49 kuryr-demo-KVM hyperkube[22161]: {'CNI_IFNAME': 'eth0', 'CNI_NETNS': '', 'CNI_PATH': '/opt/stack/cni/bin', 'CNI_COMMAND': 'DEL', 'CNI_ARGS': 'IgnoreUnknown=1;K8S_POD_NAMESPACE=kube-system;K8S_POD_NAME=svc-vdu1-e493cd-b54fd7f95-6whrz;K8S_POD_INFRA_CONTAINER_ID=a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6', 'config_kuryr': {u'debug': True, u'cniVersion': u'0.3.1', u'type': u'kuryr-cni', u'kuryr_conf': u'/etc/kuryr/kuryr.conf', u'name': u'kuryr'}, 'CNI_CONTAINERID': 'a9884acffffe59346fcba6cd7b90e4f77751af03f7e7d509a759b22cae6e20a6'} _make_request /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:179\n十一 30 22:29:53 kuryr-demo-KVM hyperkube[22161]: E1130 22:29:53.340303   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n十一 30 22:30:03 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:03.353161   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n十一 30 22:30:13 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:13.364978   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n十一 30 22:30:23 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:23.380240   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n十一 30 22:30:33 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:33.392786   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n十一 30 22:30:43 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:43.411376   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:47.596 16199 DEBUG kuryr_kubernetes.cni.api [-] CNI Daemon returned \"504 GATEWAY TIMEOUT\". _make_request /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:187\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:47.597 16199 ERROR kuryr_kubernetes.cni.api [-] CNI daemon returned error \"504 GATEWAY TIMEOUT\".\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:47.597 16199 DEBUG kuryr_kubernetes.cni.api [-] CNI output: {'msg': 'Got invalid status code from CNI daemon.', 'cniVersion': '0.3.1', 'code': 100, 'details': 'Traceback (most recent call last):\\n  File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 81, in run\\n    vif = self._add(params)\\n  File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 154, in _add\\n    resp = self._make_request(\\'addNetwork\\', params, httplib.ACCEPTED)\\n  File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 191, in _make_request\\n    raise k_exc.CNIError(\\'Got invalid status code from CNI daemon.\\')\\nCNIError: Got invalid status code from CNI daemon.\\n'} _write_dict /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:55\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:47.597 16199 DEBUG kuryr_kubernetes.cni.main [-] Exiting with status 1 run /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/main.py:78\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:47.648407   22161 cni.go:310] Error adding network: Got invalid status code from CNI daemon.; Traceback (most recent call last):\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 81, in run\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:     vif = self._add(params)\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 154, in _add\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:     resp = self._make_request('addNetwork', params, httplib.ACCEPTED)\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 191, in _make_request\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:     raise k_exc.CNIError('Got invalid status code from CNI daemon.')\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: CNIError: Got invalid status code from CNI daemon.\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:47.648452   22161 cni.go:278] Error while adding to cni network: Got invalid status code from CNI daemon.; Traceback (most recent call last):\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 81, in run\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:     vif = self._add(params)\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 154, in _add\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:     resp = self._make_request('addNetwork', params, httplib.ACCEPTED)\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:   File \"/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py\", line 191, in _make_request\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]:     raise k_exc.CNIError('Got invalid status code from CNI daemon.')\n十一 30 22:30:47 kuryr-demo-KVM hyperkube[22161]: CNIError: Got invalid status code from CNI daemon.\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: /usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]:   \"\"\")\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.250 16324 INFO kuryr_kubernetes.config [-] Logging enabled!\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.250 16324 INFO kuryr_kubernetes.config [-] /opt/stack/cni/bin/kuryr-cni version 0.5.1\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.356 16324 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.357 16324 DEBUG os_vif [-] Loaded VIF plugin class '<class 'kuryr_kubernetes.os_vif_plug_noop.SriovPlugin'>' with name 'sriov' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.357 16324 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.358 16324 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.359 16324 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.359 16324 INFO os_vif [-] Loaded VIF plugins: noop, sriov, ovs, linux_bridge, noop\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.360 16324 INFO kuryr_kubernetes.cni.main [-] Using 'CNIDaemonizedRunner' \n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:48.360 16324 DEBUG kuryr_kubernetes.cni.api [-] Making request to CNI Daemon. POST http://127.0.0.1:50036/delNetwork\n十一 30 22:30:48 kuryr-demo-KVM hyperkube[22161]: {'CNI_IFNAME': 'eth0', 'CNI_NETNS': '/proc/16057/ns/net', 'CNI_PATH': '/opt/stack/cni/bin', 'CNI_COMMAND': 'DEL', 'CNI_ARGS': 'IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=nginx;K8S_POD_INFRA_CONTAINER_ID=1c6d46152c3b892d2d056bd768c549a379188d978c455b39064f3437ddf4468c', 'config_kuryr': {u'debug': True, u'cniVersion': u'0.3.1', u'type': u'kuryr-cni', u'kuryr_conf': u'/etc/kuryr/kuryr.conf', u'name': u'kuryr'}, 'CNI_CONTAINERID': '1c6d46152c3b892d2d056bd768c549a379188d978c455b39064f3437ddf4468c'} _make_request /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:179\n十一 30 22:30:50 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:50.046 16250 DEBUG kuryr_kubernetes.cni.api [-] CNI Daemon returned \"204 NO CONTENT\". _make_request /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:187\n十一 30 22:30:50 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:50.046 16250 DEBUG kuryr_kubernetes.cni.main [-] Exiting with status 0 run /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/main.py:78\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: I1130 22:30:51.440056   22161 kubelet.go:1928] SyncLoop (PLEG): \"svc-vdu1-e493cd-b54fd7f95-6whrz_kube-system(dd7207c9-f467-11e8-bf16-525400755526)\", event: &pleg.PodLifecycleEvent{ID:\"dd7207c9-f467-11e8-bf16-525400755526\", Type:\"ContainerStarted\", Data:\"e97c1afbec315353f5a7071185c7e06e6455076291e5c04e0e344af1fbf6b8f5\"}\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: /usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]:   \"\"\")\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.725 16417 INFO kuryr_kubernetes.config [-] Logging enabled!\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.726 16417 INFO kuryr_kubernetes.config [-] /opt/stack/cni/bin/kuryr-cni version 0.5.1\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.828 16417 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.828 16417 DEBUG os_vif [-] Loaded VIF plugin class '<class 'kuryr_kubernetes.os_vif_plug_noop.SriovPlugin'>' with name 'sriov' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.828 16417 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.831 16417 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.831 16417 DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' initialize /usr/local/lib/python2.7/dist-packages/os_vif/__init__.py:46\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.832 16417 INFO os_vif [-] Loaded VIF plugins: noop, sriov, ovs, linux_bridge, noop\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.832 16417 INFO kuryr_kubernetes.cni.main [-] Using 'CNIDaemonizedRunner' \n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: 2018-11-30 22:30:51.832 16417 DEBUG kuryr_kubernetes.cni.api [-] Making request to CNI Daemon. POST http://127.0.0.1:50036/addNetwork\n十一 30 22:30:51 kuryr-demo-KVM hyperkube[22161]: {'CNI_IFNAME': 'eth0', 'CNI_NETNS': '/proc/16364/ns/net', 'CNI_PATH': '/opt/stack/cni/bin', 'CNI_COMMAND': 'ADD', 'CNI_ARGS': 'IgnoreUnknown=1;K8S_POD_NAMESPACE=kube-system;K8S_POD_NAME=svc-vdu1-e493cd-b54fd7f95-6whrz;K8S_POD_INFRA_CONTAINER_ID=e97c1afbec315353f5a7071185c7e06e6455076291e5c04e0e344af1fbf6b8f5', 'config_kuryr': {u'debug': True, u'cniVersion': u'0.3.1', u'type': u'kuryr-cni', u'kuryr_conf': u'/etc/kuryr/kuryr.conf', u'name': u'kuryr'}, 'CNI_CONTAINERID': 'e97c1afbec315353f5a7071185c7e06e6455076291e5c04e0e344af1fbf6b8f5'} _make_request /opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py:179\n十一 30 22:30:53 kuryr-demo-KVM hyperkube[22161]: E1130 22:30:53.431197   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n十一 30 22:31:03 kuryr-demo-KVM hyperkube[22161]: E1130 22:31:03.445320   22161 summary_sys_containers.go:45] Failed to get system container stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get cgroup stats for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": failed to get container info for \"/system.slice/system-devstack.slice/devstack@kubelet.service\": unknown container \"/system.slice/system-devstack.slice/devstack@kubelet.service\"\n","name":"","extension":"txt","url":"https://www.irccloud.com/pastebin/0TIEJjlE","modified":1543588540,"id":"0TIEJjlE","size":28009,"lines":113,"own_paste":false,"theme":"","date":1543588540}