{"body":"eevans@cumin1001:~$ sudo cookbook sre.discovery.service-route depool $DC sessionstore                                                                                                                              \nSTART - Cookbook sre.discovery.service-route depool sessionstore in codfw: maintenance                                                                                                                             \nException raised while executing cookbook sre.discovery.service-route:                                                                                                                                             \nTraceback (most recent call last):                                                                                                                                                                                 \n  File \"/usr/lib/python3/dist-packages/spicerack/dnsdisc.py\", line 234, in resolve                                                                                                                                 \n    answer = dns_resolver.query(record_name)                                                                                                                                                                       \n  File \"/usr/lib/python3/dist-packages/dns/resolver.py\", line 1089, in query                                                                                                                                       \n    return self.resolve(qname, rdtype, rdclass, tcp, source,                                                                                                                                                       \n  File \"/usr/lib/python3/dist-packages/dns/resolver.py\", line 1040, in resolve                                                                                                                                     \n    (nameserver, port, tcp, backoff) = resolution.next_nameserver()                                                                                                                                                \n  File \"/usr/lib/python3/dist-packages/dns/resolver.py\", line 598, in next_nameserver                                                                                                                              \n    raise NoNameservers(request=self.request, errors=self.errors)                                                                                                                                                  \ndns.resolver.NoNameservers: All nameservers failed to answer the query sessionstore.discovery.wmnet. IN A: Server 198.35.26.8 UDP port 5353 answered REFUSED\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/usr/lib/python3/dist-packages/spicerack/_menu.py\", line 234, in run\n    raw_ret = runner.run()\n  File \"/srv/deployment/spicerack/cookbooks/sre/discovery/service-route.py\", line 186, in run\n    self.pool_or_depool()\n  File \"/srv/deployment/spicerack/cookbooks/sre/discovery/service-route.py\", line 154, in pool_or_depool\n    old_ttl = update_ttl(self.dnsdisc, DNS_TTL_MEDIUM)\n  File \"/srv/deployment/spicerack/cookbooks/sre/discovery/__init__.py\", line 150, in update_ttl\n    old_ttl = max(r.ttl for r in dnsdisc.resolve())\n  File \"/srv/deployment/spicerack/cookbooks/sre/discovery/__init__.py\", line 150, in <genexpr>\n    old_ttl = max(r.ttl for r in dnsdisc.resolve())\n  File \"/usr/lib/python3/dist-packages/spicerack/dnsdisc.py\", line 236, in resolve\n    raise DiscoveryError(f\"Unable to resolve {record_name} from {nameserver}\") from e\nspicerack.dnsdisc.DiscoveryError: Unable to resolve sessionstore.discovery.wmnet from dns4004.wikimedia.org\nEND (FAIL) - Cookbook sre.discovery.service-route (exit_code=99) depool sessionstore in codfw: maintenance","name":"","extension":"txt","url":"https://www.irccloud.com/pastebin/X8NtQhNI","modified":1676475939,"id":"X8NtQhNI","size":3869,"lines":31,"own_paste":false,"theme":"","date":1676475939}