OpenStack の ダッシュボードにアクセスできなくなった時の対処メモ
OpenStack の ダッシュボードにアクセスできなくなったのでその時の対処メモ。
■ 概要
OS を立ち上げ直したらなぜか OpenStack dashboard にアクセスできなくなった。
結論から言えば下記サイトの answer 1 の通りにやったら直った。
https://ask.openstack.org/en/question/95308/dashboard-uncontactable/
それだけの話だけどその時のログを残す。
■ いくつかのコンポーネントが上手く立ち上がらなかった
VMware で動かしている CentOS をいつも通り起動して OpenStack のダッシュボードにアクセスしたら入れなかった。
特に前回シャットダウンするときも変なことをした覚えはなく、恐らく単にコンポーネントの立ち上げに失敗しただけと思われる。
もう一度OS再起動したら直りそうな気もするけど、せっかくなのでなんとかしてみる。
とりあえず状況を確認する。
# openstack-status == Nova services == openstack-nova-api: active openstack-nova-compute: active openstack-nova-network: inactive (disabled on boot) openstack-nova-scheduler: active openstack-nova-cert: active openstack-nova-conductor: active openstack-nova-console: inactive (disabled on boot) openstack-nova-consoleauth: active openstack-nova-xvpvncproxy: inactive (disabled on boot) == Glance services == openstack-glance-api: active openstack-glance-registry: active == Keystone service == openstack-keystone: inactive (disabled on boot) == Horizon service == openstack-dashboard: uncontactable == neutron services == neutron-server: failed neutron-dhcp-agent: active neutron-l3-agent: active neutron-metadata-agent: active neutron-lbaas-agent: active neutron-lbaasv2-agent: inactive (disabled on boot) neutron-openvswitch-agent: active neutron-metering-agent: active == Swift services == openstack-swift-proxy: active openstack-swift-account: active openstack-swift-container: active openstack-swift-object: active == Cinder services == openstack-cinder-api: active openstack-cinder-scheduler: active openstack-cinder-volume: active openstack-cinder-backup: active == Ceilometer services == openstack-ceilometer-api: inactive (disabled on boot) openstack-ceilometer-central: active openstack-ceilometer-compute: active openstack-ceilometer-collector: active openstack-ceilometer-notification: active == Heat services == openstack-heat-api: active openstack-heat-api-cfn: inactive (disabled on boot) openstack-heat-api-cloudwatch: inactive (disabled on boot) openstack-heat-engine: active == Support services == mariadb: active openvswitch: active dbus: active target: active rabbitmq-server: active memcached: active == Keystone users == Warning keystonerc not sourced #
openstack-dashboard: が uncontactable になってるし
neutron-server: も failed になってる。
そりゃダッシュボードにアクセスできんわ。
■ failed と uncontactable のコンポーネントの状態を直す
まずは neutron-server を restart してみる。
# systemctl restart neutron-server.service # openstack-status == Nova services == openstack-nova-api: active openstack-nova-compute: active openstack-nova-network: inactive (disabled on boot) openstack-nova-scheduler: active openstack-nova-cert: active openstack-nova-conductor: active openstack-nova-console: inactive (disabled on boot) openstack-nova-consoleauth: active openstack-nova-xvpvncproxy: inactive (disabled on boot) == Glance services == openstack-glance-api: active openstack-glance-registry: active == Keystone service == openstack-keystone: inactive (disabled on boot) == Horizon service == openstack-dashboard: uncontactable == neutron services == neutron-server: active neutron-dhcp-agent: active neutron-l3-agent: active neutron-metadata-agent: active neutron-lbaas-agent: active neutron-lbaasv2-agent: inactive (disabled on boot) neutron-openvswitch-agent: active neutron-metering-agent: active == Swift services == openstack-swift-proxy: active openstack-swift-account: active openstack-swift-container: active openstack-swift-object: active == Cinder services == openstack-cinder-api: active openstack-cinder-scheduler: active openstack-cinder-volume: active openstack-cinder-backup: active == Ceilometer services == openstack-ceilometer-api: inactive (disabled on boot) openstack-ceilometer-central: active openstack-ceilometer-compute: active openstack-ceilometer-collector: active openstack-ceilometer-notification: active == Heat services == openstack-heat-api: active openstack-heat-api-cfn: inactive (disabled on boot) openstack-heat-api-cloudwatch: inactive (disabled on boot) openstack-heat-engine: active == Support services == mariadb: active openvswitch: active dbus: active target: active rabbitmq-server: active memcached: active == Keystone users == Warning keystonerc not sourced #
neutron-server は active になった。
openstack-dashboard もそれに引きずられて自動的に直ることを期待したけどそんなこともなく。
openstack-dashboard も同じように restart してみる。
# systemctl restart openstack-dashboard.service Failed to restart openstack-dashboard.service: Unit openstack-dashboard.service failed to load: No such file or directory. #
...もしかして openstack-dashboard は単体でどうこうするものではないのか ?
その辺もまだ理解していない。
その後色々試して格闘したがどうにもならず。。
ググっていった結果、このページに辿り着いた。
思考停止で answer 1 の通りにやってみる。
# systemctl restart rabbitmq-server # systemctl restart openstack-keystone You have new mail in /var/spool/mail/root # # ls -l /etc/systemd/system/ total 16 drwxr-xr-x. 2 root root 53 Nov 3 17:34 basic.target.wants drwxr-xr-x. 2 root root 30 Nov 3 15:39 bluetooth.target.wants lrwxrwxrwx. 1 root root 41 Nov 3 15:39 dbus-org.bluez.service -> /usr/lib/systemd/system/bluetooth.service lrwxrwxrwx. 1 root root 44 Nov 3 15:40 dbus-org.freedesktop.Avahi.service -> /usr/lib/systemd/system/avahi-daemon.service lrwxrwxrwx. 1 root root 44 Nov 3 15:41 dbus-org.freedesktop.ModemManager1.service -> /usr/lib/systemd/system/ModemManager.service lrwxrwxrwx. 1 root root 46 Nov 3 15:39 dbus-org.freedesktop.NetworkManager.service -> /usr/lib/systemd/system/NetworkManager.service lrwxrwxrwx. 1 root root 57 Nov 3 15:39 dbus-org.freedesktop.nm-dispatcher.service -> /usr/lib/systemd/system/NetworkManager-dispatcher.service lrwxrwxrwx. 1 root root 36 Nov 3 15:44 default.target -> /lib/systemd/system/graphical.target drwxr-xr-x. 2 root root 85 Nov 3 15:38 default.target.wants drwxr-xr-x. 2 root root 37 Nov 3 15:40 dev-virtio\x2dports-org.qemu.guest_agent.0.device.wants lrwxrwxrwx. 1 root root 35 Nov 3 15:40 display-manager.service -> /usr/lib/systemd/system/gdm.service drwxr-xr-x. 2 root root 31 Nov 3 15:38 getty.target.wants drwxr-xr-x. 2 root root 63 Nov 3 15:39 graphical.target.wants drwxr-xr-x. 2 root root 4096 Nov 5 19:27 multi-user.target.wants drwxr-xr-x. 2 root root 38 Nov 3 18:03 openstack-cinder-volume.service.requires drwxr-xr-x. 2 root root 25 Nov 3 15:39 printer.target.wants drwxr-xr-x. 2 root root 24 Nov 3 17:40 rabbitmq-server.service.d drwxr-xr-x. 2 root root 23 Nov 3 18:21 redis-sentinel.service.d drwxr-xr-x. 2 root root 23 Nov 3 18:21 redis.service.d drwxr-xr-x. 2 root root 30 Nov 3 15:39 remote-fs.target.wants drwxr-xr-x. 2 root root 4096 Nov 5 19:16 sockets.target.wants drwxr-xr-x. 2 root root 35 Nov 3 15:40 spice-vdagentd.target.wants drwxr-xr-x. 2 root root 25 Nov 3 16:34 sssd.service.d drwxr-xr-x. 2 root root 4096 Nov 5 19:16 sysinit.target.wants drwxr-xr-x. 2 root root 43 Nov 3 15:38 system-update.target.wants # # # ln -s /usr/lib/systemd/system/httpd.service /etc/systemd/system/openstack-keystone.service # # ls -l /etc/systemd/system/open* lrwxrwxrwx. 1 root root 37 Nov 12 15:38 /etc/systemd/system/openstack-keystone.service -> /usr/lib/systemd/system/httpd.service /etc/systemd/system/openstack-cinder-volume.service.requires: total 0 lrwxrwxrwx. 1 root root 49 Nov 3 18:03 openstack-losetup.service -> /usr/lib/systemd/system/openstack-losetup.service # systemctl daemon-reload # systemctl stop httpd.service # systemctl stop openstack-keystone.service # systemctl start openstack-keystone.service # # openstack-status == Nova services == openstack-nova-api: active openstack-nova-compute: active openstack-nova-network: inactive (disabled on boot) openstack-nova-scheduler: active openstack-nova-cert: active openstack-nova-conductor: active openstack-nova-console: inactive (disabled on boot) openstack-nova-consoleauth: active openstack-nova-xvpvncproxy: inactive (disabled on boot) == Glance services == openstack-glance-api: active openstack-glance-registry: active == Keystone service == openstack-keystone: active == Horizon service == openstack-dashboard: active == neutron services == neutron-server: active neutron-dhcp-agent: active neutron-l3-agent: active neutron-metadata-agent: active neutron-lbaas-agent: active neutron-lbaasv2-agent: inactive (disabled on boot) neutron-openvswitch-agent: active neutron-metering-agent: active == Swift services == openstack-swift-proxy: active openstack-swift-account: active openstack-swift-container: active openstack-swift-object: active == Cinder services == openstack-cinder-api: active openstack-cinder-scheduler: active openstack-cinder-volume: active openstack-cinder-backup: active == Ceilometer services == openstack-ceilometer-api: inactive (disabled on boot) openstack-ceilometer-central: active openstack-ceilometer-compute: active openstack-ceilometer-collector: active openstack-ceilometer-notification: active == Heat services == openstack-heat-api: active openstack-heat-api-cfn: inactive (disabled on boot) openstack-heat-api-cloudwatch: inactive (disabled on boot) openstack-heat-engine: active == Support services == mariadb: active openvswitch: active dbus: active target: active rabbitmq-server: active memcached: active == Keystone users == Warning keystonerc not sourced #
openstack-dashboard: が active になってる!
ダッシュボードにもアクセスできるようになった。
httpd の停止と httpd.service へリンクする openstack-keystone.service の作成がポイントだったようだ。
#これも何となくしか理解できていないが......。
しかし、今までこんな問題が発生しなかったのになんで起きたのかは分からず終い。