[Nutanix] cluster ip — manual changing
** AOS 5.10 ~ **
1. deleting external cluster ip
$ ncli cluster clear-external-ip-address
$ ncli cluster clear-external-ip-address
Cluster Id : 00059ffe-d7a2-c865–00e1-ac1f6b3b9d4c::63521120702930252
Cluster Uuid : 00059ffe-d7a2-c865–00e1-ac1f6b3b9d4c
Cluster Name : aaa
Cluster Version : 5.10.9
Cluster Full Version : el7.3-release-euphrates-5.10.9-stable-7a0c244378ccf45b78039c9de2099a61a01d02d3
External IP address :
Node Count : 3
Block Count : 1
Shadow Clones Status : Enabled
Has Self Encrypting Disk : no
Cluster Masquerading I… :
Cluster Masquerading PORT :
Is LTS : true
External Data Services… :
Support Verbosity Level : BASIC_COREDUMP
Lock Down Status : Disabled
Password Remote Login … : Enabled
Timezone : Asia/Seoul
NCC Version : ncc-3.9.4.1
Common Criteria Mode : Disabled
Degraded Node Monitoring : Enabled
2. new external cluster ip setup
$ ncli cluster set-external-ip-address external-ip-address=10.100.1.180 logon-name=admin password=aaa
$ ncli cluster set-external-ip-address external-ip-address=10.100.1.180 logon-name=admin password=aaa
Cluster Id : 00059ffe-d7a2-c865–00e1-ac1f6b3b9d4c::63521120702930252
Cluster Uuid : 00059ffe-d7a2-c865–00e1-ac1f6b3b9d4c
Cluster Name : aaa
Cluster Version : 5.10.9
Cluster Full Version : el7.3-release-euphrates-5.10.9-stable-7a0c244378ccf45b78039c9de2099a61a01d02d3
External IP address : 10.209.210.20
Node Count : 3
Block Count : 1
Shadow Clones Status : Enabled
Has Self Encrypting Disk : no
Cluster Masquerading I… :
Cluster Masquerading PORT :
Is LTS : true
External Data Services… :
Support Verbosity Level : BASIC_COREDUMP
Lock Down Status : Disabled
Password Remote Login … : Enabled
Timezone : Asia/Seoul
NCC Version : ncc-3.9.4.1
Common Criteria Mode : Disabled
Degraded Node Monitoring : Enabled
3. deleting or changing NTP, DNS
ncli> cluster remove-from-name-servers servers=”168.126.63.1"
ncli> cluster remove-from-name-servers servers=”8.8.8.8"
ncli> cluster remove-from-ntp-servers servers=”210.120.246.103"
ncli> cluster remove-from-ntp-servers servers=”141.223.182.106"
4. checking metadata status (metadata 상태 확인)
$ ncli host ls | grep “Metadata store status”
“Metadata store enabled on the node” 확인
$ ncli host ls | grep “Metadata store status”
Metadata store status : Metadata store enabled on the node
Metadata store status : Metadata store enabled on the node
Metadata store status : Metadata store enabled on the node
5. login CVM via IPMI or AHV
6. running reconfig script
$ cluster stop
$ external_ip_reconfig
External IP reconfig finished successfully. Restart all the CVMs and start the cluster.
7. changing host ip
# vi ifcfg-br0
# service network restart
8. rebooting all CVMs
$ sudo reboot
9. checking cvm, host ips
$ svmips
10.209.210.21 10.209.210.22 10.209.210.23
$ hostips
10.209.210.41 10.209.210.42 10.209.210.43
$ allssh sort -k2 /etc/hosts
** old method — manual ip configuration **
- checking status (기본 설정 확인 및 변경)
(1) DNS
$ ncli cluster get-name-servers
(2) NTP
$ ncli cluster get-ntp-servers
(3) “metadata store” enabled
$ ncli host ls | grep “Metadata store status”
2. reconfiguration
(1) Cluster stop
$ cluster stop
(2) reconfiguration mode
$ cluster reconfig
(3) checking genesis messages — RPC messages
$allssh tail -n1 /home/nutanix/data/logs/genesis.out
→ “2017–03–23 00:38:41 INFO node_manager.py:5191 Waiting for node to become configured through RPC.”
3. changing all cvm ips (모든 CVM의 IP 변경)
(1) logon to hosts
(2) ssh nutanix@192.168.5.254
(3) changing ip address
$ sudoedit /etc/sysconfig/network-scripts/ifcfg-eth0
-> this will be affected after phase 6. (6. 단계의 CVM 리부팅 전까지는 실제 적용되지 않음)
4. Zookeeper configuration
(1) on one CVM
$ edit-zkmapping check_can_modify_zk_map
$ edit-zkmapping modify_zk_server_config
-> 자동으로 변경되 IP들 반영
$ edit-zkmapping check_can_modify_zk_map
5. changing host ip address
(1) each node
- AHV : /etc/init.d/network restart
- ESXi : esxcli network ip interface ipv4 set -i vmk0 -I <IPADDRESS> -N <NETMASK> -t static
esxcli network ip route ipv4 add -g <GATEWAY_IP> -n default
6. reboot all CVMs
7. disable reconfiguration mode
(1) each CVM
- rm -v ~/.node_reconfigure
- genesis restart
(2) wait over 5 minuts
- cluster status
8. cluster start
(1) cluster start
(2) changing external cluster ip address
9. changing IPMI ip address
(1) each host
- ipmitool lan print 1
- ipmitool lan set 1 ipaddr <IPADDRESS>
- ipmitool lan set 1 netmask <NETMASK>
- ipmitool lan set 1 defgw ipaddr <GATEWAY_IP>
- ESXi — /ipmitool 사용
(2) genesis restart
10. changing DNS, NTP
