VM status monitoring
|Hierarchy:||VMmanager KVM -> VIRTUAL MACHINES|
|VMmanager Cloud -> VIRTUAL MACHINES|
The VM status monitoring subsystem is specially designed for sending information about virtual machines from libvirt to VMmanager KVM/Cloud. The main goals of this subsystem:
- Synchronizes data in the control panel about virtual machines and their actual statuses
- Transmits statuses of virtual machines with minimal time lags
- Registers changes of VM statuses
- Participates in OS installation on a virtual machine
- Controls duplicated of virtual machines
Subsystem consists of 2 services: vmwatch-master и vmwatch-node.
A message queue, which is implemented using ZeroMQ library via the PUSH-PULL pattern, is used for interaction between services.
This service runs on the node, where the control panel is installed
It receives messages on the IP address and port specified in the configuration file, processes them, and calls the corresponding API functions of a control panel functions to send data about changes in statuses of virtual machines.
VMmanager Cloud: when cloud functions are turned on, if the master server changes, the service is stopped and blocked on the old master server. All the services that run vmwatch-master are configured on the new server, and the system reconfigures all vmwatch-master services in order to send message to the new master server.
The service runs on every cluster node. It sends messages to the IP address and port specified in the configuration file. It receives information about changes of VM statuses from libvirtd. If connection with libvirt is lost, the service will be stopped.
System V (System D for CentOS 7) controls service run and autorun. Script setting is executed during VMmanager package installation or upgrade to 5.86 version
vmwatch-master service and its control script are kept in vmmanager-kvm package for VMmanager KVM and in vmmanager-cloud package for VMmanager Cloud. vmwatch-node service and its control script are kept in vmmanager-pkg-vmnode package
Services run automatically during OS loading or configuration process.
After VMmanager installation vmwatch-master service is blocked until the first cluster node is added into the cluster.
Service work parameters are recorded in vmwatch-master.conf and vmwatch-node.conf configuration files, which are kept in /usr/local/mgr5/etc catalogue. Configuration files are created by the control panel automatically.
The configuration file is recorded in the NAME_PARAMETER=VALUE format, one parameter on one line.
vmwatch-master.conf configuration file
It may contain the following parameters:
- disabled - If parameter has yes value, it blocks service execution.
- listen - Parameter value in 'protocol://IP:port' format assigns socket where service recieves messages.
- secret - Undefined symbol line (A-Z, 0-9), parameter value is used for incoming messages authentification.
vmwatch-node.conf configuration file
It may contain the following parameters:
- connect - Parameter value in 'protocol://IP:port' format assigns socket where service sends messages
- secret - Parameter value is used for incoming messages authentification. It has to match with secret parameter value of vmwatch-master configuration file
- node_id - Cluster node identificator which is used in messages for message source identification
Service setting is executed by the control panel in automatic mode if it's necessary:
- During the upgrade to the version which supports monitoring subsystem, and if there are nodes in the cluster.
- When a cluster node is added into the cluster
- During master node migration to the another cluster node (only for VMmanager Cloud if cloud functions are enabled)
IP address assignment, on which vmwatch-master receives messages, is executed in the following way:
- VmwatchListenIp parameter value of configuration file is being checked. Only for VMmanager KVM
- If IP address is not assigned by the parameter, a cluster node is searched, where the control panel is installed.
- If there is no local cluster node, IP address scanning is started at all network interfaces of the server where the control panel is installed, it chooses the first IP address from the list which is not a local one.
Port where vmwatch-master receives messages, is defined by the VmwatchListenPort parameter of the control panel configuration file. Value by default is 1515.
To change IP address and/or port write the relevant parameters in the control panel configuration file, restart the panel, and call API vmwatch.configure function. This function configures all monitoring services according to the indicated parameters.
Service status control
vmwatch-master service status control is run by the panel when periodic API function is called. It is run by the cron each 60 minutes.
If control function detects that service is stopped, it will register the notification about this problem when this problem is being solved, there will be done a repeated configuration of the whole monitoring subsystem
vmwatch-node service status control is run by the panel when periodic API function is called. It is run by the cron each 5 minutes during the process of cluster node control panel status check (vmmini).
If the service is stopped, control function registers the notification about this problem when this problem is being solved, there will be done a repeated configuration on this cluster node
VM status check
In the "Cluster settings/Policy" module you can enable the check of VM statuses.
The system will use statistics data sent by VM monitoring services. The status check function analyses packages that were sent over the network and the number of write requests to a VM disk. Every 10 minutes vmwatch-node collects statistics from all domains and sends it to vmwatch-master. If any of the values equals 0, the corresponding warning will be displayed in the list of virtual machines.
vmwatch-master service log is kept in the /usr/local/mgr5/var/vmwatch-master.log file, vmwatch-node service log is kept in the /usr/local/mgr5/var/vmwatch-node.log file.
We can not identify you and respond to your message.