Backups in ISPmanager: Technical details

From ISPWiki
Jump to: navigation, search

Data storage format

  • Separate users are kept in a storage in separate files.
  • Large archives can be divided into small volumes (the default value is 100MB) allowing to decrease required free disk space, and not to use the whole archive in case of partial data extraction.
  • Storage format - .tgz. It allows to extract data using third-party application.
  • We use our own utility isptar, which creates a copy in the .tgz format and saves a list of files with displacements.

General architecture

A backup process includes a number of interconnected application.

backup2/backup2_pro
It creates a backup queue, if it is not present, extracts strings, one by one, and starts the backup2/backup2_pro process for each string. The process gets user backup settings, forms a draft of a file in the .system directory (real data will be saved into the files after starts), and starts the backup process with isptar.
isptar
An archiver, which reads data from a hard drive and pack them into .tgz archives
backup2_cp
Manages storages (local, FTP, Dropbox, Yandex, Amazon). It reads an archive generated by isptar, and uploads it to storage.

backup2_cp is started from isptar and backup2/backup2_pro (to upload information about a backup copy file listing). It also checks free disk space in a storage.

backup2_system
Backup of such data as: user settings in a control panel, databases, mailboxes, domain names, web-domains, etc. backup2_system extracts data and saves it as special files in the .system directory. backup2/backup2_pro creates a required number of files in that directory. Then, isptar starts the backup2_system process for each file.
restore2/restore2_pro
Extracts data from archive.
backup2_download
Forms a backup copy as a separate file for download. If a backup copy is saved into several volumes, they will be combined into one.
backup2_import
Allows to import an archive formed by backup2_download, and upload it to storage.
backup2_cgi
Allows to download a part of archive as a separate .tgz file. E.g. it can be a dump of separate databases, or a number of files and directories.

Application collaboration tree in ISPmanager Lite

Application collaboration tree in ISPmanager Business

The MASTER backup process is the same as in ISPmanager Lite. MASTER makes a backup copy of metadata and uploads it to storage (even if a backup server role is not set for MASTER). The node.<node role> file is created for every user role in the .system directory. backup2_system starts the backup process for every file on the corresponding node . When the role backup is completed, backup2_system downloads listing and information files (.info) from BACKUP NODE, adds the size into the .info file on MASTER. In general, the collaboration scheme in ISPmanager business looks like this. However, if collaborating roles are located on the same server, the scheme will change. For example, if BACKUP NODE matches NODE, the isptar --client and isptar --server processes are managed by isptar --create.

Data of every role, even if some of them are located on one server, are saved into separate files.

Separate users can be backed up simultaneously, if their main roles are located on different nodes.

Checking free disk space

ISPmanager Business can back up several user simultaneously. To check available disk space, the daemon backup2_cp --server (daemon) will start. This process gets information about occupied disk space from the .info files that are kept on MASTER, and checks that total data size in the storage has never exceeded the limit.

(daemon) does not process requests for space reservation itself, they are received by clients backup2_cp --client - processes that are started on every backup server and MASTER. A client will get a request from the backup2_cp --put process through UNIX socket and passed it through PIPE to (daemon).

If a local storage is used, backup size limit is set for each server. Thus, if you have 2 backup servers, and the limit is set to 1 TB, your backup copies can occupy 1 TB (if all users will be backed up to one node), and 2 TB if backups are located on different nodes. All user backups are always created on one and the same node. If there is insufficient disk space on one node, old backups will be deleted on all nodes, even if some of them have enough disk space.

All user are allowed to store the same number of backups, and this number doesn't depend on a cluster where they are kept.

The number of backup copies may exceed the limit (the default value is 14: 7 daily differential and 7 full backups). First, a backup copy will created, and then an old one will be deleted. The old copy can be deleted earlier, if the storage has insufficient disk space. But the last backup copy cannot be deleted before a new one is created.

If there is insufficient space for a full backup, the backup process will fail.

Errors

Every time backup2/backup2_pro starts the user backup process, information about a backup copy is added into var/ispmgr.backup.cleanup. When starting and completing, backup2_cp --server (daemon) will look through the file, and will delete all files associated with those backups. If the backup process completes successfully, backup2/backup2_pro will delete the record from var/ispmgr.backup.cleanup that was created at the beginning.

Conversion of backup copies

To convert backup copies from dar, execute the sbin/backup2_conv command. dar backups will be placed into the old directory located in the working directory specified in the panel's configuration file.

The default path to old backups after conversion is /usr/local/mgr5/var/backup/ispmgr/old


Old files are never deleted from the storage after conversion. You can delete them manually (before you start, check that the backup process completed successfully), or make sure that you have enough disk space.

During conversion backup copies will be created next to user backup directories causing issues with quotas