Backup module (isptar, current module starting from 5.52.0

From ISPWiki
Jump to: navigation, search

A new backup system is launched in ISPmanager Lite and Business starting from version 5.52.0.This new system is based on ISPtar (ISPtar: Technical details).

In ISPmanager Lite it will change the DAR-based system.

In ISPmanager Business this system will allow administrator to choose between a simplified version and advanced configuration options and filters.

New features

The main differences from the old backup system:

  • A separate backup copy is created for each user.
  • Filters are not used. Server administrator can exclude file copies, directories, databases (by default, all user data are backed up), or exclude a certain user from the backup.
  • Only one storage can be used, however, you can easily switch between storages. In this case, you won't be able to use backups on the first storage, and can access backups on the second storage.
  • By default, backups are made once a day. The first weekly backup copy (the week starts from Sunday) is Full, other copies - Differential. You can change the backup period in the Cron configuration form.
  • You can set limits on total backup size. Once the limit is reached, the system will delete the oldest copies, and store the equal number of daily and weekly backups, if possible.
  • A backup copy of a disabled user will be made only once after its suspension.
  • If the total backup size is not limited, and there is no free disk space on the storage, the system will delete the oldest backups.
  • The total number of backups that are kept in the system: 7 full weekly backups and 7 differential daily backups. You can change the number with the BackupCountLimit parameter in the etc/ispmgr.conf configuration file.

In version 5.137 you can specify the number of full and daily (differential) backup copies:

BackupCountLimit 7:7

The minimum value for the BackupCountLimit parameter is 2:2. It means that 2 full and 2 differential copiers will be made.

Before version 5.137 you should specify the parameter as follows:

BackupCountLimit 8

The minimum value for the BackupCountLimit parameter is 2. It means that 1 full and 1 differential copiers will be made. .


  • Users can now download their backups, and upload them to server.

ISPtar delivers a number of advantages:

  • A backup copy is divided into small volumes (20 MB in size by default) allowing to reduce recovery time in case of partial data restore, and amount of free disk space to create and restore a backup copy.
  • (will be launched in version 5.55) There is no need to compress some types of files. This enables to reduce CPU usage during backup. Disable file compression via the etc/isptar.conf file.
  • * You can extract data from backup without ISPmanager via the console(see this section for more details)

Switching from old backup system

When you update ISPmanager Lite to new version, a backup module based on DAR will be changed into ISPtar: the same storage will be connected, old backup copies won't be accessible from the web-interface (files are not deleted from storage), they can be converted to a new format by running /usr/local/mgr5/sbin/backup2_conv (this process may take a long time. Make sure you have enough user disk quota to store files after they are converted from DAR into ISPtar).

If you connected the DAR storage and started the conversion process, but neither archives nor errors appeared in var/backup2.conv, you should copy all the information files from the storage into a temporary directory of the control panel, and start the conversion again:

Note: if you import backup copies into a new server, make sure that dar is set up on that server

cd /usr/local/mgr5
cp -pr /<DARstorage>/201*dar var/backup/ispmgr/
cp -pr /<DARstorage>/*info var/backup/ispmgr/
sbin/backup2_conv &


ISPtar does not conflict with the oldest backup module (the one activated with the EnableOldBackup option), and both systems can be used in parallel.

Module configuration

Click Settings in the Backups module to select and configure a storage, set the total number of backups to store, files (you can use regular expressions) and databases that you want to exclude from backup.

You can use the nice and ionice utilities to set priorities. Add the BackupCommandPrefix parameter in the etc/ispmgr.conf configuration file to set priorities. nice -n 10 ionice -c2 -n7 is set by default.

Storage size limit

This field enables to set the maximum size for backup copies on storage (if a local storage is used on ISPmanager Business, the limit will be applied to every cluster node). When connecting a local storage for the first time, starting from version 5.63, the limit of 50% of free disk space will be set. This value can be modified. Starting from version 5.63 the system includes protection from local storage overload: when uploading data to local storage, the system will check the disk, and if less than 5% of disk space is left, will try to delete old archives. If that attampt failes, the backup process will finish with the following error:

backup DEBUG backup2_storage.cpp:119 Available limit with restriction 95 pct '0' mib
backup DEBUG backup2_storage.cpp:120 File size '54' mib
main DEBUG backup2_cp2_server.cpp:354 m_read = 'RELEASE 56686365
main ERROR Upload failed
libmgr ERROR Error: Type: 'Cant split slices' Object: ' ' Value: 'Part name prefix missed for part ' '

Storage scanning

Storage scanning is performed when connecting to a new storage type or when switching to a new directory in the storage.

From version 5.58 the first backup forms a full copy after the storage directory has been changed, regardless full backup copies made in previous days.

User's storage

From version 5.58 users of ISPmanager Business have an opportunity to set up their own remote storage (using the same user diskspace (via FTP/SFTP) is not recommended.) It is possible to set up disk space limit, but database/files exeptions, which adminitrator have set before, will be aplied (because one copy is located in administrator storage, then it is copied to user storage).

If administrator disables backups for a certain user, anyway this user will be able to set his own storage, and backups will be created automaticly and scheduled in his storage. Furthermore, if administrator disables backups, user still can activate his storage and create copies, restore data, and import archives manually.

But, if administrator enables user backups and the backup process fails, the backup copy won't be added into the user storage. In this case the user can should ask his administrator to disable backups in the user edit form.

View old file version

To view contents of an old file, download the file from the backup copy.

Importing data from another provider

It is possible to import backups from another server. You should click the "Import" button -> "Source type" in backups list, select a source and click "Ok" It is possible to import backups from ISPmanager Lite to Business and vice versa.

Starting from version 5.57.0 (released 17.05.2016) you can import archives from cPanel to ISPmanager Lite.

Full user recovery

Administrator can restore a user from backup without creating that user before this operation.

Log in as a newly created user and navigate to "Tools" -> "Backups" -- Click Import -- select a source for import, parameters (if needed), and select the Restore upon import check box. Once the archive is imported, the data will be restored.

Excluding files from a backup copy

Go to the Backups module to choose files you want to exclude from backup. Enter paths relative to a user's home directory, e.g. data/.filemgr-tmp. In filters you can use regular expressions (*[]). The files that you select won't be backed up.

In the field below you can exclude databases (you should enter a full database name in a separate line).

Changing backup start time

By default, backups are made once a day. In the Scheduler module you can change this period for ISPmanager backup task.

Setting backup time

The BackupTimeIterval parameter enables to set backup time. E.g.: BackupTimeIterval 03:00:00-06:00:00 means that the backup process will start from 3 to 6 a.m. The process for a separate user won't be interrupted. If the system failed to back up all users within that period, the process will start the next day. New copies for that day won't be created.

E.g.: on Monday, September 14, 2015 we backed up 10 users from 15, and remaining 5 users will be backed up on Tuesday. The next backup copy for all users will be created on Wednesday, September 16, 2015. User backups will be dated 14.09.2015, although some users were backed up only on September 15, 2015.

Clearing disk space in the storage

You can clear disk space by deleting backup copies you don't need from a list of backups.

You can also set the Total size in the Backups module. Once you are done, old backups will be deleted until the total size reaches the specified value.

If you run ISPmanager Business with a local storage, and there are multiple nodes for backups, this size will be applied to each node. Click the "Location" button to view distribution of backup copies on cluster nodes.

Speeding up backup process

If you want to make a large backup copy, you can change the priority with the BackupCommandPrefix parameter in the etc/ispmgr.conf configuration file. nice -n 10 ionice -c2 -n7. is set by default.

The nice utility invokes a utility, requesting that it be run with a different system scheduling priority. More details can be found here

ionice is used to get and set the class and process priority.

Service files of a backup copy

Once the backup copy has been made, the files 2015-08-12.user.tgz and 2015-08-12.user.info will be created in /usr/local/mgr5/var/backup/ispmgr. They keep service information, and do not contain the backup itself.

The tgz file contains information about backup files.

The info file contains information about its creation date, the name of the last backup file, backup size and type.

Such files make it possible to show backup copies in the list, and reduce time required to access data.

Limit on the mumber of backup copies

The number of backup copies to be saved per user is set with the BackupCountLimit parameter in etc/ispmgr.conf.

Odd number of backup copies

If the BackupCountLimit parameter is an odd number, for example, 15, 8 full backups and 7 daily incremental backups will be created. When the limit is exceeded, daily incremental backup will be deleted first.

I.e. the system will try to create the equal number of full and differential backups. However, there will be more full backup copies (+1).

Objects excluded from backup

Due to technical reasons email filters cannot be backed up.

How often backup can be made

A user can't make a backup copy more often than once an hour, otherwise a previous cached backup copy will be downloaded.

Backup copy of a removed user

After deleting a user whose data were backed up, the "!" icon will be displayed in the list of saved data. This icon contains the date when that user was deleted. It won't be deleted upon user restore.

Correspondence between full and daily backup copies

When the backup process starts for the first time, a full backup copy will be made. Next days (except for Sunday) daily differential copies will be made, and on Sunday a full backup will be created. During the first 14 days 3 full and 11 differential backups will be made. When 14 days are passed, the oldest differential copies will be deleted first. Once a full copy doesn't have any incremental parts, it will be deleted. Thus, 7 full and 7 differential copies will be left in about 2 months.

Deleting backups

Automatically

Old, non-blocked backup copies will be deleted when

  • storage disk size reaches its limit,
  • when disk space limit reaches 95% of its total size (for local storage),
  • when a user's backup limit is reached (BackupCountLimit), the oldest copy will be deleted.

The last full archive, today's archive, and the archive uploaded by user (custom) won't be deleted automatically.

Manually

Administrator can delete backups in the list of backup copies (the "Delete" button). In ISPmanager Business a user can delete backups in the custom storage.

In order to delete a backup copy, in the /usr/local/mgr5 directory you need to specify the BACKUP_TOKEN environmant variable, execute the sbin/backup2_cp --delete command, and specify a path to the info file of the backup copy. The token value is kept in etc/ispmgr.conf in the BackupToken parameter.

A storage type and token are stored in etc/ispmgr.conf, the BackupType and BackupToken parameters correspondingly.

The following example shows deletion of the backup copy of the certain user (username) from the local storage:

BACKUP_TOKEN="type=local;url=/var/backups/";sbin/backup2_cp --delete var/backup/ispmgr/2015-09-08.username.info

The following example shows deletion of all copies of the certain user (username) from the FTP storage:

BACKUP_TOKEN="password=qwerty12345;type=ftp;url=ftp://backup5.reserv.net;username=ftpuser23"; ls -1 var/backup/ispmgr/*username.info | xargs -I {} sbin/backup2_cp --delete {}
sbin/backup2_cp --delete var/backup/ispmgr/2015-09-08.username.info --type local --token /var/backup

The following example shows deletion of all copies in the local storage:

BACKUP_TOKEN="type=local;url=/var/backups/"; ls -1 var/backup/ispmgr/*.info | xargs -I {} sbin/backup2_cp --delete {}

Changing volume size

You can change the volume size in the etc/ispmgr.conf configuration file.

Example:

BackupSliceSize 30M

Restoring/downloading file/database

Administrator cannot restore files. You should log in as user (in the "Backups" module click the "Enter" button) into the backup copy, select a file(s)/database(s), and click "Restore"/"Download".

Starting from ISPmanager Business 5.68.0 (released on 09.08.2016) users can restore/download separate mailboxes.

Reserving system data

/usr/local/mgr5/etc, /usr/local/mgr5/var and /etc will be archived into archives like F2015-09-16.root.tgz in ISPmanager 5 Lite and F2016-06-11.root#system_localhost.tgz and F2016-06-11.root#system_<remote_node_name>.tgz in ISPmanager Business.


Starting from ISPmanager Business 5.59 the system backup copy will aslo include a mysql databse of the ispmgr master-server. Database dump is made in default encoding and is put into archive like F2016-06-11.root#.tgz.

These archives are not available from the panel interface, as they are used very seldom.

In order to restore data, go to the storage, e.g. /var/backup. Select a file of the root user archive (usually, root) made on a required date. Execute the commands to view the archive:

isptar -l F2015-09-16.root.tgz
tar -tf F2015-09-16.root.tgz

Example of extraction of the ispmgr.root.dashboard.xml file:

tar -zxvf F2015-09-16.root.tgz usr/local/mgr5/var/userconf/ispmgr.root.dashboard.xml

How much disk space is required for backups

On the server

The panel's temporary directory: by default /usr/local/mgr5/tmp The panel's system directory: by default /usr/local/mgr5/var/backup/ispmgr The size of one archive part: by default 100Мb, it can be changed through the option BackupSliceSize

If a non-local storage is used:

  • Each part is located one by one into the system directory and loaded into the storage for scheduled backup.

The system directory should contain more space than one part.

  • If you call a user backup by the "New" button, the copy is created in the same way as global backup.

After this, archive parts are downloaded from the storage to the temporary directory, and these parts are added in the archive inside the system directory simultaneously. Upon completion, user gets the archive in his browser, and it is kept in the system directory during 1 hour (cashed archive). The system directory space should be equal to the size of all users archives (If all users want to make a new archive during one hour) The temporary directory needs twice as much space as the size of the archive part.

  • If user imports the archive, it is uploaded to the system directory, unpacked (if necessary conversion), packed and shipped to the repository by parts.

The system directory space should be equal to the size of all users archives (If all users want to make a new archive during one hour)

  • If the archive is downloaded, every part of the archive are downloaded to the temporary directory, there they are grouped into one archive. This archive moves, and user gets it in the browser, and during one hour this result is being kept in the system directory (cashed archive).

The system directory size should be equal to the size of all users archives (If all users want to make a new archive during one hour) The temporary directory needs twice as much space as the size of the archive part.

In user disk quota

You use should have enough disk space for at least one part of backup (by default - 100 MB), because activation is made with user permissions and a part of archive belongs to the user due to security reasons.

Changing temporary directory

If you need to change a temporary directory, copy the /usr/local/mgr5/var/backup/ispmgr directory and mount a required section in /usr/local/mgr5/var/backup/ispmgr.

mount --bind /required/section /usr/local/mgr5/var/backup/ispmgr

If you change a patht to the temporary directory, the backup process may cause issues. We don't recommend using the parameter described below:

You can add the string into the /usr/local/mgr5/etc/ispmgr.conf configuration file to change a path to the temporary directory

path BackupLocalDir <path>

Execute killall core

IF the killall utility is not present, execute

/usr/local/mgr5/sbin/mgrctl -m ispmgr exit

Enabling detailed logging

To enable detailed logging, add the following strings into etc/debug.conf:

backup2.*       9
backup2_import.*        9
backup2_download.*      9
backup2_cp.*    9
restore2.*      9
backup2_cgi.*   9
backup2_conv.*   9
backup2_system.*       9

and execute killall core

You can also enable output of all backup logs:

tail -f /usr/local/mgr5/var/backup2*log /usr/local/mgr5/var/restore2.log

Making a backup copy manually

For the current date

1) business:

cd /usr/local/mgr5 && ./sbin/backup2_pro &

2) lite:

cd /usr/local/mgr5 && ./sbin/backup2 &


For a specific date

Be careful when providing a date, which is too far from the current date (large than BackupCountLimit/2 weeks)! If you make a copy in the future and the other one with the date, which exceeds the previous date, all the copies might be deleted by the system that checks backup size (even if the future backup copy doesn't contains users).


We do not recommend applying the method described unless it is strictly necessary. Be ready for unexpected results!

1) business:

cd /usr/local/mgr5 && ./sbin/backup2_pro --date 2016-05-01 &

2) lite:

cd /usr/local/mgr5 && ./sbin/backup2 --date 2016-05-01 &

For a specific date and user

Be careful when providing a date, which is too far from the current date (large than BackupCountLimit/2 weeks)! If you make a copy in the future and the other one with the date, which exceeds the previous date, all the copies might be deleted by the system that checks backup size (even if the future backup copy doesn't contains users).


We do not recommend applying the method described unless it is strictly necessary. Be ready for unexpected results!

1) business:

cd /usr/local/mgr5 && ./sbin/backup2_pro --date 2016-05-01 user1 &

2) lite:

cd /usr/local/mgr5 && ./sbin/backup2 --date 2016-05-01 user1 &

Extracting data

If you cannot extract data from the control panel, but you have the following files in the storage

F2016-11-02.user.tgz.part1
F2016-11-02.user.tgz.part2

you can add them to achieve by executing the command:

Unix (Linux/FreeBSD/MacOS):

cat F2016-11-02.user.tgz.part1 F2016-11-02.user.tgz.part2  > F2016-11-02.user.tgz

Windows:

copy /b F2016-11-02.user.tgz.part1 + /b F2016-11-02.user.tgz.part2 F2016-11-02.user.tgz

Then you will be able to extract data from the F2016-11-02.user.tgz archive using tools provided by your operating system.

FAQ

401 Unauthorized on Yandex Disk

If you see the following error when downloading a backup copy:

libmgr ERROR Error: Type: 'file' Object: 'open_w' Value: '/usr/local/mgr5/tmp/allmerge_Net1uX'

And Yandex Disk sends something like the following rather tan 2XX/3XX HTTP-code:

HTTP/1.1 401 Unauthorized

This means that after the storage was connected, Yandex account password was changed. Follow the link in the configuration form to reset the storage.

The system deletes old backups

Sometimes old backup copies may be deleted from the storage. If an FTP-storage is used and the system gets timeout after 15 minutes, the backup system considers the storage to be full, and deletes the least important old copy.

User files are not included into backup

If user directories are connected as separate partitions, their files from the home directory (eg. /var/www/user/data/www/*) won't be included into a backup copy.

GoogleDrive. Delete the directory that the backup to cart is connected

If you need to delete the directory in GoogleDrive storage, you first need to disable the backup function in ISPmanager, otherwise it may lead to unexpected results.