Migrating from BILLmanager4 to BILLmanager 5

From ISPWiki
Jump to: navigation, search

Preliminary steps

Before you migrate to BILLmanager 5, please read carefully the article "BILLmanager 4 and BILLmanager 5 functionality comparison"

Migration from BILLmanager 4 to BILLmanager 5 involves 3 steps:

  • conversion of the billing database data into a new format;
  • import of BILLmanager files to a new server;
  • start of specific BILLmanager 5 functions for user data and files conversion.

Before you migrate to a production server, we strongly recommend to perform test import and check that all the information has been migrated to BILLmanager 5 successfully.

Complete the following steps for test migration:

  • In BILLmanager 4 enable all the supported features, as the conversion procedure may refer to the tables that are present in the database only with certain features enabled.
  • Install a required version of BILLmanager and integration modules. You can import data only on a freshly installed system. We do not recommend that you create users, clients and other objects.
  • Navigate to the "Localizations" module in BILLmanager 5 to add localizations.
  • Suspend cron or disable all the cron jobs associated with BILLmanager. The cron jobs look like the following:
/usr/local/mgr5/sbin/billmaintain ...

or

/usr/local/mgr5/sbin/mgrctl -m billmgr ...

or

/usr/local/mgr5/notify/ntinternal ...
/usr/local/mgr5/notify/ntemail ...
/usr/local/mgr5/notify/ntsms ...
  • Create the /usr/local/mgr5/etc/billmgr.DoNothing file, so that the billing modules won't perform any operations with external systems.
touch /usr/local/mgr5/etc/billmgr.DoNothing

You will get a fully-functional BILLmanager that neither performs any operations with services in the control panels nor sends emails or charge clients.

Once you are done with the test setup, you can upload data to the MySQL (MariaDB) database.

Database import

You can import the database directly from the server where BILLmanager 4 is installed or from the database dump on your local machine.

The first variant has a number of drawbacks:

  • You need to enable remote access for MySQL users (the default value is root)
  • It will slow down BILLmanager 4
  • Data transfer through the network may slow down the data conversion process
  • If the connection goes off, you will have to start the operation from the very beginning
  • New objects that can be added into the database, may refer to non-existing objects

If you choose the second variant, BILLmanager 4 will be blocked while making a damp, and you can repeat the import several times, if needed.

After you prepare the database the data will be imported from execute the following command to convert the data:

/usr/local/mgr5/sbin/billimport --command check --billing billmgr4 --db <db name> [--host <db host> [--user <db user> [--password <db password>]]]

This call will check connection to the database and current conversion rules.

Once completed, the data import will start

/usr/local/mgr5/sbin/billimport --command import --billing billmgr4 --db <db name> [--host <db host> [--user <db user> [--password <db password>]]]

in both commands

  • --command - the action that the billimport utility should perform
  • --billing - a conversational name of the billing system that you are importing data from (when you import data from BILLmanager 4 to the corresponding version of BILLmanager 5 this value is billmgr4, if you import data from BILLmanager 4 Advanced to BILLmanager 5 Corporate - billmgr4adv))
  • --db - the name of the database the data for conversion are selected from
  • --host - the server to retrieve data; the optional parameter. The default value is localhost
  • --user - the username to connect to the database; the optional parameter. The default value is root.
  • --password - the user password; the optional parameter. By default the password is not use.

You can also provide the following parameters:

  • limit - the number of records that will be selected from the database you want to import. The import process includes several steps not to exceed MySQL (MariDB) limits on database packets. The default value is 10000 records.
  • lhost - the server that the data conversion script connects to if you want to upload data to a database that is not used by BILLmanager 5.
  • luser - the username to connect to BILLmanager 5 database. You should specify the username if it differs from the name in the BILLmanager 5 configuration file.
  • lpassword - the user password to connect to BILLmanager 5 database.
  • ldb - the database the data will be imported to (if the name differs from the current database).

During migration billimport selects the data from the source database, converts them into a required format, and saves into the BILLmanager 5 database. Be sure to form the database structure and add data before you start, i.e. the database should look like you have just installed BILLmanager 5 on a fresh server.

During data conversion billimport starts a number of BILLmanager 5 functions, which are executed on the data being used. If you are importing to another database, call the functions manually:

  • fix.docnumber - converts document numbers for companies
  • fix.currency.status - activates the currency for a certain provider
  • fix.orderperiod - activates/disables order periods in product types
  • fix.addon.compound - converts add-ons with subtypes into group add-ons
  • fix.processingmodule.param - encrypts data to connect to processing modules
  • fix.addon.manual - converts add-ons that were manually added to services
  • fix.certificate.cn - transfers canonical names of SSL certificates into service parameters
  • fix.itemip - re-assigns IP addresses according to the new IP addresses system
  • fix.billorder.item - converts user orders that were not processed, into the Cart
  • fix.invoice.realamount - converts the internal structure of invoices into a new format
  • fix.subscription - converts clients' notifications into a new format
  • fix.domain.dopparam - converts domains parameters into a new format
  • fix.account.country - specifies a country for clients based on payer data
  • fix.accountgroup_condition - converts rules for automatic assigning of clients into groups, into a new format
  • fix.department.rights - converts access permissions to functions into a new format, and grants full permissions
  • fix.fraudgateway - converts anti-fraud gateway parameters
  • fix.gateway - converts mail gateway parameters
  • fix.globalindex.rebuild - starts data indexing for quick search
  • fix.import.period - checks services order period and automatic renewal, and removes or changes it, if needed
  • fix.measure - converts units of measure. Removes duplicate values
  • fix.modules - installs missing processing modules, mail gateways, and payment methods
  • fix.pricelist.domain.intname - sets correspondence between domain registration tariff plans and top-level domains
  • fix.profile.localization - corrects users and companies data, adds the documents localisation parameter
  • fix.support.incident - converts information about support incidents into a new format
  • fix.taskdepartment - selects a responsible department for imported tasks
  • fix.tax.expense - recalculates unpaid expenses, if they are added to tariff prices, and recalculates open orders
  • fix.update_expiredate - starts synchronization of updates for licenses
  • fix.itemparamip - imports IP address for virtual machines and dedicated servers into the "IP addresses" module
  • processing.getconfig - retrieves information from configured processing modules

After the data have been imported and the functions have been executed, you can proceed to additional steps.

Special aspects of data import

  • Tax policy:
    • If tax rates were not set up for at least one company, taxes will be added to the service price
    • If taxes were added to the service price, tax rates will be removed for a certain company
  • Add-ons that use subtypes and add-ons with the "Enumeration with quantity" type are converted into "Client can choose" add-ons.
  • Service and IP history will be marked as "History synchronization"
  • If periods that were created in BILLmanager 4 are missing in BILLmanager 5, the system will choose the most similar period

Data that cannot be converted

The following data cannot be converted due to differences in the data storage format:

  • BILLmanager Corporate: information about partners, their clients, and other associated objects;
  • Email attachments;
  • Tariff plan Agreements;
  • Automatic invoices configuration;
  • Information on inquiries;
  • Electronic document management;
  • Knowledge bases;
  • Notifications that were not sent;
  • Running operations with services;
  • Support center statistics;
  • Business hours.

Notes

Clients IDs, their personal accounts, and services remain unchanged during the import process. Billing information, information on services and tariff plans are imported. Notification and documents templates, branding settings, images in templates and other additional files, which are not included into the structure of BILLmanager 4 won't be imported

Support tickets from the Support center section are divided into notifications (news letters) and tickets. Tickets associated with manual processing of services, cannot be imported.

If you convert BILLmanager 4 into a different version of BILLmanager 5, some data may get lost, as the BILLmanager 5 database doesn't have some fields and tables that are used in BILLmanager 4

After finishing database import, conversion of user access rights can be incomplete because it runs in the background. To make sure that this process is over, execute the command:

ps aux | grep billfix

Import of files

Beside information from the database, when migrating from BILLmanager 4 to BILLmanager 5 you need to import attachments to tickets and user configuration files that will be converted into a new format.

To import ticket attachements, use scp or rsync to copy the /usr/local/ispmgr/var/mailattach directory from the server with BILLmanager 4 to the server with BILLmanager 5. Rename it into /usr/local/mgr5/var/ticket_attach and run the function

/usr/local/mgr5/sbin/mgrctl -m billmgr fix.ticket_message_fileattach

The function will convert the attachments into a new format in BILLmanager 5.

To import and convert user configuration files, use scp or rsync to copy the /usr/local/ispmgr/var/userconf directory from the server with BILLmanager 4 to the server with BILLmanager 5. Rename it into /usr/local/mgr5/tmp_userconf and run the function

/usr/local/mgr5/sbin/mgrctl -m billmgr fix.userconf

The function will check the files in the /usr/local/mgr5/tmp_userconf directory, retrieve required information, and save the new configuration files into the /usr/local/mgr5/var/userconf directory.

To import user avatars, copy the /usr/local/ispmgr/skins/userdata/avatars directory from the server with BILLmanager 4 into the /usr/local/mgr5/skins/userdata/avatar_files directory on the server with BILLmanager 5

Additional steps

If BILLmanager 4 used specific message/document templates and plug-ins, complete the following steps before migration:

  • Perform a test data conversion
  • Test and correct the document and message templates, which you then need to save on the remote storage, because after you perform final migration, they will be deleted in the BILLmanager database.
  • Modify and test the plug-ins.

The email and fraud gateways won't be imported. You will need to configure them manually after completing the migration process.

Converting service and IP address history

This feature is available starting from version 5.74.0

During migration you can convert service history, which is kept in the BILLmanager 4 database, and executed by the specific function in BILLmanager 5 due to differences in their formats. The function call:

/usr/local/mgr5/sbin/mgrctl -m billmgr fix.history host=$host user=$user password=$password db=$db

Where

  • $host - IP address of the server where BILLmanager 4 database is located
  • $user - database username
  • $password - database user password
  • $db - database name

The conversion process may take a long time, we don't recommend using BILLmanager 5 at that time to prevent table blocking.