Backups

Owned by Chris (Unlicensed)
Last updated: Aug 08, 2017 by Odysseas Pentakalos

Backups

An essential task in maintaining an instance of OpenEMPI is performing backups of the instance, which includes making backups of the data in the database as well as backups of the software itself. Although you could easily reconstruct your instance by simply creating backups of the configuration files of your instance instead of all the software that comprise the instance, it is faster to bring the instance back using a complete backup of the software.

Backups of the Postgres Database Instance

The following script can be used as a sample for making backups of both the data in the database and of the software. Feel free to customize it to match the particulars of your installation.

#!/bin/bash

app_name=openempi
backup_dir=/home/openempi/backups
backup_filename=$app_name-backup-`date +%Y-%m-%d`
backup_filename_tar="$backup_filename".tar.gz
openempi_home=/sysnet/openempi
openempi_filename_tar=openempi.tar

# Print status
echo "Backing up $appname..."

echo "Archiving Client Registry Software"
cd $openempi_home
tar cf $backup_dir/$openempi_filename_tar .

cd $backup_dir

# Backup all the things - config files and databases
#---------------------------------------------------

# Backup the Postgress database
db_user="openempi"
db_password="openempi"
db_name="openempi"
backup_filename_sql=$backup_filename".sql"
export PGPASSWORD=$db_password
echo "Backing up the Database Instance for OpenEMPI into $backup_filename_sql"
/usr/bin/pg_dump --username=$db_user -w --host=localhost $db_name > $backup_filename_sql

# Archive all the things
#-----------------------
echo "Archiving the dabatase backup and OpenEMPI softare distribution"
cd $backup_dir
tar czf $backup_filename_tar $backup_filename_sql $openempi_filename_tar

# Cleanup
#--------

# Clean up file that aren't needed here...
echo "Cleaning up backup file in $backup_dir/$backup_filename_sql"
rm $backup_dir/$backup_filename_sql $backup_dir/$openempi_filename_tar

# Finalise

# Print status
echo "Done backing up $app_name."

# Display created backup file
echo "Created backup file: " $backup_filename_tar " in " $backup_dir

# Copy to off site server
remote_user="username"
remote_pass="password"
remote_host="www.openempi.org"
remote_dir="/home/openempi"
# scp $backup_dir/$backup_filename_tar $remote_user@$remote_host:$remote_dir
echo "Copied backup file to remote server: " $backup_filename_tar

# Cleanup old backup files
# - keep previous 30 days backups and the backup from the first of every month
find $backup_dir -name "$app_name-backup-*" -mtime +10 -not -name "$app_name-backup*-01.tar.gz" -exec rm {} \;

Backups of the Graph Database

The simplest way to backup the embedded graph database that OpenEMPI uses to store the core data that is managed by the system is by performing a backup of the filesystem data that includes the database files. The location where the database files are stored is specified using the data-directory parameter in the mpi-config.xml file. Make sure you shutdown the OpenEMPI server before performing a backup to ensure that the data does not change while the backup operation is taking place.

You can also perform a backup of just the database files using the backup facility provided by the database. To do that you must first download and extract the appropriate version of OrientDB. It is important that you match the version of OrientDB you download with the version that is embedded with the OpenEMPI version you are using. Newer versions of OrientDB may include changes in the APIs and data model that may not be compliant with OpenEMPI's use of OrientDB. If you are not sure of the version of OrientDB used with your version of OpenEMPI, you can contact us at support@sysnetint.com and we can help you. If you are a licensed customer of OpenEMPI you can contact us through the customer support portal and we will provide you with the appropriate version of the OrientDB distribution for the version of OpenEMPI you are using. Installation of the OrientDB distribution simply consists of extracting the distribution archive somewhere in your filesystem. In the example below we have extracted the OrientDB distribution under the /sysnet/servers/orientdb-community-2.2.17 directory.

First locate the directory where the database files are stored for the entity that you are trying to backup. The default entity is called "person" and the database files for it are stored in the "person-db" directory under the directory pointed to by the data-directory parameter of your OpenEMPI configuration file (mpi-config.xml). In the example below we assume that data directory is set to "/sysnet" so the database files are in the "/sysnet/person-db" directory.

NOTE: You must first shutdown the OpenEMPI instance before attempting to perform a backup. Failure to do so may cause corruption of the database files.

# cd /sysnet/servers/orientdb-community-2.2.25
# bin/console.sh
OrientDB console v.2.2.25 (build 72a35a4bb42c4ca33272e7a59ad4c7a1d3ee859d) https://www.orientdb.com
Type 'help' to display all the supported commands.
Installing extensions for GREMLIN language v.2.6.0
orientdb> connect plocal:/sysnet/person-db openempi openempi
Connecting to database [plocal:/sysnet/person-db] with user 'openempi'...OK
orientdb {db=person-db}> export database openempi-export
Exporting current database to: database openempi-export in GZipped JSON format ...
Started export of database 'person-db' to openempi-export.json.gz...
Exporting database info...OK
Exporting clusters...OK (53 clusters)
Exporting schema...OK (19 classes)
Exporting records...
- Cluster 'internal' (id=0)...OK (records=3/3)
- Cluster 'index' (id=1)...OK (records=0/0)
...

This will create the backup file in the file openempi-export.json.gz

Restoring an Instance

Restoring the Postgres Database

In this section we describe the steps involved in restoring an instance of OpenEMPI from a backup file. The assumption is that you have a previous backup of your running instance that was created the backup script in the previous section. The backup includes both the software that comprise a running instance of OpenEMPI and the data in the database. The other assumption that we are making here is that you are installing an instance of OpenEMPI on a new server which is missing the COTS software that are used to support a running instance of OpenEMPI and all you have is the latest backup file.

  • Install the Postgres database software: This is described in detail here
  • Install the JBoss application server: This is described in detail /wiki/spaces/openempi30/pages/295102
  • Extract the backup: From the OpenEMPI home directory, extract the backup using the command: tar xvfz backup-file.tar.gz. There should now be two more files in your directory, one with an extension (.sql) that comprises the database backup and one with an extension (.tar) which comprises the OpenEMPI software.
  • Restore the data in the database: To achieve this step, use the psql command to connect to your Postgres instance and then load the data from the backup file.
$ psql --user=openempi --host=localhost openempi
Password for user openempi: 
psql (9.1.5)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.
openempi=# \i openempi-backup-2013-01-01.sql
openempi=# \q
  • Restore the OpenEMPI software installation. To achieve this step, you simply need to move to the OpenEMPI home directory and extract the backup file.
$ cd $OPENEMPI_HOME
$ tar xvfz ${LOCATION_OF_BACKUP_FILE}/openempi.tar
  • Check the openempi_env.sh file to make sure that the OPENEMPI_HOME variable points to the correct location of the home directory for the OpenEMPI software. If you restored the software in a different location from its original one, you will need to make this change.

 

Restoring the Graph Database

If you have performed a backup of the database using the process described above, restoring the database from a backup is quite easy.

NOTE: You must first shutdown the OpenEMPI instance before attempting to restore the database.

You will need an installation of the OrientDB software in order to restore the backup. You can follow the instructions under "Backups of the Graph Database" subsection for installation instructions of the OrientDB database. In the instructions below we assume that the directory /sysnet/person-db does not exist and it will be created during the restore process.

# cd /sysnet/servers/orientdb-community-2.2.25
# gunzip openempi-export.json.gz
# bin/console.sh
OrientDB console v.2.2.25 (build 72a35a4bb42c4ca33272e7a59ad4c7a1d3ee859d) https://www.orientdb.com
Type 'help' to display all the supported commands.
Installing extensions for GREMLIN language v.2.6.0
orientdb> create database plocal:/sysnet/person-db
orientdb {db=person-db}> connect plocal:/sysnet/person-db admin admin
orientdb {db=person-db}> import database openempi-export
import database openempi-export
Importing database database openempi-export...
Started import of database 'plocal:/sysnet/person-db' from openempi-export.json...
Non merge mode (-merge=false): removing all default non security classes
orientdb {db=person-db}> connect plocal:/sysnet/person-db openempi openempi

Note that after the database was restored, we were able to connect to the database using the username and password for the database account openempi that was restored along with the database and not using the default administrative account that was created when the database instance was created.

Â