Upgrading from 3.5.x to 3.6.0

Upgrading from 3.5.x to 3.6.y

The following instructions explain how to upgrade an instance of OpenEMPI from 3.5.x to the 3.6.0 release (or latest 3.6.y release). If you are upgrading to a more recent release of the 3.6.0 stream, please replace 3.6.0 below with 3.6.y. Note that the 3.6.0 release of OpenEMPI includes considerable changes to the application, including changes to the REST API that introduce incompatibilities with the 3.5.x releases. We strongly recommend that you perform this upgrade on a test or development instance and fully test your integration with OpenEMPI first before upgrading your production instance to this version. Before attempting this upgrade make sure you have a complete backup of your instance data and configuration information.

1. Shutdown your Tomcat server instance. Make sure the server has shutdown before moving on to the next step.

 /sysnet/openempi/openempi-3.5.x/bin/shutdown.sh

2. Backup your Postgres and OrientDB databases. You can find instructions on how to do that here.

3. Install the 3.6.0 instance in its new OPENEMPI_HOME directory. Follow the installation instructions for installing a new instance of OpenEMPI from a distribution that can be found here. The recommended approach is to install new releases in the same directory so that openempi-3.6.0 and openempi-3.5.y are in the same directory. On our installations, we follow the convention of installing new releases in the /sysnet/openempi directory so that the 3.6.0 release would be installed in the /sysnet/openempi/openempi-3.6.0 directory and the 3.5.7 release would be installed in the /sysnet/openempi/openempi-3.5.7 directory.

4. For the purpose of these instructions, we assume that OPENEMPI_OLD_HOME points to the installation directory of the 3.5.x release

5. First, compare the contents of the following configuration files between the 3.5.x and 3.6.0 directories and if they include changes that you have made, then merge those changes into the new version of the file. If you have questions about specific differences between the two versions and are unsure how to proceed, don't hesitate to reach out to us through the customer support portal. We can perform the merge between the configuration files for you. For example, the openempi-extension-contexts.properties may have been changed to activate or deactivate specific modules in your instance of OpenEMPI and jdbc.properties may have changed to reflect a change in the database password for the relational or graph databases.

bin/setenv.sh
openempi-entity-3.5.x/conf/openempi-extension-contexts.properties
openempi-entity-3.5.x/conf/log4j.properties (not likely to have local changes for your environment)
openempi-entity-3.5.x/conf/jdbc.properties (more likely to have local changes for your environment)
openempi-entity-3.5.x/conf/applicationContext-resources.xml (more likely to have local changes for your environment)
openempi-entity-3.5.x/conf/applicationContext-services.xml (not likely to have local changes for your environment)

6. Copy the following configuration files from the 3.5.x to the 3.6.y directory.

NOTES:

Keep in mind that you may have to modify the mpi-config.xml file to reflect the change in directories for a few of the configuration parameters.
We use the convention of naming file loader mapping files file-loader-map-{VARIABLE].xml where VARIABLE is a unique name that identifies the specific mapping file. You only need to copy mapping files that are unique to your installation.
The convention for naming the internal model parameters for the probabilistic matching algorithm is to use the entity name (by default person) followed by FellegiSunterConfiguration.ser. If you are hosting multiple entities on your instance, you will need to copy the configuration files for the other entities beyond the person entities shown explicitly below.

openempi-entity-3.6.0/conf/mpi-config.xml
openempi-entity-3.6.0/conf/file-loader-map-*.xml
openempi-entity-3.6.0/conf/person_FellegiSunterConfiguration.ser

7. Migrate the Graph database to the 3.6.0 release. NOTE: This is a very important step in moving to the 3.6.0 release or later releases. Failure to perform this step can cause irreversible corruption to your database. In the following instructions we make the assumption that the data directory for the 3.5.7 (or the 3.5.x release of OpenEMPI you are upgrading from) is at: /sysnet/openempi/openempi-3.5.7/data.Your data directory is specified in the data-directory parameter of your mpi-config.xml file.

Download and extract the 3.0.28 release of OrientDB community edition or later (You can download the 3.0.28 release from here.)

$ cd /tmp
$ wget https://s3.us-east-2.amazonaws.com/orientdb3/releases/3.0.28/orientdb-3.0.28.tar.gz
$ tar xf orientdb-3.0.28.tar.gz

Export your current database using the following steps below. This process may take a while to run depending on the size of your database and the hardware configuration of your server. Make sure you have sufficient disk space and available memory for this operation.

$ cd /sysnet/openempi/openempi-3.5.7/data
$ /tmp/orientdb-3.0.28/bin/console.sh
OrientDB console v.3.0.28 - Veloce (build b636a4c891bc1b0f93df5bb16ec72dedee7a63e7, branch 3.0.x) https://www.orientdb.com
Type 'help' to display all the supported commands.
orientdb> connect plocal:./person-db openempi openempi

Connecting to database [plocal:./person-db] with user 'openempi'...OK
orientdb {db=person-db}> export database person-db
Database export completed in XXXXXXms
orientdb {db=person-db}> quit

The above step should have created a file called person-db.json.gz in the /sysnet/openempi/openempi-3.5.7/data directory. You should now import the data into a new database using the following steps.

$ /tmp/orientdb-3.0.28/bin/console.sh
$ cd /sysnet/openempi/
$ mkdir tmp
$ cd tmp
$ mv /sysnet/openempi/openempi-3.5.7/data/person-db.json.gz .
$ /tmp/orientdb-3.0.28/bin/console.sh 
Here
OrientDB console v.3.0.28 - Veloce (build b636a4c891bc1b0f93df5bb16ec72dedee7a63e7, branch 3.0.x) https://www.orientdb.com
Type 'help' to display all the supported commands.
orientdb> create database plocal:./person-db admin admin

Creating database [plocal:./person-db] using the storage type [PLOCAL]...
Database created successfully.

Current database is: plocal:./person-db
orientdb {db=person-db}> import database person-db.json.gz

Database import completed in XXXXX ms
orientdb {db=person-db}> quit

All that is left to do now is copy the person-db directory from the /sysnet/openempi/tmp directory to the data-directory specified in your mpi-config.xml file. In the example below we will make the assumption that the data-directory parameter is set to

$ cd /sysnet/openempi/openempi-3.6.0
$ rm -rf data/person-db    # Remove the target directory just in case it is already there.
$ cp -r ../tmp/person-db data   # Above we restored the database in the /sysnet/openempi/tmp directory

8. Start your instance of OpenEMPI using the startup script from the new release directory. If you are using a service startup script to automatically start your instance of OpenEMPI, make sure you update the script to reflect its new directory location (this is usually installed in /etc/systemd/system/openempi.service).

 /sysnet/openempi/openempi-3.6.0/bin/startup.sh