Upgrade Instructions

Upgrading from 3.6.x to 3.6.y (a more recent minor release of 3.6)

The following instructions explain how to upgrade an instance of OpenEMPI from 3.6.x to a more recent minor release.

1. Shutdown your Tomcat server instance. Make sure the server has shutdown before moving on to the next step (if you have installed the service file then shutdown the server using systemctl).


/sysnet/openempi/openempi-3.6.x/bin/shutdown.sh


 2. Backup your Postgres and OrientDB databases

 3. Install the 3.6.y instance in its new OPENEMPI_HOME directory. Follow the installation instructions for installing a new instance of OpenEMPI from a distribution. The recommended approach is to install new minor releases in the same directory so that openempi-3.6.x and openempi-3.6.y are in the same directory. On our installations, we follow the convention of installing new minor releases in the /sysnet/openempi directory so that the 3.6.1 release would be installed in the /sysnet/openempi/openempi-3.6.1 directory and the 3.6.0 release would be installed in the /sysnet/openempi/openempi-3.6.0 directory.

4. For the purpose of these instructions, we assume that OPENEMPI_OLD_HOME points to the installation directory of the 3.6.x release

5. First, compare the contents of the following configuration files between the 3.6.x and 3.6.y directories and if they include changes that you have made, then merge those changes into the new version of the file. If you have questions about specific differences between the two versions and are unsure how to proceed, don't hesitate to reach out to us through the customer support portal. For example, the openempi-extension-contexts.properties may have been changed to activate or deactivate specific modules in your instance of OpenEMPI and jdbc.properties may have changed to reflect a change in the database password for the relational or graph databases.


bin/setenv.sh
openempi-entity-3.6.x/conf/openempi-extension-contexts.properties
openempi-entity-3.6.x/conf/log4j.properties
openempi-entity-3.6.x/conf/jdbc.properties
openempi-entity-3.6.x/conf/applicationContext-resources.xml
openempi-entity-3.6.x/conf/applicationContext-services.xml

6. Copy the following configuration files from the 3.6.x to the 3.6.y directory.

NOTES:

  • Keep in mind that you may have to modify the mpi-config.xml file to reflect the change in directories for a few of the configuration parameters.
  • We use the convention of naming file loader mapping files file-loader-map-{VARIABLE].xml where VARIABLE is a unique name that identifies the specific mapping file. You only need to copy mapping files that are unique to your installation. We usually setup the data-directory and fileRepository parameters to point to directories relative to the /sysnet/openempi root directory so that we don't have to modify these settings across versions.
  • The convention for naming the internal model parameters for the probabilistic matching algorithm is to use the entity name (by default person) followed by FellegiSunterConfiguration.ser. If you are hosting multiple entities on your instance, you will need to copy the configuration files for the other entities beyond the person entities shown explicitly below.


openempi-entity-3.6.x/conf/mpi-config.xml
openempi-entity-3.6.x/conf/file-loader-map-*.xml
openempi-entity-3.6.x/conf/person_FellegiSunterConfiguration.ser


7. Run any database upgrade scripts that are specific to the new version you are upgrading to. Not every release includes a database upgrade script but if there is one, it would be in the conf directory under OPENEMPI_HOME. For example, version 3.6.1 has a database upgrade script that needs to be run so, when upgrading from 3.6.0 to 3.6.1 you need to do the following:

psql --user=openempi --password --host=localhost
\i update_database_schema-3.6.1.sql

8. Start your instance of OpenEMPI using the startup script from the new release directory. If you are using a service startup script to automatically start your instance of OpenEMPI, make sure you update the script to reflect its new directory location (this is usually installed in /etc/systemd/system/openempi.service).


/sysnet/openempi/openempi-3.6.y/bin/startup.sh

Upgrading from 3.5.x to 3.5.y (a more recent minor release of 3.5)

The following instructions explain how to upgrade an instance of OpenEMPI from 3.5.x to a more recent minor release.

1. Shutdown your Tomcat server instance. Make sure the server has shutdown before moving on to the next step.

/sysnet/openempi/openempi-3.5.x/bin/shutdown.sh


 2. Backup your Postgres and OrientDB databases

 3. Install the 3.5.y instance in its new OPENEMPI_HOME directory. Follow the installation instructions for installing a new instance of OpenEMPI from a distribution. The recommended approach is to install new minor releases in the same directory so that openempi-3.5.x and openempi-3.5.y are in the same directory. On our installations, we follow the convention of installing new minor releases in the /sysnet/openempi directory so that the 3.5.3 release would be installed in the /sysnet/openempi/openempi-3.5.3 directory and the 3.5.4 release would be installed in the /sysnet/openempi/openempi-3.5.4 directory.

4. For the purpose of these instructions, we assume that OPENEMPI_OLD_HOME points to the installation directory of the 3.5.x release

5. First, compare the contents of the following configuration files between the 3.5.x and 3.5.y directories and if they include changes that you have made, then merge those changes into the new version of the file. If you have questions about specific differences between the two versions and are unsure how to proceed, don't hesitate to reach out to us through the customer support portal. For example, the openempi-extension-contexts.properties may have been changed to activate or deactivate specific modules in your instance of OpenEMPI and jdbc.properties may have changed to reflect a change in the database password for the relational or graph databases.

bin/setenv.sh
openempi-entity-3.5.x/conf/openempi-extension-contexts.properties
openempi-entity-3.5.x/conf/log4j.properties
openempi-entity-3.5.x/conf/jdbc.properties
openempi-entity-3.5.x/conf/applicationContext-resources.xml
openempi-entity-3.5.x/conf/applicationContext-services.xml

6. Copy the following configuration files from the 3.5.x to the 3.5.y directory.

NOTES:

  • Keep in mind that you may have to modify the mpi-config.xml file to reflect the change in directories for a few of the configuration parameters.
  • We use the convention of naming file loader mapping files file-loader-map-{VARIABLE].xml where VARIABLE is a unique name that identifies the specific mapping file. You only need to copy mapping files that are unique to your installation.
  • The convention for naming the internal model parameters for the probabilistic matching algorithm is to use the entity name (by default person) followed by FellegiSunterConfiguration.ser. If you are hosting multiple entities on your instance, you will need to copy the configuration files for the other entities beyond the person entities shown explicitly below.
openempi-entity-3.5.x/conf/mpi-config.xml
openempi-entity-3.5.x/conf/file-loader-map-*.xml
openempi-entity-3.5.x/conf/person_FellegiSunterConfiguration.ser

7. Start your instance of OpenEMPI using the startup script from the new release directory. If you are using a service startup script to automatically start your instance of OpenEMPI, make sure you update the script to reflect its new directory location (this is usually installed in /etc/systemd/system/openempi.service).

/sysnet/openempi/openempi-3.5.y/bin/startup.sh


Upgrading to 3.5.0 from 3.4.x

The following instructions explain how to upgrade an instance of OpenEMPI from the 3.4.x release to 3.5.0.

1. Shutdown your Tomcat server instance

 2. Backup your Postgres and OrientDB databases

 3. Install the 3.5.0 instance in its new OPENEMPI_HOME directory. Follow the installation instructions for installing a new instance of OpenEMPI from a distribution.

 4. For the purpose of these instructions, we assume that OPENEMPI_OLD_HOME points to the installation directory of the 3.4.x release

 5. Upgrade the database to the 3.5.0 release by running the update_database_schema-3.5.0.sql script. Depending on how you have setup your Postgres database software, this can be done either using the command line tool psql or using the Pgadmin utility. The update_database_schema-3.5.0.sql script resides in the conf directory of the 3.5.0 distribution. You also need to install the report definitions onto your database.

NOTE: If you are using the h2 database instead of Postgres, you will need to relocate the two files openempidb.mv.db and openempidb.trace.db (may not be present on some instances) from the openempi-entity-3.5.x directory to the openempi-entity-3.5.y directory. To run the update scripts against the h2 database, you need to start the console: http://www.h2database.com/html/tutorial.html#tutorial_starting_h2_console

psql --user=openempi --password --host=localhost
\i update_database_schema-3.5.0.sql

6. Modify your Tomcat configuration to point to the 3.5.0 distribution files.

a. Change setenv.sh in the bin directory of the Tomcat server to point to the new OPENEMPI_HOME

export OPENEMPI_HOME=/tmp/openempi-entity-3.5.0

b. Changeopenempi-admin.xml in the conf/Catalina/localhost directory to point to the new OPENEMPI_HOME from

<Context path="/openemp-admin" docBase="/tmp/openempi-entity-3.4.0/openempi-entity-webapp-web-3.4.x.war"/>

to

<Context path="/openemp-admin" docBase="/tmp/openempi-entity-3.5.0/openempi-entity-webapp-web-3.5.0.war"/>

7. Copy any configuration files you may have changed from the 3.4.0 directory to the 3.5.0 directory. NOTE: Before copying the following files from the 3.4.0 directory to the 3.5.0 directory, compare the two files to ensure that no new elements were added in the new release. If there are new elements, you must merge the local changes you have made in the 3.4.0 version of the file into the 3.5.0 version of the file. Copy the mpi-config.xml file and any files with a .ser extension (e.g.: person_FellegiSunterConfiguration.ser) from the 3.4.0 directory to the 3.5.0 directory.

openempi-extension-contexts.properties
log4j.properties
applicationContext-resources.xml
applicationContext-services.xml

8. Clear the tomcat work directory (Usually under ${TOMCAT_HOME}/work/Catalina/localhost/openempi-admin). This may vary depending on how you have deployed OpenEMPI.

 rm -rf /tmp/servers/apache-tomcat-8.0.44/work/Catalina/localhost/openempi-admin

Upgrading to 3.4.0 from 3.3.x

The following instructions explain how to upgrade an instance of OpenEMPI from the 3.3.x release to 3.4.0.

1. Shutdown your Tomcat server instance

 2. Backup your Postgres and OrientDB databases

 3. Install the 3.4.0 instance in its new OPENEMPI_HOME directory. Follow the installation instructions for installing a new instance of OpenEMPI from a distribution.

 4. For the purpose of these instructions we assume that OPENEMPI_OLD_HOME points to the installation directory of the 3.3.x release

 5. Upgrade the database to the 3.4.0 release by running the update_database_schema-3.4.0.sql script. Depending on how you have setup your Postgres database software, this can be done either using the command line tool psql or using the Pgadmin utility. The update_database_schema-3.4.0.sql script resides in the conf directory of the 3.4.0 distribution. You also need to install the report definitions onto your database.

psql --user=openempi --password --host=localhost
\i update_database_schema-3.4.0.sql

6. Modify your Tomcat configuration to point to the 3.4.0 distribution files.

a. Change setenv.sh in the bin directory of the Tomcat server to point to the new OPENEMPI_HOME

export OPENEMPI_HOME=/tmp/openempi-entity-3.4.0

b. Changeopenempi-admin.xml in the conf/Catalina/localhost directory to point to the new OPENEMPI_HOME from

<Context path="/openemp-admin" docBase="/tmp/openempi-entity-3.3.0/openempi-entity-webapp-web-3.3.x.war"/>

to

<Context path="/openemp-admin" docBase="/tmp/openempi-entity-3.4.0/openempi-entity-webapp-web-3.4.0.war"/>

7. Copy any configuration files you may have changed from the 3.3.0 directory to the 3.4.0 directory

mpi-config.xml
openempi-extension-contexts.properties
log4j.properties
applicationContext-resources.xml

8. Clear the tomcat work directory (Usually under ${TOMCAT_HOME}/work/Catalina/localhost/openempi-admin). This may vary depending on how you have deployed OpenEMPI.

rm -rf /tmp/servers/apache-tomcat-8.0.44/work/Catalina/localhost/openempi-admin

Upgrade Instructions

Upgrading to 3.3.x from an earlier 3.3 release

The following instructions explain how to upgrade an instance of OpenEMPI to a newer distribution. The distribution follows the naming convention openempi-entity-3.3.x-openempi-entity.tar.gz or openempi-entity-3.3.x-openempi-entity.zip.

1. Shutdown your Tomcat server instance

 2. Backup your Postgres and OrientDB databases

 3. Extract the new war file from the distribution. The war file follows the naming convention: openempi-entity-webapp-web-3.3.x.war. Copy the new war file to your OPENEMPI_HOME directory.

4. Modify your Tomcat configuration to point to the new war file that you just.

- Changeopenempi-admin.xml in the conf/Catalina/localhost directory to point to the new war file. The only part you should need to change is the version number of the war file.

<Context path="/openemp-admin" docBase="/tmp/openempi-entity-3.3.x/openempi-entity-webapp-web-3.3.x.war"/>


Upgrading to 3.3.0 from 3.2.x

The following instructions explain how to upgrade an instance of OpenEMPI from the 3.2.x release to 3.3.0. The instructions assume that you are using version 3.2.0 currently but if you are using a later release prior to 3.3.0, such as the 3.2.1 release, then the same instructions apply.

1. Shutdown your Tomcat server instance

2. Backup your Postgres and OrientDB databases

3. Install the 3.3.0 instance in its new OPENEMPI_HOME directory. Follow the installation instructions for installing a new instance of OpenEMPI using either the installer or from a distribution.

4. For the purpose of theseinstructionswe assume that OPENEMPI_OLD_HOME points to the installation directory of the 3.2.0 release

5. Upgrade the database to the 3.3.0 release by running the update_database_schema-3.3.0.sql script. Depending on how you have setup your Postgres database software, this can be done either using the command line tool psql or using the Pgadmin utility. The update_database_schema-3.3.0.sql script resides in the conf directory of the 3.3.0 distribution.


psql --user=openempi --password --host=localhost
\i update_database_schema-3.3.0.sql

6. Copy thempi-config.xml, log4j.properties, jdbc.properties and any other configuration files you may have customized in your 3.2.0 distribution to the conf directory of your 3.3.0 installation. You may need to compare the contents of the two files with each other to ensure that any changes that were made, are applied to the configuration files of your 3.3.0 installation. Make sure that the data-directory parameter of thempi-config.xml file points to the same directory where your OrientDB database files for the 3.2.0 installation were stored.

7. The new installer for installing the 3.3.0 release configures OpenEMPI with the embedded h2 database instead of the Postgres database. In order to get the new installation to use your Postgres database instance from the 3.2.0releaseyou need to modify thejdbc.properties file in the conf directory to point to your Postgres database. Thefollowillustrates what thejdbc.properties file should look like but you may need to customize the parameters to match the parameters you are using for your Postgres instance (for example you may be using a different username/password/host/port).


jdbc.driverClassName=org.postgresql.Driver
jdbc.url=jdbc\:postgresql\://localhost/openempi
#jdbc.driverClassName=org.h2.Driver
#jdbc.url=jdbc:h2:tcp\://localhost/openempidb
jdbc.username=openempi
jdbc.password=openempi

8. You should now be able to use the exporter utility to export the data from your 3.2.0 installation. You must first go to the directory where you have installed the 3.3.0 version of OpenEMPI. The instructions below assume that <OPENEMPI_HOME> is the full path to the installation directory of your 3.3.0 instance. To make sure you are at the right place, check to make sure that the bin sub-directory includes the exporter.sh and importer.sh utilities. The following instructions run the exporter utility to export the entity "person" using the OpenEMPI administrative user "admin" with password "admin". The utility will generate a lot of output and will terminate when it has completed exporting the data. The amount of time it takes to run is proportional to the amount of data you have stored in your database and the hardware configuration of your server.

$ cd <OPENEMPI_HOME>
$ bin/exporter.sh -e person -u admin -p admin
...

The export utility will create three files in thefileRepositorydirectory of your OpenEMPI instance. The two files with the extension '.avro' store the raw data for your records and associated record links, and the single file with the extension '.avsc' contains the schema for the 'person' entity that you exporter. If you are using additional entities on your OpenEMPI instance, you will need to repeat the process for each entity.

-rw-rw-r-- 1 openempi openempi 2507844 Jan  5 19:48 person-2017-01-05-19-48-21.avro
-rw-rw-r-- 1 openempi openempi    5546 Jan  5 19:48 person-2017-01-05-19-48-21.avsc
-rw-rw-r-- 1 openempi openempi   77288 Jan  5 19:48 recordLinks-2017-01-05-19-48-21.avro

9. You should now modify the data-directory parameter in thempi-config.xml file to point to a directory where you want to store the data of your 3.3.0 release. The data directory parameter should point to a directory that youhavewrite access to and exists but which is empty. You can now run the importer utility to populate the database directory for your 3.3.0 release with the data from the 3.2.0 release. In addition to specifying the entity, usernameandcredentials for the OpenEMPI account, you must also specify the source for the record and link data which should be the two files that were created during the export process.

$ bin/importer.sh -e person -u admin -p admin -r fileRepository/person-2017-01-05-19-48-21.avro \
	 -l fileRepository/recordLinks-2017-01-05-19-48-21.avro

The process will generate a lot of output but should complete with no errors. Like the exportprocessthe import process may take a while to finish, depending on the amount of data and hardware configuration of your server.

If you used the installer to install the 3.3.0 release then you are done with the upgrade process. For those of you using the distribution to upgrade your 3.3.0 release there are a couple of steps still left to do.

10. Modify your Tomcat configuration to point to the 3.3.0 distribution files.

a. Change setenv.sh in the bin directory of the Tomcat server to point to the new OPENEMPI_HOME

export OPENEMPI_HOME=/sysnet/servers/openempi-entity-3.3.0

b. Changeopenempi-admin.xml in the conf/Catalina/localhost directory to point to the new OPENEMPI_HOME from

<Context path="/openemp-admin" docBase="/sysnet/servers/openempi-entity-3.1.0/openempi-entity-webapp-web-3.1.0.war"/>

to

<Context path="/openemp-admin" docBase="/sysnet/servers/openempi-entity-3.3.0/openempi-entity-webapp-web-3.3.0.war"/>

Upgrading to 3.2.0 from 3.1.0

The following instructions explain how to upgrade an instance of OpenEMPI from the 3.1.0 release to 3.2.0.

1. Shutdown your Tomcat server instance

 2. Backup your Postgres and OrientDB databases

 3. Install the 3.2.0 instance in its new OPENEMPI_HOME directory. Follow the installation instructions for installing a new instance of OpenEMPI from a distribution.

 4. For the purpose of these instructions we assume that OPENEMPI_OLD_HOME points to the installation directory of the 3.1.0 release

 5. Upgrade the database to the 3.2.0 release by running the update_database_schema-3.2.0.sql script. Depending on how you have setup your Postgres database software, this can be done either using the command line tool psql or using the Pgadmin utility. The update_database_schema-3.2.0.sql script resides in the conf directory of the 3.2.0 distribution. You also need to install the report definitions onto your database.

psql --user=openempi --password --host=localhost
\i update_database_schema-3.2.0.sql
\i report_definitions-entity.sql

6. Modify your Tomcat configuration to point to the 3.2.0 distribution files.

a. Change setenv.sh in the bin directory of the Tomcat server to point to the new OPENEMPI_HOME

export OPENEMPI_HOME=/tmp/openempi-entity-3.2.0

b. Changeopenempi-admin.xml in the conf/Catalina/localhost directory to point to the new OPENEMPI_HOME from

<Context path="/openemp-admin" docBase="/tmp/openempi-entity-3.1.0/openempi-entity-webapp-web-3.1.0.war"/>

to

<Context path="/openemp-admin" docBase="/tmp/openempi-entity-3.2.0/openempi-entity-webapp-web-3.2.0.war"/>

7. Copy any configuration files you may have changed from the 3.1.0 directory to the 3.2.0 directory

mpi-config.xml
openempi-extension-contexts.properties
log4j.properties