OpenBIS for Dummies
If you have considered moving from paper to electronic lab notebooks as we are at the moment, you might have come across the OpenBIS solution. You want to install OpenBIS, but have not clue how to go about it? I didn't have much of a clue either and therefore tried repeatedly until I succeeded. Luckily, I got some help from the ETHZ OpenBIS and our local computing support team, but obviously they cannot compensate for the lack of insight into Jetty and postgresql...
If you just quickly want to try it, it might be easier to download the VirtualBox image, where everything is preinstalled and preconfigured (https://wiki-bsse.ethz.ch/display/bis/openBIS+ELN-LIMS+Virtual+Machine). However, I didn't have any fast hardware to run the VirtualBox image and concluded that it would be fast enough on bare metal using an old computer.
The notes below are written from memory and from the final configuration files that worked. However, I will still setup the server from scratch executing only what I have been writing down below in order to make sure I have not missed anything important. However, in the meantime I want to get this information out. I would have been happy if I had found some "OpenBIS installation for Dummies" instructions.
In order to have a very long support, I chose for the installation Ubuntu 16.04 Server (the VirtualBox image uses Ubuntu 14.04 Desktop) and the latest version of OpenBIS 16.05.1 (you need to apply for an account to be able to download the software). The electronic lab notebook (ELN) plugin comes bundled with that release. Doing that, you have a problem with Java 7 support (it is not going to maintained for long and I will try to run OpenBIS ELN with Java 8 soon, but this walk-through uses a Java 7 PPA for Ubuntu 16.04).
1. Do a fresh install of Ubuntu 16.04.1 Server. Create during the install the admin user "openbis". When it asks for software selection, choose OpenSSH and Postgresql in addition to standard system utilities. Upon first login (as openbis user), update and upgrade all the software to the latest version and install emacs (or whatever text editor you prefer) and unzip:
sudo apt install unzip emacs24-nox
2. Install Java 7. Thi sis just one way how to do it:
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get update
sudo apt-get install openjdk-7-jre-headless
3. To setup postgresql correctly, change the configuration file /etc/postgres/9.5/main/pg_hba.conf. All lines ending in peer, the peer should be changed into trust! After that, reload postrgresql:
sudo systemctl reload postgreql
4. If you are installing to a virtual machine like virtualbox, it would make sense to install the guest utilities as this will make you life easier (virtualbox-guest-utils). Make a shared (permanent, automout folder) and add openbis to the vboxsf group:
sudo usermod -a -G vboxsf openbis
4. Place the compressed openBIS installer file into the home directory of the openbis user.
5. Untar/gzip the installer:
tar -xvzf openBIS-installation-standard-technologies-S233.0-r36799.tar.gz
6. Change into the uncompressed directory:
cd openBIS-installation-standard-technologies-S233.0-r36799
7. Change the following things in the console.properties file:
INSTALL_PATH=/home/openbis/
DSS_ROOT_DIR=/home/openbis/data
ELN-LIMS = true
PATHINFO_DB_ENABLED = true
INSTALLATION_TYPE = server
For this try I have not changed the password of the keystore, but you should do it for security reasons if you use the server for production!
8. Run the installer (not as root, but as openbis user):
./run-console.sh
9. When the installer asks to enter the password for the openBIS 'admin' user, type in the password you want to use when logging in as admin into the web interface.
10. When installation is done, go to ~/openbis/servers/openBIS-server/jetty/etc and edit the file service.properties. You actually have not to edit anything for the installation to work, but we needed to configure the system for login authentication via Ldap.
authentication-service = file-authentication-service => authentication-service = file-ldap-authentication-service
If you use ldap alone (i.e. authentication-service = ldap-authentication-service), you need to have some special users in the ldap directory. We are authenticating via our university's ldap serer only regular users of the system and hence, e.g. the admin user needs to be authenticated locally. This admin user is setup during the install, but you need to have both file and ldap authentication active for this to work.
This is our ldap server address as specified in the service.properties file. It contains the authentication base, which you need to ask from your sysadmins:
ldap.server.url = ldap://ldap2015.it.helsinki.fi/OU=people,DC=helsinki,DC=fi
This is the ldap user and his password on the ldap server (don't ask me why the ldap people use such complicated word monster for such a simple concept):
ldap.security.principal.distinguished.name = OU=openbis,OU=login,DC=helsinki,DC=fi
ldap.security.principal.password = PASSWORD
The following paramters are specific to our ldap server. We use OpenLDAP and since OpenBIS seems not to support TLS, we use ssl:
ldap.security.protocol = ssl
ldap.security.authentication-method = simple
ldap.queryTemplate = (&(%s))
OpenBIS requires https. You can either leave the self-signed certificate (provided by the ETHZ) or install an own certificate. If you keep the self-signed certificate, all users will get a warning when they try to log into the web service. There seems to be an additional problem as the uploading of files seems not to work with the self-signed certificate since the https request via port 8444 doesn't result in an intercept that is presented in the browser window to users to override.
Hence we had no choice but to get a real certificate, which is luckily easier today than still one year ago due to the letsencrypt folks. Nevertheless, getting the certificate stuff right took us some time. We are operating the OpenBIS server inside the University network and it is not visible from the outside. Users who want to access it from outside need to use a VPN.
Therefore we had to install a temporary "fake" server with the same name on a publicly reachable IP and generate a letsencrypt certificate (letsencrypt is not yet automated for jetty). I used the automated method for apache2 on my own server at Digital Ocean and then retrieved the two important files (fullchain1.pem and privkey1.pem) from the /etc/letsencrypt/archive/eln.jeltsch.org directory and moved them over to our OpenBIS server. To convert them into the correct format for the java keystore, I used the following commands:
openssl pkcs12 -export -out keystore.pkcs12 -in fullchain1.pem -inkey privkey1.pem
keytool -importkeystore -srckeystore keystore.pkcs12 -srcstoretype PKCS12 -destkeystore keystore.jks
The first command asks for an export password. It doesn't matter what you use (I used 12345678). The second command asks for a destination keystore password. Enter here "changeit" if you have not changed the default keystore password ("changeit") during the setup. Then it also asks you for the source keystore password (which is the 12345678 that I just used above).
Then you still have to add the contents of this keystore to the already exisiting keystore. There are probably many ways to do this, but I used a graphical tool called Keystore Explorer. You simply open both keystores (you need the "changeit" password for this), you delete the existing entry for ETHZ and add the only (letsencrypt) entry from the newly generated keystore.
Replace the files "keystore" and "openBIS.keystore" in the directory ~/openbis/servers/openBIS-server/jetty/etc/ with the modified keystore. Both files are identical, I don't know atm the relevance of this duplication. You also have to replace ~/openbis/servers/datastore_server/etc/openBIS.keystore with the new version of the keystore.
For some reason, uploading did not work and in order to enable uploading, the service.properties of the datastore server needs to specify the host-address of the datastore server:
https://eln needs => https://eln.jeltsch.org. Since we used a different host for the certificate generation, there was a host name mismatch and we had to override manually the hostname settings:
Change it in the files /etc/hostname and /etc/hosts, then reboot. However, the installer took only the first part of the full name for the service.properties files of the datastore server. Our server's name is eln.jeltsch.org and it resulted in https://eln, which did not resolve in our network, since we got the letsencrypt certificate signed on another server. After we changed the service.properties files and after rebooting we were finally ready to test the server:
11. Starting the server: Log into the server as openbis user.
cd ~/openbis/bin/
./allup.sh
12. On another computer, navigate in your web browser to "https://eln.jeltsch.org:8443/openbis/webapp/eln-lims". If you have not installed a publicly trusted certificate, the browser will now warn you that the connection is not secure. Just click "Advanced" and confirm the security exception. You need to login as admin/PASSWORD (which you specified during the install).
There are some tutorials on how to use the ELN. However, the people in my lab told me that the UI is not very intuitive and that I need to teach them the basics. I think it might make sense to describe in a tutorial the typical setup in a typical small life science lab: what work spaces with what access rights for whom and to describe common scenarios (e.g. if some reagent lists need to be accessible by outsiders, etc.). I also have many ideas for improvements. It would be e.g nice to get more "ELN previews" for uploaded documents. At the moment, images are previewable, but e.g. PDF files not (I have not tried SVG files yet, but we use them quite a lot for image annotation). This certainly is not my last post about OpenBIS and until we take it into production use (likely beginning of next year) I still have to learn much.