Electronic Lab Notebook on Michael’s Domain

The eLabFTW electronic lab journal: ready for prime time?

Fri, 03 Jun 2022 00:00:00 +0000

Some love it and some hate it: the electronic lab journal (ELN). Already a decade ago, the prediction was that in 10 years the paper lab notebook would be a thing of the past. In reality, the majority of academic labs are still using paper in 2022. The transition has not been helped by literally hundreds of commercial offerings that are all mutually incompatible. We started to experiment with ELNs in 2013, testing a few commercial and open source solutions. The worst of the pack did not even allow us to test drive them before buying (e.g. LabVantage seems to think you won’t buy once you have tried it, and they are spot-on!). We settled on eLabFTW , an open-source project with the main development happening at the Institute Curie in Paris, France. It’s a web application built with PHP and MySQL. There is also a peer-reviewed publication describing it: https://doi.org/10.21105/joss.00146 and a general Nature Protocols review article, that discusses ELN implementation: https://doi.org/10.1038/s41596-021-00645-8 .Deployment for teaching at University of HelsinkiWith the recent update to version 4.3 , the final hurdle for large scale deployment has been taken. Everybody with a helsinki.fi account can now log into the system with university account credentials. This makes the onboarding of many new users much easier. I am planning to use the system for our new course “Recombinant DNA technology and genetic engineering”. At the moment, the server is only reachable from inside the university network. If you want to use it from home, you need to use a VPN. Over the years we have been accruing about 25 users, and it will be interesting to see how the system performs when the user numbers will double or quadruple. The system runs in a docker container on an Ubuntu 20.04 LTS virtual server, which we, unfortunately, cannot upgrade or extend without additional financial support. So if you are interested to use an ELN, secure your spot today by logging into https://elab.ltdk.helsinki.fi , using the “Login through your institution” option on the bottom. You will have the option to choose a team. The team administrator needs to approve you before you can use the system. If you cannot find your lab among the teams, it means that you are the first user from your lab using this system. In that case, you should choose the team “Helsinki University” and contact me via email ( michael@jeltsch.org ), because I will need to register a new team for your lab, and you will be the administrator of this new team.Access for visitorsIf you do not have a University of Helsinki account, you can still use the system if you are inside the Helsinki University Eduroam network, but you will need to use the “Register now” link, and your account needs to be approved by a system administrator.Running it on your local machineThe system requirements of eLabFTW are fairly low, and your laptop is likely to have no problems running a local copy of the docker container. While installing eLabFTW on a recent Windows or macOS computer is possible, it is somehow counter-productive as it does not provide network accessiblity and team functionality. Even if you don’t need any of that, I would suggest that you repurpose an old desktop computer for this task and run it on a regular Linux server installation. I am happy to answer all possible questions that you might have (from a system administrator’s and end-user’s perspective).

Barriers to Electronic Lab Notebook adoption

Thu, 08 Jun 2017 00:00:00 +0000

Our lab has been interested in electronic lanb notebooks (ELNs) for a while. A recent publication in the Journal of Cheminformatics has focused on the barriers to ELN adoption. Especially in the academic world, more than 90% of scientists are still using the traditional paper lab notebook. Biggest barrier to adoption was the cost in terms of money and learning of new habits. Close runner-ups on places 3 and 4 were the implementation effort and security concerns. While the authors are clearly biased (they are developing the open source ELN ELN SciNote ), the barriers to adoption are the same that I experience in my lab and at the University of Helsinki.I was stunned when about 2 months ago I received an e-mail with the news that the University of Helsinki had negotiated a framework agreement with the LabVantage LIMS/ELN vendor Software Point. While I think a university-wide ELN solution is necessary, this deal seems anything but optimal for most research-oriented laboratories at the University of Helsinki. Several labs at the university are using ELNs already for a while and none of them have been aware of the efforts to provide a university-wide agreement for a LIMS/ELN solution. And none of those that already use a LIMS/ELN have been choosing LabVantage, but other solutions ( Labguru , SciNote , eLabFTW ). It would have been helpful to have somebody in the evaluation team, who has actual experience with LIMS/ELNs use (or somebody who has participated in LIMS/ELN development), especially since such expertise exists at the University of Helsinki.Only a few people have been involved in the negotiations while the majority of the stakeholders were not even asked for their input. When I asked explicitly about this secluded operation, the answer was: Very few units (mainly the chronology unit and a person from dept of chemistry) were involved with the purchase at the time even though the information was spread around the university and anyone was welcome to participate and state their needs. However, even though I try to actively follow the IT infrastructure developments, I did not see any announcements anywhere and also searching retrospectively, I could not find anything that would have alerted me and allowed me to participate. The labs contributing most to the international success of Helsinki University are the main stakeholders, but they were ignored. Maybe a little bit more active search for the stakeholdres would have been appropriate.The base package that was negotiated with LabVantage seems to be geared towards routine and clinical lab work, but not towards innovative research. Among the two modules that are mostly necessary for life science research work are the ELN and the storage management. Unfortunately, many life-science specific routines (searching for DNA and protein sequences with blast-like search algorithms) are totally absent from the offer, but are very much needed for life science. The lack of a free API (like eLAbFTW features or SciNote soon will feature) prevents users from incorporating their own, existing code or their own database systems. Every small change would require to involve the LabVantage developers and that will be a very slow and costly process. In reality, this means that shortcomings of the system will never be addressed. The base package can be taken into use by any lab at the university after a single payment of 200€ for each user (no floating licenses, only dedicated users). For our lab that would amount to 2000€, but a low price tag is not a selling point in itself.Within the framework agreement, any lab can pay for the additional modules on its own, but the heft price tag of 17500€ make it unlikely that anybody will do so. I am unwilling to lobby for a joint purchase, because there is no way of knowing, whether these modules would be suitable for everyday lab work. Unlike most other vendors, LabVantage doesn’t offer a free trial of the system. It makes me suspicious, if a vendor doesn’t offer a trial. If the product is good, a trial will convince potential customers to buy. If the product is good, what could be the downside of a trial for the vendor? We have ourselves tested several ELN solutions over the last two years and some systems received very bad feedback by those people who use them every day (e.g. cumbersome data import, unintuitive interface, slow server responses). As a matter of fact, for oligonucleotide management and protein/enzyme storage, we still use the absolutely outdated open source LIMS that I co-authored about 15 years ago as a PhD student ( http://phplabdb.sourceforge.net , now at https://github.com/mbekaert/phplabdb) . In my lab, we use eLabFTW . We have a local installation on the Helsinki University intranet (which can be accessed via VPN from everywhere) and anybody from Helsinki University can get an account there. If you are interested in trying it out, please contact me!The key question remains unanswered: Why were the majority of stakeholders not pulled into the decision-making process?

OpenBIS for Dummies

Tue, 06 Sep 2016 00:00:00 +0000

If you have considered moving from paper to electronic lab notebooks as we are at the moment, you might have come across the OpenBIS solution. You want to install OpenBIS, but have not clue how to go about it? I didn’t have much of a clue either and therefore tried repeatedly until I succeeded. Luckily, I got some help from the ETHZ OpenBIS and our local computing support team, but obviously they cannot compensate for the lack of insight into Jetty and postgresql…If you just quickly want to try it, it might be easier to download the VirtualBox image, where everything is preinstalled and preconfigured ( https://wiki-bsse.ethz.ch/display/bis/openBIS+ELN-LIMS+Virtual+Machine ). However, I didn’t have any fast hardware to run the VirtualBox image and concluded that it would be fast enough on bare metal using an old computer.The notes below are written from memory and from the final configuration files that worked. However, I will still setup the server from scratch executing only what I have been writing down below in order to make sure I have not missed anything important. However, in the meantime I want to get this information out. I would have been happy if I had found some “OpenBIS installation for Dummies” instructions. In order to have a very long support, I chose for the installation Ubuntu 16.04 Server (the VirtualBox image uses Ubuntu 14.04 Desktop) and the latest version of OpenBIS 16.05.1 (you need to apply for an account to be able to download the software). The electronic lab notebook (ELN) plugin comes bundled with that release. Doing that, you have a problem with Java 7 support (it is not going to maintained for long and I will try to run OpenBIS ELN with Java 8 soon, but this walk-through uses a Java 7 PPA for Ubuntu 16.04).1. Do a fresh install of Ubuntu 16.04.1 Server. Create during the install the admin user “openbis”. When it asks for software selection, choose OpenSSH and Postgresql in addition to standard system utilities. Upon first login (as openbis user), update and upgrade all the software to the latest version and install emacs (or whatever text editor you prefer) and unzip:sudo apt install unzip emacs24-nox2. Install Java 7. Thi sis just one way how to do it:sudo add-apt-repository ppa:openjdk-r/ppa sudo apt-get updatesudo apt-get install openjdk-7-jre-headless3. To setup postgresql correctly, change the configuration file /etc/postgres/9.5/main/pg_hba.conf. All lines ending in peer, the peer should be changed into trust! After that, reload postrgresql:sudo systemctl reload postgreql4. If you are installing to a virtual machine like virtualbox, it would make sense to install the guest utilities as this will make you life easier (virtualbox-guest-utils). Make a shared (permanent, automout folder) and add openbis to the vboxsf group:sudo usermod -a -G vboxsf openbis4. Place the compressed openBIS installer file into the home directory of the openbis user.5. Untar/gzip the installer:tar -xvzf openBIS-installation-standard-technologies-S233.0-r36799.tar.gz6. Change into the uncompressed directory:cd openBIS-installation-standard-technologies-S233.0-r367997. Change the following things in the console.properties file:INSTALL_PATH=/home/openbis/DSS_ROOT_DIR=/home/openbis/dataELN-LIMS = truePATHINFO_DB_ENABLED = trueINSTALLATION_TYPE = serverFor this try I have not changed the password of the keystore, but you should do it for security reasons if you use the server for production!8. Run the installer (not as root, but as openbis user):./run-console.sh9. When the installer asks to enter the password for the openBIS ‘admin’ user, type in the password you want to use when logging in as admin into the web interface.10. When installation is done, go to ~/openbis/servers/openBIS-server/jetty/etc and edit the file service.properties. You actually have not to edit anything for the installation to work, but we needed to configure the system for login authentication via Ldap.authentication-service = file-authentication-service => authentication-service = file-ldap-authentication-serviceIf you use ldap alone (i.e. authentication-service = ldap-authentication-service), you need to have some special users in the ldap directory. We are authenticating via our university’s ldap serer only regular users of the system and hence, e.g. the admin user needs to be authenticated locally. This admin user is setup during the install, but you need to have both file and ldap authentication active for this to work. This is our ldap server address as specified in the service.properties file. It contains the authentication base, which you need to ask from your sysadmins:ldap.server.url = ldap://ldap2015.it.helsinki.fi/OU=people,DC=helsinki,DC=fiThis is the ldap user and his password on the ldap server (don’t ask me why the ldap people use such complicated word monster for such a simple concept):ldap.security.principal.distinguished.name = OU=openbis,OU=login,DC=helsinki,DC=fi``ldap.security.principal.password = PASSWORDThe following paramters are specific to our ldap server. We use OpenLDAP and since OpenBIS seems not to support TLS, we use ssl:ldap.security.protocol = ssl``ldap.security.authentication-method = simple``ldap.queryTemplate = (&(%s))OpenBIS requires https. You can either leave the self-signed certificate (provided by the ETHZ) or install an own certificate. If you keep the self-signed certificate, all users will get a warning when they try to log into the web service. There seems to be an additional problem as the uploading of files seems not to work with the self-signed certificate since the https request via port 8444 doesn’t result in an intercept that is presented in the browser window to users to override.Hence we had no choice but to get a real certificate, which is luckily easier today than still one year ago due to the letsencrypt folks. Nevertheless, getting the certificate stuff right took us some time. We are operating the OpenBIS server inside the University network and it is not visible from the outside. Users who want to access it from outside need to use a VPN.Therefore we had to install a temporary “fake” server with the same name on a publicly reachable IP and generate a letsencrypt certificate (letsencrypt is not yet automated for jetty). I used the automated method for apache2 on my own server at Digital Ocean and then retrieved the two important files (fullchain1.pem and privkey1.pem) from the /etc/letsencrypt/archive/eln.jeltsch.org directory and moved them over to our OpenBIS server. To convert them into the correct format for the java keystore, I used the following commands:openssl pkcs12 -export -out keystore.pkcs12 -in fullchain1.pem -inkey privkey1.pemkeytool -importkeystore -srckeystore keystore.pkcs12 -srcstoretype PKCS12 -destkeystore keystore.jksThe first command asks for an export password. It doesn’t matter what you use (I used 12345678). The second command asks for a destination keystore password. Enter here “changeit” if you have not changed the default keystore password (“changeit”) during the setup. Then it also asks you for the source keystore password (which is the 12345678 that I just used above).Then you still have to add the contents of this keystore to the already exisiting keystore. There are probably many ways to do this, but I used a graphical tool called Keystore Explorer . You simply open both keystores (you need the “changeit” password for this), you delete the existing entry for ETHZ and add the only (letsencrypt) entry from the newly generated keystore. Replace the files “keystore” and “openBIS.keystore” in the directory ~/openbis/servers/openBIS-server/jetty/etc/ with the modified keystore. Both files are identical, I don’t know atm the relevance of this duplication. You also have to replace ~/openbis/servers/datastore_server/etc/openBIS.keystore with the new version of the keystore. For some reason, uploading did not work and in order to enable uploading, the service.properties of the datastore server needs to specify the host-address of the datastore server:https://eln needs => https://eln.jeltsch.org. Since we used a different host for the certificate generation, there was a host name mismatch and we had to override manually the hostname settings:Change it in the files /etc/hostname and /etc/hosts, then reboot. However, the installer took only the first part of the full name for the service.properties files of the datastore server. Our server’s name is eln.jeltsch.org and it resulted in https://eln, which did not resolve in our network, since we got the letsencrypt certificate signed on another server. After we changed the service.properties files and after rebooting we were finally ready to test the server:11. Starting the server: Log into the server as openbis user.cd ~/openbis/bin/./allup.sh12. On another computer, navigate in your web browser to “ https://eln.jeltsch.org:8443/openbis/webapp/eln-lims" . If you have not installed a publicly trusted certificate, the browser will now warn you that the connection is not secure. Just click “Advanced” and confirm the security exception. You need to login as admin/PASSWORD (which you specified during the install).There are some tutorials on how to use the ELN. However, the people in my lab told me that the UI is not very intuitive and that I need to teach them the basics. I think it might make sense to describe in a tutorial the typical setup in a typical small life science lab: what work spaces with what access rights for whom and to describe common scenarios (e.g. if some reagent lists need to be accessible by outsiders, etc.). I also have many ideas for improvements. It would be e.g nice to get more “ELN previews” for uploaded documents. At the moment, images are previewable, but e.g. PDF files not (I have not tried SVG files yet, but we use them quite a lot for image annotation). This certainly is not my last post about OpenBIS and until we take it into production use (likely beginning of next year) I still have to learn much.