Ubuntu on Michael’s Domain

Installing Zotero on Linux/Ubuntu

Mon, 11 May 2026 00:00:00 +0000

The official way to install Zotero on Linux is mildly speaking a nightmare ( https://www.zotero.org/support/installation ). The fact that you need detailed instructions and command line skills to succeed in the installation speaks for itself:

Application is not responding

Fri, 27 Mar 2026 00:00:00 +0000

In Gnome, you get a warning if an application takes too long to complete a task. This can be really annoying if you know that the application is just trying to do its job with the available resources. Gdebi, Zotero and SnapGene are some of the apps that frequently cause this timeout warning on my (rather slow) laptop. To let Gnome know to stop sending this disturbing notification (or at least make them much less frequent), you can increase the timeout like this:

The Asustor AS-304T NAS cannot mount contemporary ext4 filesystems

Thu, 22 Jan 2026 00:00:00 +0000

The Asustore AS-303T is meanwhile quite old (from 2013), but I still wanted to keep it around to store backups from the last 15 years or so. I originally had placed four 2TB-drives into the device, but now I happened to have four old 3TB-drives lying around and I wanted to use them to maximize the storage capacity. So I temporarily copied the data from the NAS’s RAID to a few external drives before upgrading the NAS with the 3TB-drives. I did some of the copying via a direct point-to-point cable connection with my laptop , which was a bad idea, because when I later tried to copy back the data using the local USB3-port on the NAS, the NAS’ operating system did not recognize the ext4 file system, that my Ubuntu 24.04 had been using. It turns out that the Asustor NAS uses BusyBox v1.19.3, and even though it can read the ext4 filesystem, it does not know the modern ext4 features and thus failed to mount the drives (ADM version 3.5.9.RWM1, which received its latest update on August 29th, 2022). In order for the Asustor OS to recogize these drives, I needed to “downgrade” the ext4 filesystem. Below are the necessary commands to disable the newer ext4 features:

Instructions how to convert an Asustor AS7004T into an Ubuntu Server

Sat, 22 Nov 2025 00:00:00 +0000

This weekend, I upgraded our old Asustor AS7004T NAS to run an Ubuntu 24.04 server. The ADM OS had not been received any updates anymore for more than a year and it had been switched off and sitting on shelf since. This is a shame because there was absolutely nothing wrong with it. I had early on upgraded the original 2 GB memory of the device with an 8 GB module to a total of 10 GB. But you can run an Ubuntu server on 2 GB if you wanted to… What do you need for the conversion? Obviously, you need to plug in a USB-keyboard and a mouse. Apple USB keyboards do not work. The cheaper the keyboard the better. Sometimes there were problems with the mouse and then it helped to plug it into another USB port. And of course you need to connect the NAS to a monitor via an HDMI cable. I recommend to install Ubuntu Server 24.04.3. Ubuntu Desktop 24.03 also works (I tested it), but I don’t need it and the server has a much smaller footprint. Unlike with older Asustor NAS devices and distributions, fan control worked without problems. However, LED control and display info do not work: The display always shows “Starting system. Please wait…”.

Exporting from Avidemux to MP4

Sat, 20 Sep 2025 00:00:00 +0000

For some reason, Helsinki University’s Unitube crashes when trying to upload an MKV file. The same used to work last year… Only proprietary formats seem to be ok (such as MP4). This creates lots of additional work for me, since I use open source tools to edit my lecture recordings (mostly to cut out unnecessary parts).However, to create an MP4 file, I need to transcode, which is a time-consuming process that requires changing many of the default settings. Below the process using the Open source software Avidemux:

t_coffee still fails on a standard Ubuntu 24.04 LTS install

Thu, 18 Sep 2025 00:00:00 +0000

The bug in t_coffee, reported on https://github.com/cbcrg/tcoffee/issues/27#issuecomment-1355339411, is still an issue after many years. I hardly dare to recommend t_coffee to my students, even though I would like to, because it is otherwise an excellent and very powerful program. Most of them fail to install it on our university’s default Ubuntu distribution (“Cubbli”), which is atm Ubuntu 24.04. I tried it out myself just recently (Ubuntu 24.04 LTS with both the version provided by the default Ubuntu repository via the package manager and the stable and beta versions from https://tcoffee.org/Projects/tcoffee/index.html (stable COFFEE_installer_Version_13.46.0.919e8c6b_linux_x64.tar.gz and beta T-COFFEE_installer_Version_13.46.1.b8b01e06_linux_x64.tar.gz). All of them still complain with –ERROR: MAX_N_PID exceeded. It gets stuck somewhere and takes approximately one minute before it throws the error, even with a simple task that normally takes a few seconds. With a more complex alignment, it can take minutes or hours before the error is thrown. The workaround is to set the environment variable before every run, i.e., you replace t_coffee with a shell script that calls the renamed t_coffee after setting the environment parameter MAX_N_PID_4_TCOFFEE to something big (like /proc/sys/kernel/pid_max). The issue is explained in the Github link above. See also https://github.com/cbcrg/tcoffee/issues/47 . Another workaround is to recompile with a different MAX_N_PID, which is rather straightforward; see also here https://github.com/cbcrg/tcoffee :

Wireguard

Sat, 30 Aug 2025 00:00:00 +0000

My home router (a GL-iNet GL-MT6000) offers built-in WireGuard support. To add a client, you push a button and can either scan a QR code or download the configuration file. The QR code works out-of-the-box with my Wireguard for Android App (the official client for Android), and the configuration file works equally well for Ubuntu Linux 24.04 with Network Manager. To set up the VPN, you simply import the configuration file. The last command is to prevent the VPN from autostarting after a reboot, since when I work from home there is little use in routing my traffic to my own VPN server at home if I am anyway at home.

Backing up DVDs

Sat, 09 Aug 2025 00:00:00 +0000

DVD ripping is one thing (and Handbrake is the tool I have been mostly using for that). But sometimes you want to just copy the whole DVD to your hard drive. On Ubuntu Linux, my preferred software for this is dvdbackup .

Python Virtual Environments

Mon, 16 Jun 2025 00:00:00 +0000

There are many different ways to run Python in a virtual environment. I have used mostly conda (together with anaconda), but I frequently end up doing work on machines that do not have it installed. Hence, I have been recently starting to use the “inbuilt” virtual environment. If you are working on a Python project in a directory, these are the steps to start using it:

Barrier (a software KVM switch)

Tue, 21 Jan 2025 00:00:00 +0000

Barrier is an incredible software that lets you use the same keyboard and mouse with different computers without the need for hardware. I have two computers on my desk, and I used to have a hardware KVM (keyboard, video, and mouse) switch, which became obsolete when my computers started to output their video via HDMI. Hence, I have two keyboards on my desk, which is not very ergonomic. I knew about software solutions to this dilemma, but I never had the time to implement them, and I also did not want to pay a subscription fee for this luxury. I now started to use Barrier: https://github.com/debauchee/barrier . It’s been working excellently so far (except for the installation and documentation), but I still need to get it to work before logging in to dump my second keyboard.here are the steps that were not well documented:When installed on Ubuntu, Barrier does not automatically create certificates. So you need to do this yourself:Execute in both client and server: /home/username/.local/share/barrier/SSL:openssl req -x509 -nodes -days 365 -subj /CN=Barrier -newkey rsa:4096 -keyout Barrier.pem -out Barrier.pemYou also need to open port 24800 if you are using a firewall. After you get Barrier working, check the correct command (ps aux) from the command line. It is probably something like this:/usr/bin/barrierc -f --no-tray --debug INFO --name computer-name [87.167.231.88]:24800startUse the Startup applications in Ubuntu and paste the above command in. This means you cannot use Barrier to log in because it is loaded only after logging in. But that’s as much as I managed to figure out so far. There is probably a way to make this work pre-login via systemd, but I did not have the time to figure out how.

Overstretched IT support

Fri, 23 Feb 2024 00:00:00 +0000

The problem: Crash during boot

More than a year ago, a few weeks after receiving my new work computer, it failed to reboot after a system update and got stuck early in the boot process. I soon realised I could still start the computer using “safe mode”. Strangely, nothing seems to be wrong because when I manually exit safe mode at the end of the boot process, the computer works fine. Our IT department has tried to fix the problem many times without success. I even had to work without a computer for about 3 weeks while it was “under repair”. You probably know how much work you can get done without a computer: close to zero.My computer runs Cubbli 20.04.06LTS, an unofficial Ubuntu spin maintained by the University of Helsinki for internal use. Although you can do bioinformatics on a Windows or macOS computer, Linux is hands-down the first choice. Many bioinformatics developers don’t even bother to release their software for Windows. MacOS works mostly fine (since it is also UNIX-compliant OS), but I would have to pay twice the price for the same calculating power.The computer is a Lenovo P350 with an NVIDIA T1000 graphics card, which I use to address two screens. I mainly need the graphics card for 3D modelling, phylogenetics analysis and similar tasks. This graphics card may have caused the trouble. I initially did not want to buy this model, but since there was a shortage of graphic cards at the time, IT convinced me to swap out my original choice against the T1000.

How to manually encrypt a second hard drive in Ubuntu 20.04

Fri, 19 Nov 2021 00:00:00 +0000

You have added a second hard drive to your encrypted Linux system. The Debian installer makes it easy to encrypt during system installation, but how do you encrypt this new drive?

Installing perl modules

Tue, 22 Jun 2021 00:00:00 +0000

I have not done this for perhaps a decade or more. But apparently, things are still the same on Ubuntu 18.04:

Multiple Python versions on Ubuntu 20.04

Fri, 30 Apr 2021 00:00:00 +0000

Adding a new python version to Ubuntu 20.04 (for my system it is the 3rd version after 2.7 and 3.8): sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.6 3

Eduroam installation on Linux

Tue, 23 Feb 2021 00:00:00 +0000

This morning it took me an hour to get Eduroam working. Eduroam is the international WLAN roaming service for university students and staff. I brought in my (non-university managed) laptop to work and tried to get connected. I finally found the instructions on the IT Helpdesk site on place place 19 (from 21 results shown on the first page when searching for “Eduroam Linux”): https://helpdesk.it.helsinki.fi/en/instructions/logging-and-connections/networks/installation-eduroam-network-on-ubuntuAs usual, the instructions are from many years back and no longer work on a “recent” system such as Ubuntu 20.04 LTS. On top of this, I had some old, dysfunctional Eduroam settings lingering around from a previous failed attempt. For the instructions below, you will need the Eduroam configuration tool, which you can download from https://cat.eduroam.org . To make it work, I had to do the following things:

Apache forward proxy

Fri, 31 Jul 2020 00:00:00 +0000

Anyway: What is a forward proxy (and what is a reverse proxy for that matter)? If the proxy is a forward proxy, the server thinks the proxy is the client. If the proxy is a reverse proxy, the client thinks the proxy is the server.

Copying with rsync

Mon, 17 Feb 2020 00:00:00 +0000

Reasons why you should (not always, but often) prefer rsync over cp:

Resume Interrupted Transfers If you are copying a large file and your terminal session crashes or the power cuts out, cp will simply fail, and you have to start over. rsync can pick up right where it left off, which is a massive time-saver for large datasets.
Efficiency (Delta-Transfer Algorithm) rsync is designed to be smart. If you are copying a file that already exists at the destination (e.g., you are updating a backup), rsync only copies the parts of the file that have actually changed (the “delta”) rather than overwriting the entire file. cp, by contrast, always copies the whole file every single time.
Real-time Progress Monitoring rsync provides excellent visual feedback. By using the -P (or –progress) flag, you get a real-time progress bar showing the percentage complete, transfer speed, and estimated time remaining. cp is famously silent, leaving you guessing whether the process is still running or has hung, requiring you to check at the destination via the size of the arriving file, whether anything is progressing (ls -lh filename, or du -hs filename)
Seamless Remote Transfers rsync is natively built to work over SSH. You can use it to copy files directly to or from a remote server as easily as copying files on your local machine. cp cannot do this; to use cp for a remote server, you would first need to mount the remote filesystem locally, which is far more complex and prone to errors.

Here is a common use case, the transfer of a directory, including all of its content:

Free digital signing of documents under Linux - an impossibility?

Sun, 01 Dec 2019 00:00:00 +0000

The whole story started when I tried to sign a LibreOffice document. When you belive the internet, document signing is inbuilt into LibreOffice. I still have to find the person that managed to digitally sign a LibreOffice document. This experience shows, that despite Edward Snowden most people do not proactively care about security and privacy. Debian removed scdaemon from the gnupg2 package and as usual, one needs to be a command line ninja to fix this. The scdaemon gives smartcard support (which I do not have, but without the scdaemon the Kleopatra key manager refuses to run). I am using the default Ubuntu 18.04 installation and it was quite an odyssey to get a document signed. In fact, I still do not have a satisfactory way to do this. However one does it, something’s not right. Ubuntu 19.10 has fixed at least the invokation of the key manager from LibreOffice and I can invoke SeaHorse from the document signing dialog, but I still have no clue how to make my gpg keys visible to LibreOffice. Anybody figured this out? Until somebody shows me how to sign with LibreOffice, I use the very good, but proprietary software PDFStudio to import my GPG keys and sign my PDF files.Signing services (DocuSign, HelloSign) So what do you do if you need to sign e.g. a PDF and you have no means or do not want to subscribe to one of the document-signing certificate service like DocuSign ? Even with DocuSign’s budget plan a single digital signing costs $2. DocuSign has a 30-day free trial, but I do not know whether the certificats that you generate during the trial with continue to be valid after the end of the trial. HelloSign ( https://www.hellosign.com , now owned by DropBox ) has also a free tier (allowing to sign 3 documents/month) and when signing, it embeds an invisible signature (which was invalid for some strange reason when I tested it even though HelloSign is in Adobe’s approved trust list).Self-signing, cacert and StartSSLTechnically you can created your own signatures (self-signed certificates), but if such PDFs are viewed with Acrobat Reader, the signature will be flagged as invalid and the fact of self-signing is displayed. There used to be http://www.cacert.org/ , but to my knowledge, all browsers have removed the CAcert certificates and the same is likely true for Acrobat. StartSSL used to give out free certificates, but they do not exist anymore (they were seriously challenged with their own security).PDF Viewer supportInterestingly many PDF Viewers do anyway ignore the signing (e.g. the inbuilt PDF viewer from Firefox does not display anything). Other PDF viewers will display the signature, but NOT indicate, that it is not trusted (e.g. the Chrome Browser’s PDF viewer and Ubuntu’s default PDF viewer Evince). Since you have no idea what viewer your target will use to display your signed PDF, you are anyway in a bad situation (even if you subscribe to a document signing service). LetsencryptTo increase the trust in the signing, one can use a Letsencrypt certificate for signing. This signature certifies that the author of the document controls a specific website (in my case jeltsch.org). That is more than a self-signed certificate (and if the website is trusted, this is arguably also more than buying a subscription from DocuSign), but the re-purposed Letsencrypt certificate is not being trusted by Adobe since obviously the Letsencrypt endeavor was never meant for document signing (“Signer’s identity is unknown because it has not been included in your list oif trusted certificates and none of its parent certificates are trusted certificats”). However, the maximum lifetime of such a certificate is 3 months, after which it becomes invalid. It can still be used, but it will display that it is not valid because it has expired (or is not valid yet).How to misuse the Letsencrypt certificateFirst, you need a web server, that uses Letsencrypt certificates to verify the web site identity. This is out-of-scope for this blog post, but there are several good tutorials (e.g. from the Let’s Encrypt people themselves or from Digital Ocean ). Once you have your Let’s Encrypt certificates, this is the process to “misuse” them for signing documents:Since Letsencrypt requires certificate renewal every three months, there will be lots of fullchain.pem and privkey.pem files in the same directory and they are numbered. You obviously want to use the newest (the only valid) certificate and perhaps you want to renew the Let’s Encrypt certificate immediately before exporting it for document signing:certbot --apache --force-renewal -n -d jeltsch.org or if you want to renew all certificates: certbot --apache --force-renewal If you choose to renew all certificates, certbot will try to issue a single certificate for all domains that exist on your server (this possibility did not exist in the beginning of the Letsencrypt ecosystem, but was introduced later). If your server serves more than one domain, you need to manually specify the domain name, for which you want the certificate.For more details about how to use the certbot script, see https://certbot.eff.org/docs/using.html#certbot-commandsThis is the command, that converts the certs into a PKCS#12 file:openssl pkcs12 -export -out signing_certificat.p12 -in /etc/letsencrypt/archive/website-name/fullchain1.pem -inkey /etc/letsencrypt/archive/website-name/privkey1.pemThe PKCS#12 file stores the certificate and the private key in one encrypted file (with the file extension .p12). Therefore, the command will ask from you a keyphrase, which you absolutely need to remember to be able to use the certificate. Then you can transfer the p12 file to your desktop computer and use it to sign PDF files.Time stamping serversIf your PDF application asks for a time stamping server, you can use one of the free services, e.g. ca.signFiles.com/TSAServer.aspx or http://zeitstempel.dfn.de . However, also these time stamping services are not trusted by Adobe Acrobat.Here the commands to generate a self-signed certificate (it asks for a (temporary) passphrase, just make up something and remember it, you need it in the second step):openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 3650Conversion into a signing certificate (it first asks you for the temporary passphrase from above and then for the final passphrase, which you need to remember in order to use the certificate:openssl pkcs12 -export -out signing_certificat.p12 -in cert.pem -inkey key.pem

PXE-booting from Netgate Pfsense SG-3100

Fri, 15 Nov 2019 00:00:00 +0000

To install Linux without the need of a CD/DVD/USB-stick, I now use PXE-booting (“pixie”-booting) on our local home network. I could not find good instructions and had to try out things before it started working, but the process itself is fairly simple. Here are the steps:

Nautilus (Ubuntu's file manager) and bookmarks

Tue, 06 Aug 2019 00:00:00 +0000

In the very old days, the Ubuntu’s file manager’s bookmarks were stored directly as invisible file in the home folder ($HOME/.gtk-bookmarks). Since 14.04, they have been hiding two levels deep in $HOME/.config/gtk-3.0/bookmarks.And if you wonder where the Nautilus scripts are, they are nowadays at $HOME/.local/share/nautilus/scripts/.

Passwordless login via the GUI and ssh defaults

Mon, 25 Mar 2019 00:00:00 +0000

I use cloud services at CSC (e.g. pouta.csc.fi), and they do not allow SSH login with a traditional username/password combo. When I want to make a bookmark in my file manager (Nemo oder Nautilus/Files), pointing to this location. I need to specify an RSA key file that is used for the login. On the command line, it looks as follows:

Mounting group directories via the fstab

Mon, 11 Feb 2019 00:00:00 +0000

At the University of Helsinki, one can apply for group directories, which is simply disk space on a NAS. These are easily mounted when you use a university-managed computer via the university menu, but what about if you use your own machine? At least for Ubuntu Linux, the way to mount these directories has changed multiple times during the years and it again broke recently. The current entry in the /etc/fstab file in my (Vanilla, non-university-managed) Ubuntu 16.04 is as follows:# HY group directories//group2.ad.helsinki.fi/h204 /home/local_username/GROUP-drive cifs noauto,user,user=hy_username,nobrl,uid=1000,gid=1000,file_mode=0666,dir_mode=0777 0 0Then you can simply execute “mount /home/local_username/GROUP-drive” and you will be asked for your HY password. The number 2 and the string “h204” in the server address (group2.ad.helsinki.fi/h204) are specific for the cost center which is in this case H2042 (taking the first digit from the cost center string and first four characters from the cost center string, respectively).And here are some instruction on how to do it manually on a Macintosh:https://helpdesk.it.helsinki.fi/en/instructions/saving-and-sharing/group-storage-space/remote-access-home-and-group-directory-mac

OpenVPN server on pfsense and client on Ubuntu 16.04

Fri, 02 Nov 2018 00:00:00 +0000

I have been setting up an OpenVPN server on my Netgate SG-3100 router . I hope this makes syncronizing backups to a physically separate location easier. There are many walkthroughs to set up an OpenVPN server on a pfsense router and that works nicely. However, I am using Ubuntu 16.04 at work and setting up the client requires a bit more than doing the same on MacOS or Windows. On Ubuntu, it is mandatory to update DNS information manually after establishing the VPN tunnel if you have opted for the setting to route all internet traffic originating from the client through the VPN server. If you do no update the DNA resolver information on the Ubuntu client, you can access the the VPN-internal network (in my case 10.0.0.0/24), but you cannot use hostnames. E.g. ping www.google.com will fail, but ping 172.217.21.164 will succeed. Similarly, your browser will not find any URLs. And browsing with IP-addresses (“http://172.217.21.164”) is not very practical.The default configuration on Ubuntu does not allow for this update of the DNS resolver to happen automatically for security reasons. There is a script included in the Ubuntu package of openvpn that updates this information (/etc/openvpn/update-resolv-conf). But in order for this to work one needs to

Remote desktop sessions to your Helsinki University work computer

Mon, 15 Jan 2018 00:00:00 +0000

If you have a work laptop, you can take it home to do work. But what if you have a desktop computer and need to access it from home? The technology to make this possible exists for more than 20 years, but if you think that University IT has made this easy for you, you would be wrong. In fact, I don’t know anybody who knows how to do this (let alone how to make the process easy). Even with the setup explained below, some things do not work well (e.g. I never could figure out how to get the file sharing to work with a Mac-to-Mac connection and thus I still use Fugu with a separate tunnelled sftp connection to transfer files). You have several options:

The magic sequence to wake up Apple's SuperDrive on Linux

Sat, 09 Dec 2017 00:00:00 +0000

sudo apt-get install sg3-utils (only needed the first time)ls /dev (to check whether sr0 or sr1, usually it’s sr0 unless you had already other USB drives connected)sg_raw /dev/sr0 EA 00 00 00 00 00 01

Deleting files with "illegal characters"

Fri, 08 Sep 2017 00:00:00 +0000

In the cross-platform environment of our university, at least three different OS connect to the smb shared group directories. Apparently Macs are able to save files with characters that are not allowed on other platforms and which cannot be easily deleted. Recently we had several files which ended with a ‘.’ (dot) or a ’ ’ (space), and we were not able to remove them via the GUI. In Windows 7, I managed via the command line with:delete "file." ordelete "file "I have, however, not found a way to so from my Ubuntu machine and it is annoying to boot into Windows merely to delete some files. Unfortunately, this became necessary because the Cloudberry backup system (which is running on Ubuntu) chokes on such files. I am looking into a new backup system for our lab’s computers since CrashPlan decided to discontinue support for non-business customers…

Windows changes my computers clock

Mon, 28 Aug 2017 00:00:00 +0000

If I am not mistaken, Windows is the only OS, that uses local time for the system clock (instead of UTC). This means that in multiple boot configurations, every restart of the system into Windows will screw up the system time. In order to reset it (in Ubuntu 16.04), usesudo timedatectl set-timezone Etc/UTC

Handbrake and protected DVDs

Thu, 02 Mar 2017 00:00:00 +0000

Ripping DVDs is a thing of the past, but occasionally I still need to do it. However, it happens so rarely that evertime I have either a new computer or a new Linux distribution and I need to install the necessary software. This time I am on Ubuntu 16.04. I usually use Handbrake and it is very comfortable, but handbrake sometimes fails to do the job. This time, I had to resort to MakeMKV . However MakeMKV only rips the video to an MKV container but it does not compress it. If you need the space reduction you should still push the file through a compressor. Here’s the whole chain of commands:I only have an Apple USB-Superdrive. Apple doesn’t want me to use it with any other computer but Macs. Therefore, in order to make it work under Linux, you needs to sens it a “Magic cookie” (see here: ):sg_raw /dev/sr0 EA 00 00 00 00 00 01If you have Handbrake installed from the default Ubuntu repository, you need to remove it (it’s crippled and you want to be able to use it also with encrypted DVDs):(sudo apt remove handbrake && sudo apt autoremove)Install this version:sudo add-apt-repository ppa:stebbins/handbrake-releasessudo apt updatesudo apt install handbrake-gtk handbrake-cliInstall software to read encrypted DVDs:sudo apt-get install libdvd-pkgHowever, you have to execute some manual commands after the install is ready, but the installer instructs you during installation.Select as source the TS_Audio folder on the DVD. If the scan never finishes or you do not see any tracks after the scan finishes, you might need something like MakeMKV. I didn’t find any PPA or deb file, so I compiled it from source, which succeeded without any problems following the isntructions from here: http://www.makemkv.com/forum2/viewtopic.php?f=3&t=224 . Here the short version:wget http://www.makemkv.com/download/makemkv-bin-1.10.4.tar.gzwget http://www.makemkv.com/download/makemkv-oss-1.10.4.tar.gztar -xvzf makemkv-bin-1.10.4.tar.gztar -xvzf makemkv-oss-1.10.4.tar.gz cd makemkv-oss-1.10.4/./configuremakesudo make installcd ../makemkv-bin-1.10.4/makesudo make installThe executable is /usr/bin/makemkv

1000 times too slow

Thu, 01 Dec 2016 00:00:00 +0000

Helsinki University has outsourced parts of its IT infrastructure to Microsoft. Even if outsourcing is cheaper in the short term than generating equivalent services locally, the net effect is likely negative due to the lost jobs, know how and independence. It doesn’t need a conspiracy to explain this self-destructive behavior, just bad decision criteria, which do not include long term and externalized costs.In its push to upgrade to newer Windows versions, Microsoft ended extended support for Windows XP on April 8, 2014 and the university obeyed by denying network access to XP machines. The argument was that XP was becoming a security risk. At the same time, Windows XP accounted still for about 20% of all Windows installations on this planet ( https://www.statista.com/statistics/218089/global-market-share-of-windows-7/ ). Even in October 2015, well above 100 Million computers did still run Windows XP ( https://en.wikipedia.org/wiki/Usage_share_of_operating_systems and https://www.statista.com/statistics/218089/global-market-share-of-windows-7/ ) without any catastrophic consequences.The ban of Windows XP was a problem, since four of our devices, that we need for research, are still running Windows XP. And there is no way to upgrade the OS without upgrading the equipment (which would cost thousands or tens of thousands of Euros and therefore is mostly impossible in the present tight financial situation of the university). And of course, the software that is needed to operate the devices is not compatible with Windows 7. As a consequence, we can neither remotely operate this machinery nor do automated backups. Even taking the data for analysis to another computer requires the Adidas network .I had contacted IT support well before the extended support of Windows XP ended and asked them to find a solution. Apparently there were different possible solutions and IT support did start to implement them, but because of personnel fluctuations, the project had to be “restarted” several times and different support professionals had different opinions on how to solve this problem. With the big “fire and hire” action at the university, the whole project disintegrated again. This autumn I again discussed our needs with the IT staff, but did not receive any concrete help so far.This Tuesday, I finally wanted to know how difficult it really is to connect a Windows XP machine to the network in a way that would not compromise security, but enable file distribution, backup and remote control. I went to the Institute’s garbage place where broken electronic equipment is gathered and took three old 10/100 NICs and a few ethernet cables. One card and one cable were still functioning and I dropped the NIC into a Ubuntu 16.04 computer and connected it via ethernet cable to one of the XP machines.After manually assigning an IP to the NIC and installing samba onto the Ubuntu machine, I was able to mount the samba share as a drive on the Windows XP machine. Then I just made the samba share available via a web page. All this took about one hour. Not being an IT professional, I spend maybe an additional three hours researching how to do it (samba setup, apache setup, configuration of a secondary NIC, which is not automatic on Ubuntu). This setup fulfills all of our requirements, didn’t cost anything and was implemented within one day.A write-up of the technical details will follow once I get around documenting what I did. Agility is perhaps what is mostly missing when I look at many of our university’s IT projects. Notably I think of switching to Drupal as content management system for the university’s web pages. Sadly, our faculty is still using Dreamweaver to create its web presence and as a consequence many web pages are never updated.

Automated reinstall of software from package list

Sun, 20 Nov 2016 00:00:00 +0000

If I need to reinstall a Ubuntu/Debian-based Linux OS (or mirror a software selection to another machine), this is how it can be done. On the source machine:sudpkg --get-selections > ~/Package.listcp -R /etc/apt/sources.list* ~/apt-key exportall > ~/repository.keysThen just copy the files to the target machine:suapt-key add ~/repository.keyscp -R ~/sources.list* /etc/apt/apt-get updateapt-get install dselectdselect updatedpkg --set-selections < ~/Package.listapt-get dselect-upgrade -yIf some packages are not available, this will fail. This concerns in my case manually installed packages like teamviewer or snapgene / snapgene_viewer . I usually remove those packages manually from the list (there are luckily not many of them).However, the upgrade is not fully automatic, since you need to e.g. agree to various licenses (e.g. for Microsoft’s True Type fonts) and acknowledge manually other stuff (e.g. libdvd-pkg legal issues), which kind of defeats the purpose of making this automatic and painless…

Let's encrypt duplicated my log files

Wed, 19 Oct 2016 00:00:00 +0000

I have not been keeping log files for my web server until the beginning of 2016, when I needed to trace access to certain files (I started to use awstats , mostly because I was familiar with it since I had used it years ago when my site was still running on a Red Hat server). In March 2016 I luckily started to use Let’s Encrypt . I had used another “free” service before, which got recently into big trouble as they apparently had not control over their own security. When I looked at my Apache server’s log files (on Ubuntu 14.04), I noticed that apache did double logging (to both the individual vhost’s log file and a common log file). I realized that Let’s Encrypt specifies into every vhost’s configuration file an Import directive which sources /etc/letsencrypt/options-ssl-apache.conf. And this file in turn specifies common access.log and error.log files for all vhosts in the /var/log/apache2/ directory. I uncommented the five lines associated with this logging and the duplicate logging stopped (originally, I had thought, that this letsencrypt directive was only used for the initial Let’s Encrypt setup for the cert generation).

How to start openvpn or ssh server under Ubuntu 16.04 and 18.04

Sat, 03 Sep 2016 00:00:00 +0000

OpenVPNsudo systemctl start openvpn@clientThe “client” is derived from the configuration file name (/etc/openvpn/client.conf). If your configuration file is server.conf, the command needs to besudo systemctl start openvpn@serverWhen you want the service to start up automatically during system boot, you issue:sudo systemctl enable openvpn@serverTemplated versus non-templated servicesOpenVPN is a so-called “templated” service (it needs a configuration file when being invoked). In contrast to this, the ssh server is a non-templated service. Hence the command is simpler:sudo systemctl start sshd

The Staden package on Ubuntu for bioinformatics dinosaurs

Wed, 27 Jul 2016 00:00:00 +0000

Mostly we use the SnapGene software when we check the sequences of our DNA constructs. However, sometimes SnapGene’s alignment view is not flexible enough and then I fall back to using the ancient Staden Package . I just had upgraded from Ubuntu 14.04 to 16.04 and hence did not have Staden installed. I was pleasantly surprised when the installation of Staden took only about 15 seconds because finally - thanks to the Debian Med team - Staden is available from the universe repository (actually already since October 2014).sudo apt install stadenStaden is clearly not as intuitive as it could be, but it is very powerful and lends itself to automated processing of data. If you have the opportunity to learn it, I would encourage you to do so. The Finnish CSC recorded the Staden course from 2004, in which I participated and you can get the recordings from Funet TV . I just had briefly considered switching from Ubuntu to SuSE because of the ongoing wireless connection debacle that Canonical can’t seem to fix, but considering how non-trivial a manual install of Staden is, this is a big plus for Ubuntu. There are obviously dedicated Linux distributions for bioinformatics purposes, but they all tend to lag behind the latest and greatest developments of the major distros.

BSD and Linux

Fri, 10 Jun 2016 00:00:00 +0000

I am used to the fact that a Linux installer honours a pre-existing install of Windows and offers to setup the computer with a dual-boot option during installation. Vice-versa no Windows installer honours any other pre-exisiting OS. Therefore I was surprised that when I tried to install Ubuntu 16.04 on my PFSense box (FreeBSD), the Ubuntu installer did not even see that a BSD install exists on the drive. I chose the “erase all” option, but when I rebooted after the installer has finished, the system went straight into PFSense without giving me any option to select Ubuntu. I guess the boot loader had not been touched by the Ubuntu installer. I booted from a live Ubuntu USB stick, reformatted the drive with fdisk and wrote zeros to the boot loader:dd if=/dev/zero of=/dev/sda bs=512 count=1Then I repeated the install and everything was fine. However, my PFSense installation was lost…

BackupPC and Macs

Wed, 13 Apr 2016 00:00:00 +0000

I myself have been using the backup software BackupPC for almost a decade, and I started to backup our lab computers to a central backup server about two years ago. BackupPC supports deduplication and therefore much data can be stored on a couple of 2 TB drives.BackupPC supports many protocols (smb, ftp, tar/rsync via ssh), but we mostly use rsync via ssh. The data is not encrypted before the backup. Strangely not even the upcoming version 4 will support pre-egression encryption. However, we store the backup on an encrypted volume. That way it is at least protected if the backup server is stolen. And during transit, the data is protected by ssh. However, the system is not TNO (“trust no-one”), since I (as the backupc administrator) can access the files. When using Linux, one feasible method would be to encrypt the user’s home directory using eCryptFS (which is an inbuilt option when creating users on Ubuntu ) and then backup the /home/.ecryptfs directory instead of the users home directory. However, recovery would be much more of a problem. You would be able to browse the diectory structure and files of the backup, but the filenames would be meaningless since they are also encrypted in the process.There is one peculiarity in backing up Mac OSX machines: BackupPC normally connects as root via ssh into the client computer and executes the rsync backup command. In order to make this possible on university-managed Mac OSX computers, we had to create a dedicated user (“backuppc”) on the client machines and allow for this user the execution of rsync with root privileges, which is done by adding this line in the /etc/sudoers file:backuppc ALL=NOPASSWD: /usr/bin/rsync. Then we have to change the ssh/rsync command for the Mac client on the backuppc server changing “root” into “backuppc”.

Managing OpenVPN with Network Manger

Thu, 05 Nov 2015 00:00:00 +0000

I just switched back from MacOSX to Ubuntu for work. Mostly for financial reasons. We need more computers at work and a really good PC laptop is just half as expensive as a MacbookPro or iMac. Today I wanted to connect from home to the University’s VPN network and I had a look at the instructions provided by the university.As usually, documentation was virtually absent and what was available was wrong. And exclusively in Finnish ( http://www.helsinki.fi/helpdesk/ohjeet/tietoliikenne_ja_etakaytto/yhteydet_yliopiston_ulkopuolelta/vpn_ubuntu-asennus.html) . From the image (Hardy Heron) it is clear that this page has not been updated for about at least 6 years (Hardy Heron was realased in the beginning of 2008).So what do you do if you downloaded and extracted the hy-vpn-config.tar.gz file from https://ohjelmistojakelu.helsinki.fi?First you need to install a plugin for the Network Manager: sudo apt-get install network-manager-openvpn network-manager-openvon-gnome. Then you go to the Network Manager via the icon in the menu bar on the top right of your Desktop. Below all the available wireless networks, there is an entry “VPN Connections”. Follow “VPN Connections” -> “Configure VPN” -> “Add” -> “Import a saved VPN configuration” -> “Create”. Then select the “openvpn.conf” from the downloaded and extracted files. After that, fill in the rest of the dialog box: User name, Password. For the CA Certificate, select the “HY-vpn-CA.pem” file from the downloaded files (you should have put it first somewhere safe, e.g. to “/etc/openvpn”).One problem that people are complaining about is the fact that the import does not honor the “redirect-gateway def1” directive and as a consequence you won’t be able to connect anywhere (I guess this is due to the Network Manager using dnsmasq and dnsmasq is apparently not smart enough to realize that it should send the queries somewhere else now). That’s why people are complaining that Network Manager doesn’t work to route all traffic via the VPN network. The box that you need to uncheck for this to work is well hidden: It’s in the connection editor dialog under the IPv4 Settings tab -> Routes (at the bottom right) -> “Use this connection only for resources on its network”. Why on earth do they have to call it in a way that nobody understands its meaning? Why not to call it “Do not route all traffic through this VPN connection”? I also had to check the box that said “Ignore automatically obtained routes”, although I don’t know why…As usual, setting up the OpenVPN sucks and the important tunneling back of VPN traffic needed to be added manually on the OpenVPN server:iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADEI did it by making an additional file called openvpn2 in the /etc/network/if-up.d/ directory with the following content:#!/bin/shiptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADEOf course you can still start and stop the VPN via the command line. However, since systemd, the password entry is not straightforward. When you execute sudo systemctl start openvpn.service you need to execute (e.g. in another terminal) sudo systemd-tty-ask-password-agent and enter your password there. That’s clearly a kludge until they get a decent password agent…

Upgrading mysql on Ubuntu Lucid Lynx (10.04) hangs

Wed, 28 Nov 2012 00:00:00 +0000

Already the third time this happens to me: An update is available for the mysql server and I just “apt-get upgrade”. The upgrade process gets totally stuck at the following task:Preparing to replace mysql-server-5.1 5.1.66-0ubuntu0.10.04.1 (using …/mysql-server-5.1_5.1.66-0ubuntu0.10.04.2_i386.deb) …mysql stop/waitingThe process that got again stuck in some eternal loop (or whatever) is an egrep replacement of some textfile:egrep -qi -r ^[^#]*ndb.connectstring|^[:space:]*\[[:space:]*ndb_mgmd /etc/mysql/I just killed that process and the upgrade resumes as nothing would have happened.

A permanent static route in Ubuntu Karmic Koala (9.10), Precise Pangolin (12.04) & Trusty Tahr (14.04)

Sat, 26 Dec 2009 00:00:00 +0000

Karmic KoalaTo add a permanent static route to a Karmic Koala system with one NIC, you need to edit the/etc/network/interfaces file. The following section needs to be replaced:# The primary network interfaceauto eth0#iface eth0 inet dhcpModify as follows:# The primary network interfaceauto eth0iface eth0 inet dhcpup route add -net 10.8.0.0 netmask 255.255.255.0 gw 192.168.0.3down route del -net 10.8.0.0 netmask 255.255.255.0 gw 192.168.0.3**Precise Pangolin (12.04)**With the Precise Pangolin, I did it by adding a script called add_route to /etc/network/if-up.d:#!/bin/sh#if [ "$IFACE" = "eth0" ]; then route add -net 10.8.0.0/24 gw 192.168.0.18#fiI never added a corresponding script (del_route) to /etc/if-down.d, but that seems to be OK.**Trusty Tahr (14.04)**With Trusty Tahr, I was adding the route command as a line to the /etc/rc.local script:…route add -net 192.168.1.0/24 gw 192.168.0.3exit 0The route command needs to be before the “exit 0” line!**MacOS X 10.8 (Mountain Lion)**And here is how it is done on MacOS X 10.8: http://nellen.it/blog/2012/01/permanent-static-routes-for-mac-os-x . They details of how to to it in MacOS X have changed over time; here is the way how it was done on MacOS X 10.4.7 .

Changing the MAC (Media Access Control) address of NICs (Network Interface Cards)

Sun, 25 Jan 2009 00:00:00 +0000

In some networks, computers have to be registered to get connected. Upon registration, the computer will receive a name and an IP address. Every time it connects to the network, the DHCP server assigns it the same name and IP address. The DHCP server identifies computers based on the MAC address, which is unique to every NIC. However, if you know the MAC address of a registered computer, you can use it to connect to the network, faking the MAC address:

cryptsetup on lvm on Ubuntu Feisty

Thu, 20 Sep 2007 00:00:00 +0000

I wanted to encrypt my Documents. My home folder is on a logical volume. So I made some space by removing another logical volume and creating a new one, which gets encrypted by cryptsetup. Here are the commands:

Installing awstats on Ubuntu Feisty

Wed, 27 Jun 2007 00:00:00 +0000

I started to use awstats for creating the site statistics on our server. After installing the ubuntu package, there are still a few things that I had to do:

Changing the Runlevel (inittab, telinit)

Sat, 26 May 2007 00:00:00 +0000

Usually you change the default runlevel by editing /etc/inittab. However, sometimes you boot into runlevel 3 (e.g. by selecting safe mode from the grub menu) and then want to change to runlevel 5. This you accomplish on the fly with the command: telinit 5In RedHat 9, just change in the /etc/inittab file the runlevel from 5 to 3 and you will have only the command line left by default upon booting. No X11 graphical login etc. In newer Ubuntu releases, this apparently doesn’t work anymore, although telinit + number should still change the runlevel, but at least in the Precise Pangolin (12.04), telinit 2-5 appears not to do anything. Telinit 1 tries to shutdown the machine, but the shutdown got stuck halfway when I tested it. To achive something similar to changing to runlevel 3, you will have to shutdown the GUI (meaning to kill the gdm or lightdm process). The runlevel concept seems to be dead.

Ubuntu 5.04 or Debian 3.1 on the original PB G3 (aka Kanga aka PB 3500)

Thu, 29 Mar 2007 00:00:00 +0000

I am trying to get Ubuntu 5.04 or Debian 3.1 running on the original PB G3 (aka Kanga aka 3500). The only problem: the screen is blank after the inital reboot after the installation. Nothing at all (Ubuntu) or only an inverted penguin (Debian). The boot process continues only if one copies over the ramdisk image from the /boot folder onto the HFS partition and specifies it as boot argument. Might it be that the root=/dev/hda8 (in my case the root partition is hda8) is somehow not recognized? Here the story: I rebooted using the MacOS 9.1 CD, I reformatted the drive into two partitions: one 1GB and the rest (about 4GB) unallocated. Then I installed Mac 9.1 on the 1GB partition, downloaded BootX and installed it according to the instructions. Then I burned the Ubuntu PPC from the iso onto a writable CD (using a “regular” i386 Linux distribution and K3b, BTW: RW-CDs are apparently not recognized by Kanga’s CD drive). Then I copied both the installation kernel and initrd from the installation CD to their respective places into the Macintosh System folder (Ubuntu_PowerPC_hoary/install/powerpc/vmlinux to Macintosh HD:System Folder:Linux Kernels and Ubuntu_PowerPC_hoary/install/powerpc/initrd.gz to Macintosh HD:System Folder:ramdisk.image.gz). Then I rebooted and selected Linux. Installation works like a charm, I couldn’t believe it. The network card is detected correctly, so is apparently all other hardware. I choose the guided partitioner (select largest unused space) which created the ext3 filesystem on /dev/hda10. After the installer had finished and was about to reboot, I needed to copy the deafult kernel from the /boot to the HFS partition in order to have BootX start Linux. The fasted way I concluded was for me to switch of the machine, take out the hard disk and connect it via a USB enclosure to my i386 SuSE Linux 9.3 machine. Both the HFS partition and the ext3 partition were automatically mounted and I copied /boot/vmlinux-2.6.10-5-powerpc to my SuSE machine. Then I put the hard drive back to the Powerbook G3 and rebooted into Linux and copied the kernel via http to Macintosh HD:System Folder:Linux Kernels. Then I executed the BootX application, but here my luck ended. What happens is that the first couple of lines of the boot messages are displayed and then the screen goes blank and never appears again. The last lines that are displayed read “arch:exit”. However, the machine continues booting if one specifies the ramdisk image (initrd) from /boot of the installed system, but since there are still some installation tasks to be done, the machine never reaches a state where I could ssh into it and fix stuff. Now I have not the faintest idea where to start as there are no error messages and nothing. Since the installer manages to address the monitor, it should be possible to get it done, but how? I have been trying to pass almost every possible combination of kernel arguments to get the video working, but to no avail (the correct kernel argument should be video=chipsfb:vmode:10,cmode:16; maybe fbdev instead of chipsfb?). My next attempt will be to install it with an external monitor connected. BTW: I had YDL 3 running on the same machine quite a while ago. But it seems there are no viable alternatives ATM to Ubuntu when it comes to PPCs. The same procedure using Ubuntu 4.1 leads to similar (though not identical) results. The network card has to manually selected during install (de4x5 module). After the first reboot the penguin appears in the upper left corner in inverted colors and that’s where the system hangs. I somehow refuse to accept that this 250 MHz machine is only worth to be thrown away. When I bought it in 1997, it was a “high-end” machine (it could even play back divx without problems). I had YDL 3 installed on it, but would like to install something more up-to-date now.