Files on Michael’s Domain

Counting files (list and word count, ls & wc)

Wed, 29 Apr 2020 00:00:00 +0000

How do you count the number of files in a directory? That becomes non-trivial if you have thousands of files in a directory. Your file manager chokes on counting them, especially if they are not local. On the Linux command line, this task is fast and easy:ls | wcls lists your files one per row and wc returns three numbers: lines, words and bytes. In the file counting example, lines and words are the same and equal the number of files in the directory. To count all files recursively in the directory tree “backcheck”:find backcheck/ -type f | wc -l

Copying with rsync

Mon, 17 Feb 2020 00:00:00 +0000

Reasons why you should (not always, but often) prefer rsync over cp:

Resume Interrupted Transfers If you are copying a large file and your terminal session crashes or the power cuts out, cp will simply fail, and you have to start over. rsync can pick up right where it left off, which is a massive time-saver for large datasets.
Efficiency (Delta-Transfer Algorithm) rsync is designed to be smart. If you are copying a file that already exists at the destination (e.g., you are updating a backup), rsync only copies the parts of the file that have actually changed (the “delta”) rather than overwriting the entire file. cp, by contrast, always copies the whole file every single time.
Real-time Progress Monitoring rsync provides excellent visual feedback. By using the -P (or –progress) flag, you get a real-time progress bar showing the percentage complete, transfer speed, and estimated time remaining. cp is famously silent, leaving you guessing whether the process is still running or has hung, requiring you to check at the destination via the size of the arriving file, whether anything is progressing (ls -lh filename, or du -hs filename)
Seamless Remote Transfers rsync is natively built to work over SSH. You can use it to copy files directly to or from a remote server as easily as copying files on your local machine. cp cannot do this; to use cp for a remote server, you would first need to mount the remote filesystem locally, which is far more complex and prone to errors.

Here is a common use case, the transfer of a directory, including all of its content:

Passwordless login via the GUI and ssh defaults

Mon, 25 Mar 2019 00:00:00 +0000

I use cloud services at CSC (e.g. pouta.csc.fi), and they do not allow SSH login with a traditional username/password combo. When I want to make a bookmark in my file manager (Nemo oder Nautilus/Files), pointing to this location. I need to specify an RSA key file that is used for the login. On the command line, it looks as follows:

File permissions on html server (recursive chmod) differentiating between files and directories

Sun, 12 Apr 2009 00:00:00 +0000

I recursively screwed up the file permissions in my html servers root directory. To fix it I needed to deploy some “advanced” chmod settings. Obviously I want to treat directories and files differently. I didn’t come up with a better solution than the following:chmod -R u=rw-x,g=rw-x,o=r-wx *chmod -R a+X *