Missing trailing new line when using cat in a bash script

Recently, I was writing a bash script that read a file into a string and then passed that string elsewhere that then calculated the hash of it. To my surprise, I was getting a different hash to doing the equivalent in PHP with file_get_contents().

Digging into it, I discovered that when you assign the output of cat to a variable, the trailing new line in the file is stripped off, which explained the difference.

You can see this by printing out the hex format using xdd.

Let’s start with cat on the command line:

$ cat foo.txt | xxd -p -u
5468697320697320666F6F0A

Note the 0A at the end; that’s the trailing new line character (\n).

Now, in a script, we may do this:

foo=$(cat foo.txt)

The foo variable doesn’t have the trailing new line and we show this by adding to our script:

hex="$(printf '%s' "$key_contents" | xxd -p -u)"
echo "$hex"

Running it:

$ ./test.sh
5468697320697320666F6F

As you can see the new line character is missing.

Now that we know what’s happening, it’s easy enough to add back in:

foo=$(cat foo.txt)$'\n'

In bash, single quotes preceded by a $ allows insertion of escape characters and so \n becomes the new line character.

Alternatively, we can use printf to create the variable;

printf -v foo "%s\n" "$(cat foo.txt)"

Either works, so you can take your pick!