Pragmatism in the real world

Missing trailing new line when using cat in a bash script

Recently, I was writing a bash script that read a file into a string and then passed that string elsewhere that then calculated the hash of it. To my surprise, I was getting a different hash to doing the equivalent in PHP with file_get_contents(). Digging into it, I discovered that when you assign the output of cat to a variable, the trailing new line in the file is stripped off, which explained the difference. You… continue reading.

Use xxd to convert to hex

If you want to see the hex values of a file, the easiest way to do this is to use xxd. Given foo.txt that contains "This is some text!": $ xxd foo.txt 00000000: 5468 6973 2069 7320 736f 6d65 2074 6578 This is some tex 00000010: 7421 0a t!. There are three columns in the output: number of first character in hex Hex representation of up to 16 bytes, separated every 2 bytes ASCII representation,… continue reading.

Changing port maps in Docker Compose

When using Docker Compose, I often map the ports to local ones in the compose.yaml like this: services: web: build: context: ./docker/web ports: – "8080:80" Adding to the port map If port 8080 is in use already on a developer's machine, then we can add a new mapping using compose.override.yaml services: web: build: context: ./docker/web ports: – "8081:80" The compose.override.yaml file is merged with the compose.yaml file and the effective configuration is now: services: web:… continue reading.

A short primer on SPF, DKIM and DMARC

I use FastMail for my email and as I control my own domain, I needed to set up SFP, DKIM and DMARC on it. These are DNS records that help the email servers put the emails that I send into my recipient's inbox and to mark any forged emails as spam. These are my tidied up notes so that I can find them again when I next need them. SPF SPF is a DNS record… continue reading.

Indenting output in a shell script with pr

When printing output in a shell script, it's quite useful to be able to indent information to group it. The easiest way to do this is with pr, like this: body=$(curl -s "http://localhost:8888/") echo "Response:" echo "$body" | pr -to 4 This generates: Response: { "links": { "games": "/games" } } Given that pr appeared in version 1 of AT&T Unix, I'm late to the party with this one, but it's a useful thing to… continue reading.

Using Monolog's TestHandler

When I write integration tests with PHPUnit, I find it helpful use Monolog's TestHandler to check that the logs I expect are generated. It's reasonably common that classes that write to a log take a PSR\Log\LoggerInterface in their constructor and then use that for logging. This usually logs to the error handler for pushing to Sentry or whatnot. This makes testing easy! For testing we can use the TestHandler which stores the logs it receives… continue reading.

Getting status code and body from curl in a bash script

When writing a shell script recently, I realised that it would be really handy to get the the status code from a curl command in addition to the body. Usually, I call curl like this: body=$(curl -s "http://localhost:8888/") echo "Curl exit code: $?" echo "Body: $body" This works well, but no status code is available as curl's exit code is related to its own error system, not the HTTP call. Searching Stack Overflow, I came… continue reading.

Release process checklist

I recently released a new version of rst2pdf. We don't do this frequently and it would be very easy to get it wrong. As a result, we have a RELEASE_PROCESS.rst document in our repo that provides a step-by-step list of what to do. I can't emphasise enough how useful such a document is and every project should have one and I've used them with client projects too. In general the following items need to be… continue reading.

Renaming PDF files based on their content

My accountant recently moved my business accounts system over to FreeAgent. One thing I like to do is keep a copy of every invoice PDF that I issue in a folder on my computer as a back up, just in case the online systems let me down. As I was learning new systems, I took advantage of this time to write a script to rename the invoice PDFs from Nineteen-Feet-Limited_Invoice_123.pdf to my preferred format of… continue reading.

Small scripts for repetitive tasks

One thing that I like to do is write a script for seemingly trivial things that I do more than once. I do this as it turns out that I end up needing them again. One example is a pair of script I use when testing rst2pdf. rst2pdf's tests work by creating a PDF and then comparing this to a reference PDF. If they are different, then the test has failed. When a test fails,… continue reading.