Do you need training or consultancy? Get in touch!

2016 in pictures

Another year has passed which gives me an excuse to to reflect on what's happened. As usual, I look at the photos that I've taken and frame my thoughts around them.

January

At the very end of January I visited Phoenix, Arizona to see Evan, Priscilla & other friends. I also attended FOSDEM again and spent a tourist day in Brussels.

The Europeans insisted on seeing the sun!Atomium


February

I was fortunate enough to speak at PHPUK again in February.

The conference is about to startStart of the second day

March

The highlight of March was visiting the NYMR to see Flying Scotsman.

Flying Scotsman from the linesideUp close

April

In April, my eldest drove a Lamborghini and now intends to buy one! It's important to be able to dream when you're 14. I also visited the NRM.

Eldest drove a Lamborghini!Mallard

May

May is the month of birthdays in our household. I also attended the StatusCode event in Nottingham

Happy Birthday!Andrew


June

Two conferences in June. I spoke at PHP South Coast and took my favourite photo of Andrew Smith ever! I also attended Lead Developer.

Best pose everMichael Lopp

July

July was the inaugural UK edition of REST Fest in Edinburgh. We also went on holiday where I attempted sunrise photos and ate lots of ice cream.

Mike talks about Conway's four lawsSunrise

August

Worcester Comic Con happened in August where we got to meet "our" Doctor! Unfortunately this was also the month when my dad was taken ill.

Georgina with her DoctorLearning table top gaming

September

September was about family and friends with an extended family gathering and a trip to Leeds to see friends. Dad was transferred to a closer hospital too and started to get better. September was also the start of conference madness that lastest to November, with a trip to Dublin for DrupalCon.

Dad & meA group of amazing people

October

October was the month of conferences when I spoke at PHPNW in Manchester, attended OSCON Europe and spoke at the Software Architecture conference, both in London! The other welcome news this month was that Dad came out of hospital.

Coffee breakAttendies to my talk!

November

November is Fireworks Night in the UK which we celebrated with friends. This year, we contributed wood for the bonfire as we've been tiding our garden. I also spoke at Velocity in Amsterdam as did my friend Kevin.

Bonfire nightKevin discussing how to survive the Grand National

December

As usual, the final month of the year found me in a pub with some of my oldest Internet friend from our MMORPG gaming group that I've been part of since 1999. I visited Northern Ireland for the first time. Family from America also visited; I haven't seen my cousin Leigh in 28 years and so had not met her husband or children, so this was very much a highlight.

BL Cronxmas meetupFamilies



All in all a good year, though I will try and space out my conferences a little better in 2017!

SSH keys in macOS Sierra

Now that I've upgraded to macOS 10.12 Sierra, I noticed that SSH required me to enter my passphrase to keys every time I used them. This was a surprise as it's not how 10.11 El Capitan worked.

This is how to fix it.

Firstly, add your SSH key's passphrase to the keychain using ssh-add -K ~/.ssh/id_rsa (or any other key file). You can now use your SSH key without re-typing the password all the time which is very handy for use with GitHub/GitLab/Bitbucket/etc.

You can add as many keys as you like and ssh-add -l will show you which keys are registered.

When you reboot, you'll notice that ssh-add -l is empty which is different from how it works on macOS 10.11 and earlier which automatically re-added the keys it knew about. In Sierra, Apple has changed it so that you now need to explicitly add the known identities to the ssh agent. This is done using ssh-add -A which you need to run every time you reboot.

To save having to do this, you can either add ssh-add -A to your ~/.bash_profile file or update your SSH config by editing ~/.ssh/config and adding:

SSH will now work as expected and you'll never need to reenter your passphrase once it has been added to the system keychain.

Handling JSON data errors in Slim 3

When you send JSON data into a Slim Framework application with a content-type of application/json, then Slim will decode it for you if you use getParsedBody():

Using curl to test:

If there's an error however, you get this:

If you care about this, you can use json_last_error_msg() and return an error:

(note – in real code, you should check that the Accept header was a JSON one…)

Don't forget the JSON_PRETTY_PRINT as a human is going to be reading this error, so make it easier for them.

Use jsonlint for more information

If you really want to provide great diagnostics, then use jsonlint:

Update your handler like this:

(lint() will return NULL or a ParsingException, so we don't need to test for anything else.)

The result looks like this:

This is much more informative!

Use curl to create a CouchDB admin user

This too me longer to find than it should have done, so I'm writing it here for future me.

When you install CouchDB, it is in a mode where anyone can do anything with the database including creating and deleting databases. This is called "Admin Party" mode which is a pretty cool name, but not what I want.

Creating admin users

To create a user in 1.6 (I've not used 2.0 yet, but assuming it's the same) you simply click on the "Fix This" link in Futon which is available at http://localhost:5984/_utils/ by default.

As CouchDB's entire API is essentially a RESTFul API, to do this via the command line, you simply PUT a new user to into the _configs/admins collection like this:

This creates an admin user called rob with a password of 123456. Note that the password within the body of the PUT request must be a quoted string. This caught me out for a while!

From this point on, we can then use basic authentication to do admin-y things, such as create a bookshelf_api database:

Other users

You can also set up per-database users which is handy for limiting what your application can do when connected to CouchDB. This is done creating users in the /_users/ collection and then assigning them to a class in the _security collection of the database. There are two default classes: "members" and "admins" where members can modify data, but not design documents and admins can modify all documents including user roles on that database.

On respect in the workplace

I recently came across Don’t be that dude: Handy tips for the male academic, an article covering 20 actions that the author identified as things that men do everyday to perpetuate inequality. A number seem specific to the academic world, but the majority are relevant everywhere.

Now, 20 items is a lot to remember, so I want to call out a few that I see regularly when I'm in offices and at conferences. They are all about respect for the individual and while they aren't necessarily gender specific, I personally have seen many more men doing these things to women than vice-versa and cringe every time I notice.

Don't talk about someone's appearance

Don’t comment on a woman’s appearance in a professional context. It doesn’t matter what your intentions are; it’s irrelevant. Similarly, don’t tell someone they don’t look like a scientist/professor/academic, that they look too young, or they should smile.

Complimenting someone on they way that they look rather than what they have achieved or do is demeaning and disrespectful. Nobody tells me that they like the way that I've ironed my shirt, but I've been in meetings with potential clients where a man has introduced his colleague to me by saying that she must have dressed up today because of the meeting. He didn't comment on the dress sense of any of his male colleagues when introducing them and left me feeling uncomfortable. I have no idea how the woman felt; different and unwelcome I expect.

Be aware of what you say about people, especially women. If you find that you do seem to mention your colleague's clothing or (lack of) makeup, then train yourself to stop talking about it. If you feel like you must make small talk, then find a different topic: something in the news, the travel situation, anything else!

Don't interrupt

Don’t talk over your female colleagues. There is a lot of social conditioning that goes into how men and women communicate differently. You may not realize that you’re doing it, but if you find yourself interrupting women, or speaking over them, stop.

The general case here is don't interrupt. However, it's much more common to see a man who listens intently when his male colleagues are talking, but interrupts and talks over a woman in the meeting. I think this happens because the man doesn't even hear the woman. This is especially frustrating for me as when if you're paying for my time at a meeting, then everyone's input is important.

Pay attention to your own behaviour in meetings. In particular, check that someone else isn't talking the you want to make your point. If you notice that you are interrupting someone else talking, then start consciously waiting for a pause before talking. Your point won't be less important because you waited a few minutes before speaking.

Make tea for and minute your own meetings

Volunteer when someone asks for a note-taker, coffee-run gopher, or lunch order-taker at your next meeting. Don’t let this task fall to women, even if they tend to volunteer (we’re socially conditioned to do so). Make sure that women aren’t being asked to do this more than men.

I see this all the time in client meetings where there is a mix of gender in the room and I now get a bit embarrassed as I used to be that man.

At a previous job, I didn't initially even realise that I was letting a woman team member do much more than her fair share of the tea making rounds. Once I did realise, I told myself that as the senior lead, it was a better use of resources to let the junior members do tea making. That was rubbish. At some point I was educated on this phenomenon and worked on changing the culture by explicitly asking different team members to make the tea.

I have one client where the rule is that the host of the meeting is the note taker and lunch handler which works really well too as it's clear that if you want this meeting to take place, then you get to do some of the leg work. I don't know why they introduced this system: I suspect that it was to promote fewer meetings, but the end result is fairer too.

To sum up

You should read the full list of 20 points as all are worthy of consideration and behaviour change. I've called out the three that I've had to personally work on and improve at and are the ones that I most want to see change on.

In this day and age, be a professional; show some respect.

Hide the ST3 sidebar automatically

As a mostly keyboard user, I take advantage of the keyboard shortcuts in Sublime Text. However, the sidebar is quite a lot of effort to manage, especially as I mostly leave it closed.

Firstly, you need cmd+k,cmd+b to open it. Then you type ctrl+k,ctrl+0 to focus it as opening it doesn't automatically set focus. Then you can navigate to the file you want and open it via pressing return. Finally, you type cmd+k,cmd+b to close it again.

That's a lot of keyboard combinations! Note also that you swap from cmd to ctrl too.

Revealing the current file

I find that I often want to open the side bar with the currently open file highlighted. There's a command for this: reveal_in_side_bar which you assign to a keyboard shortcut like this:

    { "keys": ["super+shift+1"], "command": "reveal_in_side_bar"},

Now, pressing cmd+shift+1 will open the sidebar with the current file highlighted. You still need to press ctrl+k,ctrl+0 to focus though.

Automating with plugins

There's a plugin called FocusFileOnSidebar that solves this problem, so if you have this workflow, then install it now.

Again, you need to set a keyboard binding:

    { "keys": ["super+shift+1"], "command": "focus_file_on_sidebar"},

It doesn't automatically close the sidebar for you though, so I created HideSidebarWhenNotFocussed that does just that.

Done

With the combination of FocusFileOnSidebar and HideSidebarWhenNotFocussed, the sidebar works the way I want it to!

It's closed most of the time, I press shift+cmd+1 to open it with focus set on the currently highlight file. If I open another file by pressing return or go back to what I was doing by pressing Esc, then the sidebar automatically closes as it should.

Using CharlesProxy's root SSL with home-brew curl

Once I installed Homebrew's curl for HTTP/2 usage, I discovered that I couldn't automatically proxy SSL through Charles Proxy any more.

$ export HTTPS_PROXY=https://localhost:8888
$ curl https://api.joind.in/v2.1/
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

This is a nuisance.

As I've noted previously, you need to install Charles' root certificate to use it with SSL. On OS X, you do Help -> SSL Proxying -> Install Charles Root Certificate which installs it into the system keychain.

However, this doesn't work with the Homebrew curl or with the curl functions inside PHP. To fix this, we need to add the Charles root certificate to OpenSSL's default_cert_file.

I've talked about this file before. The quickest way to find it is to run:

$ php -r "print_r(openssl_get_cert_locations());"

on the command line. The output should be similar to:

Array
(
    [default_cert_file] => /usr/local/etc/openssl/cert.pem
    [default_cert_file_env] => SSL_CERT_FILE
    [default_cert_dir] => /usr/local/etc/openssl/certs
    [default_cert_dir_env] => SSL_CERT_DIR
    [default_private_dir] => /usr/local/etc/openssl/private
    [default_default_cert_area] => /usr/local/etc/openssl
    [ini_cafile] =>
    [ini_capath] =>
)

As you can see, the file I need is /usr/local/etc/openssl/cert.pem.

Grab the root certificate from the Charles app. On Mac, that's Help -> SSL Proxying -> Save Charles Root Certificate menu item.

You can then append the root certificate to the default cert file:

$ cat charles_root.crt >> /usr/local/etc/openssl/cert.pem

Now, everything works:

$ curl https://api.joind.in/v2.1/
{"events":"https:\/\/api.joind.in\/v2.1\/events","hot-events":"https:\/\/api.joind.in\/v2.1\/events?filter=hot","upcoming-events":"https:\/\/api.joind.in\/v2.1\/events?filter=upcoming","past-events":"https:\/\/api.joind.in\/v2.1\/events?filter=past","open-cfps":"https:\/\/api.joind.in\/v2.1\/events?filter=cfp","docs":"http:\/\/joindin.github.io\/joindin-api\/"}

(It also works in PHP as that's linked against the same curl if you followed my post on enabling HTTP/2 in PHP.)

Using HTTP/2 with PHP 7 on Mac

if you want to use HTTP/2 with PHP on OS X, you need a version of curl compiled with HTTP/2 support. You can then link your PHP's curl extension to this version.

The easiest way to do this is to use Homebrew:

At the time of writing, this will install PHP 7.0.10 with Curl 7.50.1:

Using Curl on the command line

If you want to use your shiny new curl from the command line, then the easiest way to do this is:

You can now do:

and you should get:

(at the time of writing!)

Using Guzzle to test HTTP/2 is working

To prove it works in PHP, use Guzzle!:

testhttp2.php:

Run this PHP code at the command line:

As we've turned on debugging, the output looks like this:

The key things to notice:

We tell the server that we want HTTP/2 (h2), but can accept HTTP/1.1. If the server doesn't support HTTP/2 it will send back HTTP/1.1

This is a good sign!

The response's status line is:

The version number is 2 and we got a 200, so all is OK!

Everything is working as intended.

RESTful APIs and media-types

Evert Pot recently posted REST is in the eye of the beholder. Go ahead and read it; it's good!

It discusses the fact that most APIs that are self-described as RESTful are not hypermedia driven and hence are not actually RESTful as originally defined by Roy Fielding. Evert goes on to state that therefore the term "REST API" has changed to mean an HTTP API that isn't hypermedia-driven.

I think that what the world currently calls RESTful APIs fall short in more ways than just not serving Hypermedia links.

Media-types provide value

The majority of HTTP APIs out there do not reap the benefits of using media types correctly either. The majority seem to use application/json and call it done. The client gets absolutely no information about how to read the data, other than they'll need a JSON decoder… This is like having an HTML file and using a `.txt` file extension. It's correct in the sense that the the file is plain text, but absolutely useless for interpreting the file to render a web page.

Media types allow an API to inform the client how to interpret the data in the payload. This is arguably much harder than adding hypermedia to an API. The correct media types enforces the structure of the payload and also what the payload data means.

This is why most APIs just use application/json. If you're lucky, there's some documentation somewhere that explains things. Half the time, we seem to expect client developers to read the JSON and interpret it based on experience.

The best example of where this works well is the media type "text/html". Given a payload in this media type, a web browser can render the information in the way that the web developer (api server) automatically. This is because every server that sends a document with a text/html payload sends the same tags for the same meaning.

We can do this in the API world too, but it requires thinking about and is harder, so it doesn't happen…

There are three uses of media-types that I see in the world:

  • Plain: application/json (or application/xml
  • Vendor specific: e.g. application/vnd.github.v3+json
  • Standard: e.g. HAL, Collection+JSON or JSON-API, Siren, etc.

There is no practical difference between an API that uses a Plain media type and one that uses a Vendor specific one. As a client developer, you have no clue how to interact with the API or deal with its payload. To integrate with this API you need Google and hope you find enough documentation.

An API that uses what I've called standard media types give a client developer a leg-up. There's an official specification on how the data in the payload is organised. The standard is documented! This makes a difference.

You always need human-readable documentation to integrate with an API. A standard media type, implemented properly, makes it much easier. For starters you don't need to write as much as the standard has taken care of a good proportion of the boring bits. Something like HAL's curies provides an unambiguous way for the developer to find the right documentation for the endpoint they are dealing with.

JSON-API's pagination and filtering rules mean that I can write code once that will work with every API that uses JSON-API that I have to integrate with. This is powerful!

We can even go further with structured data as defined at schema.org to provide standardised field names for our data, but that's a topic for another day.

Evert is correct

I've used Evert's post to point out that when we talk about Hypermedia in a RESTful API, we mean more than simply putting in a few links in the payload.

Going back to Evert's article, he is correct; the term "REST API" is pretty much meaningless and at best simply means "An API that works over HTTP with some awareness of what an HTTP method is". The current term for an API that meets the constraints of REST is "Hypermedia API".

I think that providing an API with hypermedia & a well-documented media-type is beneficial for every API. APIs last longer than you believe and it's a competitive advantage if a developer can integrate with yours easier and faster than with your competitor's.