Pragmatism in the real world

Serverless PHP on AWS Lamda

Like, Simon Wardley, I think that serverless computing is an interesting space because the billing is granular (pay only when your code executes) and you don’t need to worry about maintaining and provisioning servers or containers. So much so, that I maintain the Open Source PHP Runtime for Apache OpenWhisk which is available commercially as IBM Cloud Functions

There are other serverless providers, and AWS Lambda is the market leader, but until recently PHP support could most charitably described as cumbersome. That all changed at the end of 2018 with Lambda’s new runtime API and support for layers.

Let’s look at the practicalities of serverless PHP on Lambda with Serverless Framework.

TL;DR

The source code for a simple Hello World is in my lambda-php Github repository. Just follow the Notes section and you should be good to go.

PHP runtime

The runtime API allows for any runtime to be used with Lambda. In some ways it looks a bit like the way OpenWhisk runtimes work in that there’s an HTTP API between the serverless platform and the runtime. One very obvious difference is that with Lambda, the runtime calls back to the platform to get its invocation data whereas OpenWhisk calls an endpoint that the runtime must implement. More details are in Michael Moussa’s article on the AWS blog, which inspired my work.

To get back on track, we need a PHP runtime for Lambda! This will comprise the PHP binary, the code to invoke our PHP serverless function and a bootstrap file as required by the platform. We put these three things into a layer. Layers are re-usable across accounts, so I’m quite surprised that AWS doesn’t provide a PHP one for us. Stackery do, but they aren’t using PHP 7.3, so we’ll build our own.

We’ll put all the files in the layer/php directory in our project.

Building the PHP binary

We need a PHP binary that will run inside Lambda’s containers. The easiest way to do this is to compile it on the same platform as Lambda, so we use EC2. Michael’s article explains how to do it and so I turned those commands into a compile_php.sh script, so that I could copy it up to the EC2 instance, run it & then copy the binary back to my computer:

$ export AWS_IP=ec2-user@{ipaddress}
$ export SSH_KEY_FILE=~/.ssh/aws-key.rsa

$ scp -i $SSH_KEY_FILE compile_php.sh $AWS_IP:doc/compile_php.sh
$ ssh -i $SSH_KEY_FILE -t $AWS_IP "chmod a+x compile_php.sh && ./compile_php.sh 7.3.0"
$ scp -i $SSH_KEY_FILE $AWS_IP:php-7-bin/bin/php layer/php/php

This makes it nicely repeatable and hopefully it will be fairly simple to update to newer versions of PHP.

Bootstrapping

As we are using the runtime API, we need a bootstrap file. This filename is required by Lambda and is responsible for invoking the function by making relevant API calls in a while loop.

Essentially, we need to sit in a loop and call the /next endpoint to find out what to invoke, invoke it and then send the response to the /response endpoint.

AWS provides an example in BASH using curl:

while true
do
  HEADERS="$(mktemp)"
  # Get an event
  EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
  REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

  # Execute the handler function from the script
  RESPONSE=$($(echo "$_HANDLER" | cut -d. -f2) "$EVENT_DATA")

  # Send the response
  curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response"  -d "$RESPONSE"
done

We want to do the same thing in PHP and while I could write it myself, Parikshit Agnihotry has already done so in PHP-Lambda-Runtime/runtime.php, so we’ll use that and copy it into layer/php/runtime.php. I made a couple of changes to my version, so that it does the json_encoding and also to add better error handling.

The layer/php/bootstrap file is very simple as all it needs to do is run the PHP binary with this file:

#!/bin/sh
cd $LAMBDA_TASK_ROOT
/opt/php /opt/runtime.php

That’s it. We have three files in layer/php:

  • php – the PHP executable
  • runtime.php – The runtime API worker
  • bootstrap – The Lambda required boostrap stub

These will become our PHP Layer in our Lambda application.

Set up Serverless Framework

Serverless Framework allows repeatable configuration and deployment of a serverless application. I’m a fan of this concept and want to use tools like this more. We’ll use it for our PHP Hello World.

As there’s no handy Serverless Framework tempate for PHP applications, we’ll just create a serverless.yml file in our project directory.

Firstly, the basics:

service: php-hello-world
provider:
  name: aws
  runtime: provided
  region: eu-west-2
  memorySize: 128

We name our application php-hello-world and we’re using AWS as our provider. As I’m in the UK, I set the region to London & we don’t need much memory, so 128MB is enough.

The runtime is usually the language that you want your function to be executed in. To use the runtime API which will execute our bootstrap stub, you set this to provided.

You’ll also want a .gitignore file containing:

.serverless

as we don’t want that directory in git.

Let’s add our layer to serverless.yml next, by adding:

layers:
  php:
    path: layer/php

This will create the AWS layer and give it a name of PhpLambdaLayer which we can then reference in our function.

Write our Hello World function

We can now write our PHP serverless function. This goes in handler.php:

<?php
function hello($eventData) : array
{
    return ["msg" => "hello from PHP " . PHP_VERSION];
}

The function takes the information about the event and returns an associative array.

To tell Serverless Framework to deploy it, we add it to serverless.yml:

functions:
  hello:
    handler: handler.hello
    layers:
      - {Ref: PhpLambdaLayer}

Serverless Framework supports multiple functions per application. Each one has a name, hello, in this case and a handler which is the file name without the extension followed by a full stop and then the function name within that file. So a handler of handler.hello means that we will run the hello() function in handler.php.

Finally, we also tell the function about our PHP layer, so that it can execute the PHP code.

Deploy to Lambda

To deploy our function with its layer we run:

$ sls deploy

This will whirr and click for a bit and produce output like this:

Sls deploy

Invoke our function

Finally, we can invoke our function using:

$ sls invoke -f hello -l 

Sls invoke

And we’re done!

To sum up

With the new layers and runtime API, it’s now possible to easily run PHP serverless functions on Lambda. This is great news and worth playing if you’re a PHP developer stuck tied to AWS.

I should also note that you should look into Bref which makes PHP on Lambda much easier!

21 thoughts on “Serverless PHP on AWS Lamda

  1. Hi Rob,

    Very interesting, but are you 300% sure all this stuff is worth such "effort" compared to e.g. just have a VPS with PHP in place?

    Thanks a lot for sharing and happy new year!

    1. As someone who has been building Serverless applications for 3 years now and has been waiting for PHP support all that time (have had to use Node till now) I can say yes it is. Very much worth it. A VPS is FAR more work to get up and running in the long run.

  2. Nice article!
    Interesting point of view on the use of Lambda and serverless in general as it stands today, in this paper:
    https://arxiv.org/abs/1812.03651
    "[…] Recent studies show that a single Lambda function
    can achieve on average 538Mbps network bandwidth; numbers
    from Google and Azure were in the same ballpark [26]. This is
    an order of magnitude slower than a single modern SSD. Worse,
    AWS appears to attempt to pack Lambda functions from the
    same user together on a single VM, so the limited bandwidth
    is shared by multiple functions. The result is that as compute
    power scales up, per-function bandwidth shrinks proportionately. With 20 Lambda functions, average network bandwidth was 28.7Mbps—2.5 orders of magnitude slower than a single
    SSD[…]"

  3. Can I use php 5.3.6 or codeigniter with the above tutorial for custom runtime for aws lambda

  4. {
    "errorType": "Runtime.ExitError",
    "errorMessage": "RequestId: a37014bc-cfcf-4f6e-b09e-03b8ed2163aa Error: Runtime failed to start: fork/exec /opt/bootstrap: exec format error"
    }

  5. This is what i got after testing the function on Lambda Console:

    Response:
    {
    "errorType": "Runtime.ExitError",
    "errorMessage": "RequestId: 6aafdf1c-f3da-4dfb-a2b3-34dcf1c692b3 Error: &{0xc00005e2a0 map[invoke_id:6aafdf1c-f3da-4dfb-a2b3-34dcf1c692b3 sandbox_id:0] 2019-09-25 16:21:25.468208404 +0000 UTC m=+0.035853851 panic Runtime failed to start: fork/exec /opt/bootstrap: permission denied }"
    }

    1. I got the same thing

      {
      "errorType": "Runtime.ExitError",
      "errorMessage": "RequestId: 3e62501d-8948-4bf1-9e07-c3bacd9e51ac Error: &{0xc00005e2a0 map[invoke_id:3e62501d-8948-4bf1-9e07-c3bacd9e51ac sandbox_id:0] 2019-10-29 18:42:38.866710436 +0000 UTC m=+44.600552684 panic Runtime failed to start: fork/exec /opt/bootstrap: permission denied }"
      }

      1. I was having the same issue until I noticed that my `bootstrap` file wasn't executable.

        Try running `chmod 755 bootstrap` and updating your PHP binary layer.

  6. Hi,
    I tried out your version after trying Michael Moussa's article with same results

    START RequestId: 117235f8-c40c-4621-b6c9-c3199bccd097 Version: 2
    /opt/bin/php: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /opt/bin/php)

    Which exact AMI did you use to build the PHP binary? I used amzn2-ami-hvm-2.0.20190313-x86_64-gp2 (ami-09ead922c1dad67e4) as recommended here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html

    1. I ran into this problem myself, it was a real headscratcher :-)

      I've been trying all sorts of remedies I found online which actually improved my understanding of the Lambda containers.

      Unfortunately, as always, it was the simplest fix that I left until last…

      Although the AWS documentation lists two compatible Amazon AMIs, the default runtime runs the original Amazon Linux AMI (currently: amzn-ami-hvm-2018.03.0.20181129-x86_64-gp2).

      I had been spending all my time compiling PHP in the Amazon Linux 2 environment, which has newer libraries, hence the incompatibility.

  7. I recommend using Bref – or at least it's runtime layer. It has kept up to date with the changes in Lambda's layers & runtime API.

    1. Thanks Rob. Will give that a go.
      BTW, your link to Bref is broken. Think it needs https:// rather than https:/

  8. Great article Thank you!

    I am trying to use API gateway and I get 'internal server error' response whenever I execute hello function using the HTTP endpoint.

    Any thoughts on that?

Comments are closed.