Category: PHP

Stand-alone usage of Zend-InputFilter

Any data that you receive needs to be checked and validated. There are number of ways to do this including PHP's filter_var, but I prefer Zend-InputFilter. This is how to use it as a stand-alone component.

Installation

Firstly, we install it using Composer:

$ composer require zendframework/zend-inputfilter
$ composer require zendframework/zend-servicemanager

You don't have to have ServiceManager, but it makes working with InputFilter much easier, so it's worth installing.

Create the InputFilter

The easiest way to create an InputFilter is to use the provided Factory class. Let's consider an Author entity that has the properties: author_id, name, biography & date_of_birth. We can create an input filter like this:

use Zend\InputFilter\Factory as InputFilterFactory;

class Author
{
    protected $author_id;
    protected $name;
    protected $biography;
    protected $date_of_birth;

    // ...

    protected function createInputFilter()
    {
        $factory = new InputFilterFactory();
        $inputFilter = $factory->createInputFilter([
            'author_id' => [
                'required' => true,
                'validators' => [
                    ['name' => 'Uuid'],
                ],
            ],
            'name' => [
                'required' => true,
                'filters' => [
                    ['name' => 'StringTrim'],
                    ['name' => 'StripTags'],
                ],
            ],
            'biography' => [
                'required' => false,
                'filters' => [
                    ['name' => 'StringTrim'],
                    ['name' => 'StripTags'],
                ],
            ],
            'date_of_birth' => [
                'required' => false,
                'validators' => [
                    ['name' => 'Date'],
                    [
                        'name' => 'LessThan',
                        'options' => [
                            'max' => date('Y-m-d'),
                            'inclusive' => true,
                        ],
                    ],
                ],
            ],
        ]);

        return $inputFilter;
    }
}

The createInputFilter() method takes an associative array where the key is the name of the input and then the value is a specification. There are a number of elements in the specification, but we usually just specify required, filters and validators.

required This can be either true or false. If false, then the validators do not execute, but the filters do.
filters An optional array of Zend-Filters. A filter modifies the supplied data before it is passed to the validators (if any). The filtered data is used by the rest of the application. In this example, we have added two filters: StringTrim & StripTags.
validators An optional array of Zend-Validators. A validator will test the filtered value for the input and fail if the data is not valid. If any validator fails, then the entire InputFilter is invalid.

This particular input filter requires that author_id and name are present, but that biography and date_of_birth are optional. The author_id must be a UUID, the name & biography must not have leading or trailing whitespace or no HTML tags and the date_of_birth, if present, must be a valid date in the past.

Using the InputFilter

To use the InputFilter, we set the data and then call isValid(). This can be done in a validate() method that looks like this:

Use Crell\ApiProblem\ApiProblem;
use Error\Exception\ProblemException;

Class Author
{
    // ...

    /**
     * Create an author
     *
     * @param  array $data
     * @return Author
     * @throws ProblemException
     */
    public static function createAuthor($data)
    {
        $inputFilter = $this->createInputFilter();
        $inputFilter->setData($data);

        if ($inputFilter->isValid()) {
            return new Author($inputFilter->getValues());
        }

        $problem = new ApiProblem('Validation failed');
        $problem->setStatus(400);
        $problem['errors'] = $inputFilter->getMessages();

        throw new ProblemException($problem);
    }
}

In this case, it's an API, so the data has come from a PUT or POST request. We call setData() to pass the array of data into the InputFilter and then call isValid(). If the data is valid, we can return a newly instantiated Author object that is constructed with the filter data. If the validation fails, then we throw a ProblemException which needs an ApiProblem instance, so we create one for it.

To find out which validators failed, getMessages() provides a nested array which is very useful for passing back to the API client.

As an example, this is what failure looks like:

$ curl -i -X "POST" "http://localhost:8888/authors" \
     -H "Accept: application/json" \
     -H "Content-Type: application/json" \
     -d $'{ "name": "", "author_id": "1234" }'

HTTP/1.1 400 Bad Request
Host: localhost:8888
Connection: close
X-Powered-By: PHP/7.0.14
Content-type: application/problem+json

{
    "errors": {
        "author_id": {
            "valueNotUuid": "Invalid UUID format"
        },
        "name": {
            "isEmpty": "Value is required and can't be empty"
        }
    },
    "title": "Validation failed",
    "type": "about:blank",
    "status": 400
}

Fin

That's all there is to it. Zend-InputFilter is a very flexible data filter and validator and works really well for APIs, such as those written in Slim.

Rendering problem details in Slim

As I've already noted, in the project I'm currently building, I'm rendering errors in my Slim Framework API using RFC 7807: Problem Details for HTTP APIs via Larry Garfield's ApiProblem component and rka-content-type-renderer.

One place where we need to integrate this approach into Slim is in the error handlers. Let's look at NotFound. To ensure that we return a response in the right format, we need to implement our own NotFound handler:

src/App/Handler/NotFound.php:

<?php
namespace App\Handler;

use Crell\ApiProblem\ApiProblem;
use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;

class NotFound
{
    /**
     * Invoke not found handler
     *
     * @param  ServerRequestInterface $request  The most recent Request object
     * @param  ResponseInterface      $response The most recent Response object
     *
     * @return ResponseInterface
     */
    public function __invoke(ServerRequestInterface $request, ResponseInterface $response)
    {
        $problem = new ApiProblem(
            'Not Found',
            'http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html'
        );
        $problem->setStatus(404);

        $renderer = new RKA\ContentTypeRenderer\ApiProblemRenderer();
        return $renderer->render($request, $response, $problem);
    }
}

The NotFound handler must return a Response. To do this, we create a new ApiProblem object with the title "Not Found" and set the type to HTML page that defines the 404 status code. We then instantiate the ApiProblemRenderer and call its render method. The renderer will then return a Response object in either XML or JSON based on the Accept header, with the correct Content-Type header. We then set the status code and return it.

To register our new handler, we use the container. If you're using the skeleton application as your base, then this goes in dependencies.php:

src/dependencies.php:

// Error handlers
$container['notFoundHandler'] = function () {
    return new App\Handler\NotFound();
};

Slim will now use our handler whenever a NotFoundException is raised.

This is it in action:

$ curl -i -H "Accept: application/xml" http://localhost:8888/foo
HTTP/1.1 404 Not Found
Host: localhost:8888
Connection: close
X-Powered-By: PHP/7.0.14
Content-type: application/problem+xml

<?xml version="1.0"?>
<problem>
  <title>Not Found</title>
  <type>http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html</type>
  <status>404</status>
</problem>

We can of course apply this to the other error handers: NotAllowed, Error & PhpError which all follow the same pattern as we have done for NotFound.

Rendering ApiProblem with PSR-7

In the API I'm currently building, I'm rendering errors using RFC 7807: Problem Details for HTTP APIs. As this is a Slim Framework project, it uses PSR-7, so I updated rka-content-type-renderer to support problem.

RFC 7807 defines a standard for sending details of an error in an HTTP response message. It supports both XML and JSON formats. From the RFC, an example response is:

HTTP/1.1 403 Forbidden
Content-Type: application/problem+json
Content-Language: en

{
    "type": "https://example.com/probs/out-of-credit",
    "title": "You do not have enough credit.",
    "detail": "Your current balance is 30, but that costs 50.",
    "instance": "/account/12345/msgs/abc",
    "balance": 30,
    "accounts": [
        "/account/12345",
        "/account/67890"
    ]
}

Only title and type are required, though status should also be set. Full information is in the RFC, which is one of the easier ones to read.

In PHP, Larry Garfield has created the crell/ApiProblem component. The code to implement the above message is:

Use Crell\ApiProblem\ApiProblem;

$problem = new ApiProblem("You do not have enough credit.", "http://example.com/probs/out-of-credit");
$problem
  ->setDetail("Your current balance is 30, but that costs 50.")
  ->setInstance("http://example.net/account/12345/msgs/abc");

$problem['balance'] = 30;
$problem['accounts'] = array(
  "http://example.net/account/12345",
  "http://example.net/account/67890"
);

$jsonString = $problem->asJson();
$xmlString = problem->asXml();

ApiProblemRenderer

The only tricky bit is working out if we need to send back JSON or XML. This is called content negotiation and we read the Accept header to find out what the client wants. However, the Accept header has a complicated format with quality levels as you can see from RFC 7321, section 5.3.2. Fortunately, we can use Negotiation by Will Durand to deal with this, which is how rka-content-type-renderer works.

rka-content-type-render now has an ApiProblemRenderer which will read the Accept header and work out the if the client would prefer JSON or XML. If it can't determine, it will default to JSON.

In Expressive or Slim, it's used like this:

use Crell\ApiProblem\ApiProblem;
use RKA\ContentTypeRenderer\ApiProblemRenderer;
use Zend\Diactoros\Response;

$app->get('/', function ($request, $response, $next) {
    $problem = new ApiProblem(
        'Unauthorised',
        'http://www.example.com/api/docs/authentication'
    );
    $problem->setStatus(403);

    $renderer = new ApiProblemRenderer();
    return $renderer->render($request, new Response(), $problem);
});

This is it in action, with an XML accept header:

$ curl -i -H "Accept: application/vnd.akrabat.api+xml" http://localhost:8888/
HTTP/1.1 403 Forbidden
Host: localhost:8888
Connection: close
X-Powered-By: PHP/7.0.14
Content-type: application/problem+xml

<?xml version="1.0"?>
<problem>
  <title>Unauthorised</title>
  <type>http://www.example.com/api/docs/authentication</type>
  <status>403</status>
</problem>

And with a JSON one:

$ curl -i -H "Accept: application/vnd.akrabat.api+json" http://localhost:8888/
HTTP/1.1 403 Forbidden
Host: localhost:8888
Connection: close
X-Powered-By: PHP/7.0.14
Content-type: application/problem+json

{
    "title": "Unauthorised",
    "type": "http://www.example.com/api/docs/authentication",
    "status": 403
}

Ideally, the client should specify application/problem+json in their accept header, but in practice, I've never seen that happen, which is why rka-content-type-renderer works out the preferred format based on the media types specified.

A note on framework performance

A question came up recently wondering why Slim Framework was around 5 times slower than plain PHP.

All frameworks are by definition slower than no-code as there's more going on.

i.e. an index.php of:

    <?php
    header('Content-Type: application/json');
    echo json_encode(['result' => 1]);

is going to be much faster than a Slim application with this code:

    use \Slim\App;
    include 'vendor/autoload.php';
    
    $config = include 'Config/Container.php';
    $app = new App($config);
    $app->get('/do-it', function($request, $response){
        return $response->withJson(['result' => 1]);
    });
    
    $app->run();

This is not an apples-to-apples comparison of course as the Slim application is doing a lot more than the plain PHP one. It supports routing so that it can respond to different URLs (even ones with dynamic parameters in that URL) and HTTP methods, along with the ability to handle errors. It also a set of separate classes so that you can replace specific parts of our code with your own if you want it to do something different while still using the rest of our code.

To actually compare the performance of Slim to plain PHP, you need to make the plain PHP version have the same functionality as the Slim version and you'll discover that the performance difference is much smaller. Of course, custom written, application specific code will always be faster than generic code though. Slim is a micro framework, so already is pretty tight (more to do though and something I'm looking at for 4.0), but by definition, it supports more routing features than you may need.

However, you have to write it all that code yourself and, to be honest, parsing requests, sending responses, dealing with URL routing, formatting errors based on the Accept header, and so on are solved problems that don't add any value to your app if you write them yourself. You're much better off concentrating your efforts on the parts of your app that are unique to your app as that's where it brings value.

The definitive resource on this is Paul Jones' articles. Start with A Siege On Benchmarks. This was back in 2009 and somehow he managed to not test Slim! However, even if you take the fastest framework he measured (ZF 0.2!), you can see that at 180.56 requests per second, that's around 4.5 times slower than the baseline PHP of 829.82 requests/second.

Finally, the speed of this code will pale in comparison to the rest of your application's code, especially if you access a database or a web s service!

I'm interested in making Slim the fastest it can be, but the benefits of having a reliable framework that provides good routing middleware functionality out of the box cannot be overstated.

Homestead per-project crib sheet

I wanted a drop-dead simple way to try and replicate a problem someone was having on the Slim forums. I couldn't reproduce with php -S which is my go-to for this sort of thing, so I thought I'd try Homestead.

I had recently listend to a Voices of the Elephpant episode with Taylor Otwell & Joe Ferguson where Joe mentioned that Homestead worked on a per-project basis too. I didn't know this, so tried it out. The docs are fine, but there's a lot there that covers the global installation option when I just want to get up and running on a per-project basis.This is my crib sheet:

1. Create project

We just need a project that uses Composer. You probably have one already. If not, Slim Framework is a good choice!

2. Add Homestead to the project

The make command creates VagrantFile and a Homestead.yaml for configuration.

3. Deal with IP address and hostname

By default, the Homestead vagrant box is set up on 192.168.10.10 with the hostname homestead.app. You can change this in Homestead.yaml.

Add the IP address to /etc/hosts. This only needs to be done once if you don't change the defaults.

All done

We're all done, so we can use vagrant up to run our new website Go to homestead.app in a browser to see it. To shut down, use vagrant halt or vagrant destroy.

Slim home page

Using CharlesProxy's root SSL with home-brew curl

Once I installed Homebrew's curl for HTTP/2 usage, I discovered that I couldn't automatically proxy SSL through Charles Proxy any more.

$ export HTTPS_PROXY=https://localhost:8888
$ curl https://api.joind.in/v2.1/
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

This is a nuisance.

As I've noted previously, you need to install Charles' root certificate to use it with SSL. On OS X, you do Help -> SSL Proxying -> Install Charles Root Certificate which installs it into the system keychain.

However, this doesn't work with the Homebrew curl or with the curl functions inside PHP. To fix this, we need to add the Charles root certificate to OpenSSL's default_cert_file.

I've talked about this file before. The quickest way to find it is to run:

$ php -r "print_r(openssl_get_cert_locations());"

on the command line. The output should be similar to:

Array
(
    [default_cert_file] => /usr/local/etc/openssl/cert.pem
    [default_cert_file_env] => SSL_CERT_FILE
    [default_cert_dir] => /usr/local/etc/openssl/certs
    [default_cert_dir_env] => SSL_CERT_DIR
    [default_private_dir] => /usr/local/etc/openssl/private
    [default_default_cert_area] => /usr/local/etc/openssl
    [ini_cafile] =>
    [ini_capath] =>
)

As you can see, the file I need is /usr/local/etc/openssl/cert.pem.

Grab the root certificate from the Charles app. On Mac, that's Help -> SSL Proxying -> Save Charles Root Certificate menu item.

You can then append the root certificate to the default cert file:

$ cat charles_root.crt >> /usr/local/etc/openssl/cert.pem

Now, everything works:

$ curl https://api.joind.in/v2.1/
{"events":"https:\/\/api.joind.in\/v2.1\/events","hot-events":"https:\/\/api.joind.in\/v2.1\/events?filter=hot","upcoming-events":"https:\/\/api.joind.in\/v2.1\/events?filter=upcoming","past-events":"https:\/\/api.joind.in\/v2.1\/events?filter=past","open-cfps":"https:\/\/api.joind.in\/v2.1\/events?filter=cfp","docs":"http:\/\/joindin.github.io\/joindin-api\/"}

(It also works in PHP as that's linked against the same curl if you followed my post on enabling HTTP/2 in PHP.)

Using HTTP/2 with PHP 7 on Mac

if you want to use HTTP/2 with PHP on OS X, you need a version of curl compiled with HTTP/2 support. You can then link your PHP's curl extension to this version.

The easiest way to do this is to use Homebrew:

At the time of writing, this will install PHP 7.0.10 with Curl 7.50.1:

Using Curl on the command line

If you want to use your shiny new curl from the command line, then the easiest way to do this is:

You can now do:

and you should get:

(at the time of writing!)

Using Guzzle to test HTTP/2 is working

To prove it works in PHP, use Guzzle!:

testhttp2.php:

Run this PHP code at the command line:

As we've turned on debugging, the output looks like this:

The key things to notice:

We tell the server that we want HTTP/2 (h2), but can accept HTTP/1.1. If the server doesn't support HTTP/2 it will send back HTTP/1.1

This is a good sign!

The response's status line is:

The version number is 2 and we got a 200, so all is OK!

Everything is working as intended.

Passing on the baton

Lorna Mitchell has posted Joind.in Needs Help:

For the last 6 years I've been a maintainer of this project, following a year or two of being a contributor. Over the last few months, myself and my comaintainer Rob Allen have been mostly inactive due to other commitments, and we have agreed it's time to step aside and let others take up the baton.

I'm proud of my contributions to joind.in as a contributor and maintainer. I couldn't have done it without Lorna's encouragement (and willingness to point out my mistakes!). The project has had a significant influence in my Open Source journey as my work has led to my leadership role in Slim Framework and interest in writing APIs.

If you want to guide the next stage of joind.in's journey, please find us in #joind.in on Freenode IRC.

Checklist for releasing Slim

Release process for Slim so that I don't forget any steps; based on a check list created by Asgrim. I should probably automate some of this!

Preparation:

  • Ensure all merged PRs have been associated to the tag we're about to release.
    Find them via this search: [is:pr is:closed no:milestone is:merged].
  • Close the milestone on GitHub.
  • Create a new milestone for the next patch release.
  • Ensure that you have the latest master & that your index is clean.
  • Find the ID of the current milestone. This is the numeric id found in the URL of the milestone detail page (e.g. 34).
  • Generate the changeling using changelog_generator & copy to clipboard:
    changelog_generator.php -t {your-github-api-token} -u slimphp -r Slim -m {ID} | pbcopy

Tag and release:

  • Edit App.php and update the VERSION constant to the correct version number. Commit the change.
  • Tag the commit: git tag -s {x.y.z} & paste the change log generated above into the tag message.
  • Update App.php again and set the VERSION constant to {x.y+1.0}-dev & commit.
  • Push the commits and the new tag: git push --follow-tags
  • Go to the releases page and click "Draft a new release":
    • Enter the tag name and ensure that you see a green tick and "Existing tag" appear.
    • Set the release title to the same as the tag name.
    • Paste the change log generated above into the release notes box (it is already formatted with Markdown).
    • Click "Publish release".
  • Write announcement blog post for website & publish.

Standalone Doctrine Migrations redux

Since, I last wrote about using the Doctrine project's Migrations tool independently of Doctrine's ORM, it's now stable and much easier to get going with.

Installation and configuration

As with all good PHP tools, we simply use Composer:

This will install the tool and a script is placed into vendor/bin/doctrine-migrations.

To configure, we need to add two files to the root of our project:

migrations.yml:

This file sets up the default configuration for Migrations. The most important two settings are table_name which is the name of the database table holding the migration information and migrations_directory which is where the migration files will be stored. This directory name may be a relative path (relative to the directory where migrations.yml is stored, that is).

migrations-db.php:

This file is simply the standard DBAL configuration for a database connection. In this case, I'm using SQLite. Again, the path is relative.

Using Migrations

Migrations is a command line tool called doctrine-migrations that has been installed into the vendor/bin/ directory. Running the tool without arguments will give you helpful information:

The two important command are: migrations:generate and migrations:migrate.

Creating a migration

To generate a new migration, we run:

This will create a new file for us in our migrations directory which we then need to fill in. There are two methods class: up() and down() and the rule is that whatever we do in up() must be reversed in down().

For example to create a new table, we code it like this:

If you want to write your schema changes as SQL, then use the $this->addSql() method.

We can also populate the newly created table adding the postUp() method to the class. As its name suggests, the code in the method is execute after the code in up(). In postUp() we access the connection property and then call the appropriate methods. For example:

Alternatively, we could have used executeQuery() and written it as SQL:

Running the migrations

To update the database to the latest migration we run the migrations:migrate command. This will run the up() method on all migration files that haven't already been run. For example:

In this case, I have two migration files in my migrations directory and started with an empty database.

That's it

That's all there is to it. Migrations is very easy to use as a standalone tool and worth considering for your database migrations.