Manuel Stosic: Understanding Zend Framework 3 before it’s out

Manuel Stosic has posted Understanding Zend Framework 3 before it’s out

ZF3 is not close around the corner. It’s still many, many months ahead. But there are reasons why you should bother and get information about ZF3 as soon as possible.

Manuel goes on to explain that you can find out information about ZF3′s development on Google Moderator, PRs on GitHub, the wiki and an upcoming Hangout next week.

Lorenzo Ferrara: Testing Apiagility Code-Connected REST API

Lorenzo Ferrara has posted Testing Apigility Code-Connected REST API

First thing, I’ve installed Apigility following the readme on GitHub, opened the admin interface and clicked on the “Get Started!” button. Things are pretty straightforward: added the new API clicking on the “Create New API” button located in the top-right corner, typed in FortuneCookie and pressed the “Create API”. Next thing, I’ve added the new Code-Connected REST service called OpenCookie. So far so good.

He goes through the creation of a simple Code-Connected REST API by providing the code for a Resource and an Entity class. He also noticed a gotcha where the hydrator wasn’t set up automatically and points out how to update the module.config.php file to add this.

Investigating Apigility

At ZendCon 2013, Zend announced Apigility which is intended to ease the creation of APIs.

It consists of these things:

  • A set of ZF2 modules that do the heavy lifting of creating an API
  • A application wrapper for creating standalone web API applications
  • A built-in administration website for use in development to define the API

Rather nicely, it supports REST and RPC and deal with error handling, versioning & content negotiation for you.

Getting started from nothing to create a new API is quite easy: you simply install the Apigility Skeleton Application following the the instructions in the README and then create your new APIs using the admin application.

The more interesting case for me is how to use Apigility to supply an API to an existing application, so I’m going to explore how to do this.

Let’s start by adding an API to the ZF2 tutorial application.

Install the tutorial application

The easiest way to do this is to use my zf2-tutorial-to-go GitHub repository. Just follow the instructions in the README to set it up. (Make sure that you use PHP 5.4+ so you can use the built in web server.)

Add Apigility

To add Apigility to our application we update composer.json:

We set the minimum stability and add a new repository and then add some packages to the require and require-dev sections.

Update the entire composer.json so that it contains:

{
    "name": "akrabat/zf2-tutorial-apigility",
    "description": "ZF2 Tutorial with Apigility",
    "license": "BSD-3-Clause",
    "minimum-stability": "dev",
    "repositories": [
        { "type": "composer",
          "url": "https://packages.zendframework.com" }
    ],
    "require": {
        "php": ">=5.3.3",
        "zendframework/zendframework": "dev-develop",
        "zfcampus/zf-apigility": "dev-master"
    },
    "require-dev": {
        "zendframework/zftool": "dev-master",
        "zendframework/zend-developer-tools": "dev-master",
        "zfcampus/zf-apigility-admin": "dev-master"
    }
}

and run composer:

$ php composer.phar update

Once it’s finished, we can update our application.

Update the application

There are a number of things we need to do to update our application to support Apigility.

Firstly, we need to add the modules, but some of them are development only and not intended for production. One way to do support this is to provide a separate config/development.config file that is only loaded if it exists and then ensure that this file isn’t deployed to production.

To support this, we modify public/index.php:

Replace:

// Run the application!
ZendMvcApplication::init(require 'config/application.config.php')->run();

with

$appConfig = require 'config/application.config.php';
if (file_exists('config/development.config.php')) {
    $appConfig = Zend\Stdlib\ArrayUtils::merge($appConfig, require 'config/development.config.php');
}

// Run the application!
ZendMvcApplication::init($appConfig)->run();

As you can see, this simply checks for the presence of development.config.php and merges it.

We can now add our modules. Firstly add the following modules to config/application.config.php:

    'ZF\Apigility',
    'AssetManager',
    'ZF\ApiProblem',
    'ZF\Hal',
    'ZF\ContentNegotiation',
    'ZF\Rest',
    'ZF\Configuration',
    'ZF\Versioning',

And then create config/development.config.php with the following contents:

< ?php
// NOTE: DO NOT deploy this file to LIVE

return array(
    'modules' => array(
        'ZFTool',
        'ZF\Apigility\Admin',
    ),
);

Within Apigility, the DB-Connected functionality relies on ZF2′s Adapter\AbstractServiceFactory, so we need to register this with the Service Manager. The easiest place to do this is in config/autoload/global.php.

Update config/global.php and add:

        'abstract_factories' => array(
           'Zend\\Db\\Adapter\\AdapterAbstractServiceFactory',
        ),

within the 'service_manager' array.

As we’re in development mode, we can now start up Apigility’s admin system and define an API for our albums. From the command line, in the application’s base directory you can start the PHP internal web server using:

$ php -S 0.0.0.0:8080 -t public/ public/index.php

and then navigate to http://localhost:8080 to see a list of albums and http://localhost:8080/admin to see the Apigility admin:

Apiligity admin

Use Apigility admin to create an API

The simplest API to create with Apigility is a DB-Connected one. In this model, we tell Apigility how to connect to our database and it generates an API for us.

The first part of this to create a database adapter for apigility’s use.

Click on the Database Adapters link on the admin dashboard and press the Create New DB Adapter button.

Enter the following into the Create new DB Adapter form:

  • Adapter Name: DBAlbum
  • Driver Type: Pdo_Sqlite
  • Database: data/album.sqlite

And then press the Create Db Adapter button to create our new adapter.

We can now create an API, so press the Create New API button in the top right of the admin. Enter AlbumApi into the form field and press the Create API button.

This creates version 1 of our API and we now need to create a service. We will create a REST service, so click on the REST Services link and then click the Create New REST Service button.

Choose the DB-Connected tab and then fill out the form:

  • DB Adapter Name: DBAlbum
  • Table Name: album

Press the Create DB-Connected REST Service button to create the service.

We now need to set our new service up correctly, so click on the AlbumApiV1RestAlbumAlbumEntity link and select the Edit tab.

Firstly we update the route as we already have a /album route. Update Route to match to be /api/album[/:id]

Also, in the REST Parameters section change the Identifier Name from album_id to id.

Finally, press the Save button.

(Note that I had trouble with the Identifer Name reverting back to to album_id, so I edited module/AlbumApi/config/module.config.php and replaced all instances of album_id with id and that solved it. I assume that this is a minor bug somewhere in the admin system.)

You can also edit other aspects of your API here. I particularly like that you can choose which HTTP verbs that your API responds to, so to make a read-only API, you could simply uncheck all verbs other than GET.

Test your new API

I use curl on the command line for testing things:

curl -s -H "Accept: application/json" 
http://localhost:8080/api/album | python -mjson.tool

And you should see a nicely formatted list of albums with HAL links.

GitHub repository

I have created a GitHub repository called zf2-tutorial-apigility which contains the code. The master branch is the fully working DB-connected API and the base tag is at the point before we used the Apigility admin system.

Conclusion

Apigility doesn’t seem too hard to integrate into an existing ZF2 application after all. I particularly like that the admin system is development only and the code it creates can be checked into your VCS for easy inspection of what it has done.

Obviously a DB-connected resource is a very simply integration, but I’m pleased to see that Apigility has provided an AlbumCollection.php and an AlbumEntity.php file with the AlbumApi module to allow for customisation.

Given that Apigility was announced at ZendCon, I’m fairly confident that more work will be done the project and I look forward to new features such as authentication and validation that are sure to be coming soon.

Changing the GitHub IRC hooks notification events

As joind.in uses GitHub to host its source code, we use the IRC hook to receive notifications to the IRC channel (#joind.in on freenode) when interesting things happen on the GitHub repositories.

We noticed recently that we were being notified about more types of things happening on some repositories compared to others, so I decided to investigate.

The code for this is here and a quick perusal of it shows that it will do something for these events:

  • push
  • commit_comment
  • pull_request
  • pull_request_review_comment
  • issues
  • issue_comment

However, when you go the administration page on the website, you cannot set which events to be notified on.

Fortunately, GitHub has an API and so you can quite easily manipulate the hooks using curl.

To read all the hooks attached to a project, you do this:

curl -u '{your github username}' -H "Accept: application/json" 
https://api.github.com/repos/{owner name}/{repository name}/hooks

so, for the joind.in API project, I do:

curl -u 'akrabat' -H "Accept: application/json" 

https://api.github.com/repos/joindin/joindin-api/hooks

This will return a list of hooks attached, including all the information about them. In this case, I’m interested in the IRC hook and there’s two interesting keys in the results:

    "url": "https://api.github.com/repos/joindin/joindin-api/hooks/932001",
    "events": [
      "push",
      "pull_request"
    ],

The url provides the information on the end point to use to change the data and the events tell me what’s set up. As you can see, only “push” and “pull_request” are set up for this repository.

To change this, we simply use CURL to POST a new set of events:

curl -u 'akrabat'  -H "Accept: application/json" -H "Content-type: application/json" -X PATCH 

https://api.github.com/repos/joindin/joindin-api/hooks/932001

-d '{"events":["push", "pull_request", "commit_comment", "pull_request_review_comment"]}'

Helpfully, the response includes a representation of the hook we have just modified, so we can see that the events list has changed. For more information, look at GitHub’s API documentation for hooks.

I therefore went through all the joind.in repositories and set them all to the same set of events for consistency. We’ll no doubt find out soon if that results in too many notifications to the channel!

Returning JSON errors in a ZF2 application

If you have a standard ZF2 application and accept application/json requests in addition to application/html, then you have probably noticed that when an error happens, HTML is created, even though the client has requested JSON.

One way to fix this is to create a listener on MVC’s render event to detect that an error has occurred and substitute a JsonModel in place of the ViewModel.

The easiest way to do this in your ApplicationModule.

Firstly, attach a lister to render:

class Module
{
    public function onBootstrap(MvcEvent $e)
    {
        // attach the JSON view strategy
        $app      = $e->getTarget();
        $locator  = $app->getServiceManager();
        $view     = $locator->get('ZendViewView');
        $strategy = $locator->get('ViewJsonStrategy');
        $view->getEventManager()->attach($strategy, 100);

        // attach a listener to check for errors
        $events = $e->getTarget()->getEventManager();
        $events->attach(MvcEvent::EVENT_RENDER, array($this, 'onRenderError'));
    }

Now write the error detector:

    public function onRenderError($e)
    {
        // must be an error
        if (!$e->isError()) {
            return;
        }

        // Check the accept headers for application/json
        $request = $e->getRequest();
        if (!$request instanceof HttpRequest) {
            return;
        }

        $headers = $request->getHeaders();
        if (!$headers->has('Accept')) {
            return;
        }

        $accept = $headers->get('Accept');
        $match  = $accept->match('application/json');
        if (!$match || $match->getTypeString() == '*/*') {
            // not application/json
            return;
        }

        // make debugging easier if we're using xdebug!
        ini_set('html_errors', 0); 

        // if we have a JsonModel in the result, then do nothing
        $currentModel = $e->getResult();
        if ($currentModel instanceof JsonModel) {
            return;
        }

        // create a new JsonModel - use application/api-problem+json fields.
        $response = $e->getResponse();
        $model = new JsonModel(array(
            "httpStatus" => $response->getStatusCode(),
            "title" => $response->getReasonPhrase(),
        ));

        // Find out what the error is
        $exception  = $currentModel->getVariable('exception');

        if ($currentModel instanceof ModelInterface && $currentModel->reason) {
            switch ($currentModel->reason) {
                case 'error-controller-cannot-dispatch':
                    $model->detail = 'The requested controller was unable to dispatch the request.';
                    break;
                case 'error-controller-not-found':
                    $model->detail = 'The requested controller could not be mapped to an existing controller class.';
                    break;
                case 'error-controller-invalid':
                    $model->detail = 'The requested controller was not dispatchable.';
                    break;
                case 'error-router-no-match':
                    $model->detail = 'The requested URL could not be matched by routing.';
                    break;
                default:
                    $model->detail = $currentModel->message;
                    break;
            }
        }

        if ($exception) {
            if ($exception->getCode()) {
                $e->getResponse()->setStatusCode($exception->getCode());
            }
            $model->detail = $exception->getMessage();

            // find the previous exceptions
            $messages = array();
            while ($exception = $exception->getPrevious()) {
                $messages[] = "* " . $exception->getMessage();
            };
            if (count($messages)) {
                $exceptionString = implode("n", $messages);
                $model->messages = $exceptionString;
            }
        }

        // set our new view model
        $model->setTerminal(true);
        $e->setResult($model);
        $e->setViewModel($model);
    }

You’ll also need some use statements:

use ZendHttpRequest as HttpRequest;
use ZendViewModelJsonModel;
use ZendViewModelModelInterface;

(This code is heavily inspired from PhlyRestfully – Thanks Matthew!)

Essentially, we check that we are in an error situation and that the client wants JSON. If we are, we create a JsonModel and populate it information. I’ve used the fields from the draft Problem Details for HTTP APIs IETF spec as it seems sensible to do so.

Note though, that if you run this, you’ll see that the response’s content-type is application/json, not application/api-problem+json. You can’t set this in onRenderError though as the view’s JSON strategy will override it.

A brute-force solution to this is to override the content-type header in a listener on the finish event. Firstly we update onBootstrap():

class Module
{
    public function onBootstrap(MvcEvent $e)
    {
        // ...
        $events->attach(MvcEvent::EVENT_FINISH, array($this, 'onFinish'));
    }

Then we write the onFinish listener:

    public function onFinish($e)
    {
        $response = $e->getResponse();
        $headers = $response->getHeaders();
        $contentType = $headers->get('Content-Type');
        if (strpos($contentType->getFieldValue(), 'application/json') !== false
            && strpos($response->getContent(), 'httpStatus')) {
            // This is (almost certainly!) an api-problem
            $headers->addHeaderLine('Content-Type', 'application/api-problem+json');
        }
    }

This method simply looks at the response and tries to guess if it’s an api-problem. If it is, then it changes the content-type in the Response’s header.

Now you’re done.

You can test with curl:

$ curl -s -i -H "Accept: application/json" "http://localhost/booklist/public/invalid" 

which will return:

HTTP/1.1 404 Not Found
Date: Mon, 09 Sep 2013 08:55:01 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.4.19 mod_ssl/2.2.22 OpenSSL/0.9.8x
X-Powered-By: PHP/5.4.19
Content-Length: 100
Content-Type: application/api-problem+json

{"httpStatus":404,"title":"Not Found","detail":"The requested URL could not be matched by routing."} 

Of course, in an ideal world, someone would package this up into a module :)

rst2html does not support :start inline:

I’m currently writing some documentation in Restructured Text that I’m targeting at HTML and PDF using rst2html and rst2pdf.

For syntax highlighting, both rst2html and rst2pdf use Pygments, however rst2html doesn’t support any Pygments options.

So a typical PHP snippet in rst targeting rst2pdf, would be written as:

.. code-block: php
    :startinline: true

    $this = str_replace('foo', 'bar', $that);

The startinline Pygments option is to allow it to highlight the snippet, even though the opening <?php is missing.

If you run rst2html, you get this error:

(ERROR/3) Error in "code-block" directive: unknown option: "startinline".

aargh!

To remove the error, I’ve simply run my rst file through see first, so I have a create-html.sh script that does this:

#!/bin/bash

sed '/startinline/d' ${1}.rst > temp.rst
rst2html.py --syntax-highlight=short --stylesheet=syntax.css temp.rst > ${1}.html
rm temp.rst

The sed command removes any lines that contain the word startinline and obviously, this means that the HTML version doesn’t syntax highlight the snippet, but at least it doesn’t generate an error!

Setting up Zend Server 6 on OS X for PHP development

I recently decided to upgrade my Mac’s PHP to 5.4. One of the available options is Zend Server 6.1, Free edition. These are my notes how to set it up so that it works the way I develop.

Installation

Mount the disk image and follow the installation wizard. On OS X, Zend Server installs PHP, Apache and MySQL inside /usr/local/zend.

The Zend Server admin interface is at at http://localhost:10081. On first run you have to go through a wizard. I picked development for my Launch Option. You have to set an admin password – as it’s my dev machine, I picked something simple :)

More information is available from Joe Stagner’s guide on devzone. Ignore the bit about editing php.ini though as you can do that via Zend Server’s admin system.

Apache

Like Joe, I also want the Apache running on port 80. Edit /usr/local/zend/apache2/conf/httpd.conf and replace Listen 10888 with Listen 80.

Note that the default localhost directory is /usr/local/zend/apache2/htdocs and the Apache processes run as user daemon with group daemon.

Zend Server admin

Once logged in, you can configure Zend Server:

Configuration -> PHP:
Set your PHP settings are required. Then press Save. To restart the server, click the button with two arrows in a circle on the toolbar.

You must set the date.timezone to stop the PHP warning. I also set sendmail_path as I like to redirect email on my development box.

Configuration -> Components:
Look down the list and turn off what you don’t need. I turned off Zend Data Cache, Zend Job Queue and Zend Page Cache in addition to what is already off.

Command line control

Add the relevant Zend Server paths to your environment. Edit .bash_profile and add

export PATH=/usr/local/zend/bin:/usr/local/zend/mysql/bin:$PATH 

to the bottom. This will give you access to the php, mysql and other Zend Server related CLI tools.

zendctl.sh

The main CLI tool to control Zend Server is zendctl.sh. The most interesting commands are:

  • sudo zendctl.sh restart-apache to restart Apache
  • sudo zendctl.sh restart-mysql to restart MySQL

MySQL

Zend Server supplies MySQL version 5.5.27 rather than 5.6 for some reason. If you want 5.6, then grab it from MySQL’s website or homebrew.

If you use your own MySQL, then stop Zend Server from starting its one by editing /etc/zcd.rc and changing MYSQL_EN=true to MYSQL_EN=false.

If you’re using Zend Server’s MySQL, then set a root password:

$ mysqladmin -u root password {new-password}
$ mysqladmin -u root -p{new-password} -h localhost password {new-password}
$ mysqladmin -u root -p{new-password} reload
$ history -c

I also had to link mysql.sock to /tmp so that some mysql CLI applications would work:

  • sudo ln -s /usr/local/zend/mysql/tmp/mysql.sock /tmp/mysql.sock

Apache vhosts

I couldn’t work out how to configure Apache virtual hosts in the Zend Server admin unless you had an “application package”, so I did them manually.

Apache is configured to look in /usr/local/zend/etc/sites.d/ for files names globals-*.conf and vhost_*.conf.

I created globals-vhost.conf:

NameVirtualHost *:80

LoadModule vhost_alias_module modules/mod_vhost_alias.so

And then I have vhost_dev.conf to configure my virtual hosts:

<virtualhost *:80>
    ServerAdmin webmaster@akrabat.com
    DocumentRoot "/www"
    ServerName localhost
    <directory "/www">
	    Options Indexes FollowSymLinks MultiViews
    	AllowOverride All
        Order allow,deny
	    Allow from all
    </directory>
</virtualhost>

<virtualhost *:80>
    VirtualDocumentRoot "/www/dev/%-2+/public"
    ServerName subdomains.dev
    ServerAlias *.dev
    UseCanonicalName Off
    LogFormat "%V %h %l %u %t "%r" %s %b" vcommon
    ErrorLog "/www/dev/vhosts-error_log"
    DirectoryIndex index.php index.html
    <directory "/www/dev/*">
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Order allow,deny
        Allow from all
        
        SetEnv APPLICATION_ENV development
                
        RewriteEngine On
        RewriteBase /
        RewriteCond %{REQUEST_FILENAME} -s [OR]
        RewriteCond %{REQUEST_FILENAME} -l [OR]
        RewriteCond %{REQUEST_FILENAME} -d
        RewriteRule ^.*$ - [NC,L]
        RewriteRule ^.*$ index.php [NC,L]        
    </directory>    
</virtualhost>

This allows me to put my development files in /www and sets up automatic virtual hosts for http://*.dev.

Fix sending email

Sending email doesn’t work out of the box. To fix:

$ sudo mv /usr/local/zend/lib/libsasl2.2.dylib /usr/local/zend/lib/libsasl2.2.dylib.old
$ sudo ln -s /usr/lib/libsasl2.2.dylib /usr/local/zend/lib/libsasl2.2.dylib

See here for details.

Logs

The Zend Server admin displays your logs via Overview -> Logs. There is a list of log files down the left hand side. The interesting ones for me are:

  • error: Apache error log (/usr/local/zend/var/log/error.log)
  • access: Apache access log (/usr/local/zend/var/log/access.log)
  • php: PHP log (/usr/local/zend/var/log/php.log)

That’s it. Zend Server now works as a development stack and you get the nice diagnostics features like code tracing and the events list.