All posts by Rob

Simple way to add a filter to Zend-InputFilter

21 June 2017PHP, Zend FrameworkRob

Using Zend-InputFilter is remarkably easy to use:

use Zend\InputFilter\Factory as InputFilterFactory;

// set up InputFilter
$specification = [
    'my_field' => [
        'required' => false,
        'filters' => [
            ['name' => 'StringTrim'],
        ],
    ],
];
$factory = new InputFilterFactory();
$inputFilter = $factory->createInputFilter($specification);

// use InputFilter on some data
$data['my_field] = 'Some string';
$inputFilter->setData($data);
if ($inputFilter->isValid()) {
    Return false;
}
return $inputFilter->getValues(); // my_field is now trimmed

use Zend\InputFilter\Factory as InputFilterFactory;

// set up InputFilter

$specification = [

'my_field' => [

'required' => false,

'filters' => [

['name' => 'StringTrim'],

];

$factory = new InputFilterFactory();

$inputFilter = $factory->createInputFilter($specification);

// use InputFilter on some data

$data['my_field] = 'Some string';

$inputFilter->setData($data);

if ($inputFilter->isValid()) {

Return false;

}

return $inputFilter->getValues(); // my_field is now trimmed

How do you add your filter to it though?

This is the world's most simple filter that does absolutely nothing: We'll call it MyFilter and store it in App\Filter\MyFilter.php:

<?php
namespace App\Filter;

use Zend\Filter\AbstractFilter;

class ToString extends AbstractFilter
{
    public function filter($value)
    {
        // do something to value here
        return $value;
    }
}

<?php

namespace App\Filter;

use Zend\Filter\AbstractFilter;

class ToString extends AbstractFilter

{

public function filter($value)

{

// do something to value here

return $value;

}

Now you have a couple of choices:

Extend Zend\InputFilter\Factory

I needed to add my own filter in the least invasive way that I could and so I created App\InputFilter\Factory which extends Zend\InputFilter\Factory:

<?php
namespace App\InputFilter;

use App\Filter\MyFilter;
use Zend\InputFilter\Factory as BaseFactory;
use Zend\InputFilter\InputFilterPluginManager;
use Zend\ServiceManager\Factory\InvokableFactory;

class Factory extends BaseFactory
{
    /**
     * @param InputFilterPluginManager $inputFilterManager
     */
    public function __construct(InputFilterPluginManager $inputFilterManager = null)
    {
        parent::__construct($inputFilterManager);

        // register our own filters
        $filterPluginManager = $this->getDefaultFilterChain()->getPluginManager();
        $filterPluginManager->setFactory(MyFilter::class, InvokableFactory::class);
        $filterPluginManager->setAlias('MyFilter', MyFilter::class);
    }
}

<?php

namespace App\InputFilter;

use App\Filter\MyFilter;

use Zend\InputFilter\Factory as BaseFactory;

use Zend\InputFilter\InputFilterPluginManager;

use Zend\ServiceManager\Factory\InvokableFactory;

class Factory extends BaseFactory

{

/**

* @param InputFilterPluginManager $inputFilterManager

public function __construct(InputFilterPluginManager $inputFilterManager = null)

{

parent::__construct($inputFilterManager);

// register our own filters

$filterPluginManager = $this->getDefaultFilterChain()->getPluginManager();

$filterPluginManager->setFactory(MyFilter::class, InvokableFactory::class);

$filterPluginManager->setAlias('MyFilter', MyFilter::class);

}

This class extends the standard Factory class and registers our filter into the filter chain's plugin manager. Note that we have to register the factory for the fully qualified filter classname and also we register an alias for the short form ('MyFilter') as that's much nicer to use in the specification.

To use our new factory, we change the use statement to use our new factory:

use App\InputFilter\Factory as InputFilterFactory;

1	use App\InputFilter\Factory as InputFilterFactory;

Now we can use 'MyFilter' in our specification:

$specification = [
    'my_field' => [
        'required' => false,
        'filters' => [
            ['name' => 'StringTrim'],
            ['name' => 'MyFilter'],
        ],
    ],
];

$specification = [

'my_field' => [

'required' => false,

'filters' => [

['name' => 'StringTrim'],

['name' => 'MyFilter'],

];

Update your container's factory

If you're already injecting the InputFilter's Factory to the class that's specifying the InputFilter, then it's easier to update that factory. For Pimple, this looks something like:

<?php
use App\Filter\MyFilter;
use Zend\InputFilter\Factory;
use Zend\ServiceManager\Factory\InvokableFactory;

$container['InputFilterFactory'] = function ($c) {
    $factory = new Factory();

    // register our own filters
    $filterPluginManager = $factory->getDefaultFilterChain()->getPluginManager();
    $filterPluginManager->setFactory(MyFilter::class, InvokableFactory::class);
    $filterPluginManager->setAlias('MyFilter', MyFilter::class);
    $inputFilter = $factory->createInputFilter($specification);

    return $factory;
};

<?php

use App\Filter\MyFilter;

use Zend\InputFilter\Factory;

use Zend\ServiceManager\Factory\InvokableFactory;

$container['InputFilterFactory'] = function ($c) {

$factory = new Factory();

// register our own filters

$filterPluginManager = $factory->getDefaultFilterChain()->getPluginManager();

$filterPluginManager->setFactory(MyFilter::class, InvokableFactory::class);

$filterPluginManager->setAlias('MyFilter', MyFilter::class);

$inputFilter = $factory->createInputFilter($specification);

return $factory;

};

We don't need to change anything else and we can use 'MyFilter' in our specification:

$specification = [
    'my_field' => [
        'required' => false,
        'filters' => [
            ['name' => 'StringTrim'],
            ['name' => 'MyFilter'],
        ],
    ],
];

$specification = [

'my_field' => [

'required' => false,

'filters' => [

['name' => 'StringTrim'],

['name' => 'MyFilter'],

];

Default route arguments in Slim

14 June 2017Slim FrameworkRob

A friend of mine recently asked how to do default route arguments and route specific configuration in Slim, so I thought I'd write up how to do it.

Consider a simple Hello route:

$app->get("/hello[/{name}]", function ($request, $response, $args) {
    $name = $request->getAttribute('name');
    return $response->write("Hello $name");
})

$app->get("/hello[/{name}]", function ($request, $response, $args) {

$name = $request->getAttribute('name');

return $response->write("Hello $name");

})

This will display "Hello " for the URL /hello and "Hello Rob" for the URL /hello/Rob.

If we wanted a default of "World", we can set an argument on the Route object that is returned from get() (and all the other routing methods):

$app->get("/hello[/{name}]", function ($request, $response, $args) {
    $name = $request->getAttribute('name');
    return $response->write("Hello $name");
})->setArgument('name', 'World');

$app->get("/hello[/{name}]", function ($request, $response, $args) {

$name = $request->getAttribute('name');

return $response->write("Hello $name");

})->setArgument('name', 'World');

This works exactly as you would expect.

The route arguments don't have to be placeholder and you can set multiple route arguments. For example:

$app->get("/hello[/{name}]", function ($request, $response, $args) {
    $name = $request->getAttribute('name');
    $foo = $request->getAttribute('foo');
    return $response->write("Hello $name, foo = $foo");
})->setArguments([
	'name' => 'World,
	'foo' => 'bar',
]);

$app->get("/hello[/{name}]", function ($request, $response, $args) {

$name = $request->getAttribute('name');

$foo = $request->getAttribute('foo');

return $response->write("Hello $name, foo = $foo");

})->setArguments([

'name' => 'World,

'foo' => 'bar',

]);

Now, we have a foo attribute in our request, which is a per-route configuration option that you can do with as you wish – e.g. setting acl rules like this:

$app->get("/users", App\HelloAction::class)->setArgument("role", "userAdministrator");

1	$app->get("/users", App\HelloAction::class)->setArgument("role", "userAdministrator");

Slim's route cache file

31 May 2017MeRob

When you have a lot of routes, that have parameters, consider using the router's cache file to speed up performance.

To do this, you set the routerCacheFile setting to a valid file name. The next time the app is run, then the file is created which contains an associative array with data that means that the router doesn't need to recompile the regular expressions that it uses.

For example:

$config = [
    'settings' => [
        'addContentLengthHeader' => false,
        'displayErrorDetails' => true, // set to false in production
        'routerCacheFile' => __DIR__ . '/../routes.cache.php',
    ],
];

$app = new Slim\App($config);

$config = [

'settings' => [

'addContentLengthHeader' => false,

'displayErrorDetails' => true, // set to false in production

'routerCacheFile' => __DIR__ . '/../routes.cache.php',

];

$app = new Slim\App($config);

Note that there's no invalidation on this cache, so if you add or change any routes, you need to delete this file. Generally, it's best to only set this in production.

As a very contrived example to show how it works, consider this code:

<?php
require __DIR__ . '/../vendor/autoload.php';

$config = [
    'settings' => [
        'addContentLengthHeader' => false,
        'displayErrorDetails' => true, // set to false in production
        'routerCacheFile' => __DIR__ . '/../routes.cache.php',
    ],
];

$app = new Slim\App($config);

for ($groupName = 1; $groupName <= 25; $groupName++) {
    $app->group("/$groupName", function() {
        for ($i = 1; $i <= 1000; $i++) {
            $this->get("/$i/{id:\d+}", App\Action::class);
            $this->post("/$i/{id:\d+}", App\Action::class);
            $this->put("/$i/{id:\d+}", App\Action::class);
            $this->delete("/$i/{id:\d+}", App\Action::class);
        }
    });
}

// Run!
$start = microtime(true);
$app->run();
$diff = microtime(true) - $start;
echo "\n" . sprintf("%0.03f", $diff * 1000);

<?php

require __DIR__ . '/../vendor/autoload.php';

$config = [

'settings' => [

'addContentLengthHeader' => false,

'displayErrorDetails' => true, // set to false in production

'routerCacheFile' => __DIR__ . '/../routes.cache.php',

];

$app = new Slim\App($config);

for ($groupName = 1; $groupName <= 25; $groupName++) {

$app->group("/$groupName", function() {

for ($i = 1; $i <= 1000; $i++) {

$this->get("/$i/{id:\d+}", App\Action::class);

$this->post("/$i/{id:\d+}", App\Action::class);

$this->put("/$i/{id:\d+}", App\Action::class);

$this->delete("/$i/{id:\d+}", App\Action::class);

}

});

}

// Run!

$start = microtime(true);

$app->run();

$diff = microtime(true) - $start;

echo "\n" . sprintf("%0.03f", $diff * 1000);

This application creates 25 groups, each with 4000 routes, each of which has a placeholder parameter with a constraint. That's quite a lot of routes, but things take long enough that we can see timing. The App\Action does nothing.

On my computer, using PHP 7.0.18's built-in web server, the first time we run it, we see this:

$ curl "http://localhost:8888/1/2/3"
2722.414

1 2	$ curl "http://localhost:8888/1/2/3" 2722.414

This took 2.7 seconds to execute. At the same time, it also created a file called routes.cache.php which is then used for the next run:

$ curl "http://localhost:8888/1/2/3"
263.228

1 2	$ curl "http://localhost:8888/1/2/3" 263.228

This time, it took just 263ms.

That's a big difference!

If you have a lot of complex routes in your Slim application, then I recommend that you test whether enabling route caching makes a difference.

Inserting binary data into SQL Server with ZF1 & PHP 7

22 May 2017PHP, Zend Framework 1Rob

If you want to insert binary data into SQL Server in Zend Framework 1 then you probably used the trick of setting an array as the parameter's value with the info required by the sqlsrv driver as noted in Some notes on SQL Server blobs with sqlsrv.

Essentially you do this;

$data['filename'] = 'test.gif';
$data["file_contents"] = array(
    $binaryData,
    SQLSRV_PARAM_IN,
    SQLSRV_PHPTYPE_STREAM(SQLSRV_ENC_BINARY), 
    SQLSRV_SQLTYPE_VARBINARY('max')
);
$db->insert($data);

$data['filename'] = 'test.gif';

$data["file_contents"] = array(

$binaryData,

SQLSRV_PARAM_IN,

SQLSRV_PHPTYPE_STREAM(SQLSRV_ENC_BINARY),

SQLSRV_SQLTYPE_VARBINARY('max')

);

$db->insert($data);

Where $db is an instance of Zend_Db_Adapter_Sqlsrv.

If you use SQL Server with ZF1 and happen to have updated to PHP 7, then you may have found that you get this error:

An invalid PHP type for parameter 2 was specified

1	An invalid PHP type for parameter 2 was specified

(At least, that's what happened to me!)

Working through the problem, I discovered that this is due to Zend_Db_Statement_Sqlsrv converting the $params array to references with this code:

// make all params passed by reference
$params_ = array();
$temp    = array();
$i       = 1;
foreach ($params as $param) {
    $temp[$i]  = $param;
    $params_[] = &$temp[$i];
    $i++;
}
$params = $params_;

// make all params passed by reference

$params_ = array();

$temp = array();

$i = 1;

foreach ($params as $param) {

$temp[$i] = $param;

$params_[] = &$temp[$i];

$i++;

}

$params = $params_;

The Sqlsrv driver (v4) for PHP 7 does not like this!

As Zend Framework 1 is EOL, we can't get a fix into upstream and update the new release, so we have to write our solution.

We want to override Zend_Db_Statement_Sqlsrv::_execute() with our own code. To do this we firstly need to override Zend_Db_Adapter_Sqlsrv. (Also, let's assume we already have a App directory registered with the autoloader)

Firstly our adapter:

App/Db/Adapter/Sqlsrv.php:

<?php

class App_Db_Adapter_Sqlsrv extends Zend_Db_Adapter_Sqlsrv
{
    protected $_defaultStmtClass = 'App_Db_Statement_Sqlsrv';
}

<?php

class App_Db_Adapter_Sqlsrv extends Zend_Db_Adapter_Sqlsrv

{

protected $_defaultStmtClass = 'App_Db_Statement_Sqlsrv';

}

This class simply changes the default statement class to our new one. Now, we can write our Statement class:

App/Db/Statement/Sqlsrv.php:

<?php

class App_Db_Statement_Sqlsrv extends Zend_Db_Statement_Sqlsrv
{
    /**
     * Executes a prepared statement.
     *
     * @param array $params OPTIONAL Values to bind to parameter placeholders.
     * @return bool
     * @throws Zend_Db_Statement_Exception
     */
    public function _execute(array $params = null)
    {
        $connection = $this->_adapter->getConnection();
        if (!$this->_stmt) {
            return false;
        }

        if ($params !== null) {
            if (!is_array($params)) {
                $params = array($params);
            }

            // make all OUT or INOUT params passed by reference
            $params_ = array();
            $temp    = array();
            $i       = 1;
            foreach ($params as $param) {
                $temp[$i]  = $param;
                if (is_array($param) && in_array($param[1], [SQLSRV_PARAM_OUT, SQLSRV_PARAM_INOUT])) {
                    $params_[] = &$temp[$i];
                } else {
                    $params_[] = $temp[$i];
                }
                $i++;
            }
            $params = $params_;
        }

        $this->_stmt = sqlsrv_query($connection, $this->_originalSQL, $params);

        if (!$this->_stmt) {
            require_once 'Zend/Db/Statement/Sqlsrv/Exception.php';
            throw new Zend_Db_Statement_Sqlsrv_Exception(sqlsrv_errors());
        }

        $this->_executed = true;

        return (!$this->_stmt);
    }
}

<?php

class App_Db_Statement_Sqlsrv extends Zend_Db_Statement_Sqlsrv

{

/**

* Executes a prepared statement.

* @param array $params OPTIONAL Values to bind to parameter placeholders.

* @return bool

* @throws Zend_Db_Statement_Exception

public function _execute(array $params = null)

{

$connection = $this->_adapter->getConnection();

if (!$this->_stmt) {

return false;

}

if ($params !== null) {

if (!is_array($params)) {

$params = array($params);

}

// make all OUT or INOUT params passed by reference

$params_ = array();

$temp = array();

$i = 1;

foreach ($params as $param) {

$temp[$i] = $param;

if (is_array($param) && in_array($param[1], [SQLSRV_PARAM_OUT, SQLSRV_PARAM_INOUT])) {

$params_[] = &$temp[$i];

} else {

$params_[] = $temp[$i];

}

$i++;

}

$params = $params_;

}

$this->_stmt = sqlsrv_query($connection, $this->_originalSQL, $params);

if (!$this->_stmt) {

require_once 'Zend/Db/Statement/Sqlsrv/Exception.php';

throw new Zend_Db_Statement_Sqlsrv_Exception(sqlsrv_errors());

}

$this->_executed = true;

return (!$this->_stmt);

}

This class, takes the _execute() method from Zend_Db_Statement_Sqlsrv and makes the necessary changes the section that creates parameter references. Specifically, we only create a reference if the parameter has a direction of SQLSRV_PARAM_OUT or SQLSRV_PARAM_INOUT:

// make all OUT or INOUT params passed by reference
$params_ = array();
$temp    = array();
$i       = 1;
foreach ($params as $param) {
    $temp[$i]  = $param;
    if (is_array($param) && in_array($param[1], [SQLSRV_PARAM_OUT, SQLSRV_PARAM_INOUT])) {
        $params_[] = &$temp[$i];
    } else {
        $params_[] = $temp[$i];
    }
    $i++;
}
$params = $params_;

// make all OUT or INOUT params passed by reference

$params_ = array();

$temp = array();

$i = 1;

foreach ($params as $param) {

$temp[$i] = $param;

if (is_array($param) && in_array($param[1], [SQLSRV_PARAM_OUT, SQLSRV_PARAM_INOUT])) {

$params_[] = &$temp[$i];

} else {

$params_[] = $temp[$i];

}

$i++;

}

$params = $params_;

Finally, we need to register our new adapter with Zend_Application's Database resource. This is done in the config file:

application/configs/application.ini:

resources.db.params.adapterNamespace = "App_Db_Adapter"

1	resources.db.params.adapterNamespace = "App_Db_Adapter"

That's it.

We can now insert binary data into our SQL Server database from PHP 7 using the latest sqlsrv drivers.

Autocomplete Composer script names on the command line

15 May 2017Computing, PHPRob

As I add more and more of my own script targets to my composer.json files, I find that it would be helpful to have tab autocomplete in bash. I asked on Twitter and didn't get an immediate solution and as I had already done something similar for Phing, I rolled up my sleeves and wrote my own.

Start by creating a new bash completion file called composer in the bash_completion.d directory. This file needs executable permission. This directory can usually be found at /etc/bash_completion.d/, but on OS X using Homebrew, it's at /usr/local/etc/bash_completion.d/ (assuming you have already installed with brew install bash-completion).

This is the file:

# Store this file in /etc/bash_completion.d/composer

_composer_scripts() {
    local cur prev
    _get_comp_words_by_ref -n : cur

    COMPREPLY=()
    prev="${COMP_WORDS[COMP_CWORD-1]}"

    #
    #  Complete the arguments to some of the commands.
    #
    if [ "$prev" != "composer" ] ; then
        local opts=$(composer $prev -h --no-ansi | tr -cs '[=-=][:alpha:]_' '[\n*]' | grep '^-')
        COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
        return 0
    fi


    if [[ "$cur" == -* ]]; then
        COMPREPLY=( $( compgen -W '-h -q -v -V -n -d \
            --help --quiet --verbose --version --ansi --no-ansi \
            --no-interaction --profile --working-dir' -- "$cur" ) )
    else
        local scripts=$(composer --no-ansi 2> /dev/null |  awk '/^ +[a-z]+/ { print $1 }')
        COMPREPLY=( $(compgen -W "${scripts}" -- ${cur}) )
    fi

    __ltrim_colon_completions "$cur"
    return 0
}

complete -F _composer_scripts composer

# Store this file in /etc/bash_completion.d/composer

_composer_scripts() {

local cur prev

_get_comp_words_by_ref -n : cur

COMPREPLY=()

prev="${COMP_WORDS[COMP_CWORD-1]}"

# Complete the arguments to some of the commands.

if [ "$prev" != "composer" ] ; then

local opts=$(composer $prev -h --no-ansi | tr -cs '[=-=][:alpha:]_' '[\n*]' | grep '^-')

COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )

return 0

if [[ "$cur" == -* ]]; then

COMPREPLY=( $( compgen -W '-h -q -v -V -n -d \

--help --quiet --verbose --version --ansi --no-ansi \

--no-interaction --profile --working-dir' -- "$cur" ) )

else

local scripts=$(composer --no-ansi 2> /dev/null | awk '/^ +[a-z]+/ { print $1 }')

COMPREPLY=( $(compgen -W "${scripts}" -- ${cur}) )

__ltrim_colon_completions "$cur"

return 0

}

complete -F _composer_scripts composer

(Note that __ltrim_colon_completions is only in recent versions of bash-completion, so you may need to remove this line.)

Reading from the bottom, to get the list of commands to composer, we create a list of words for the -W option to compgen by running composer --no-ansi and then manipulating the output to remove everything that isn't a command using awk. We also create a separate list of flag arguments when the user types a hyphen and then presses tab.

Finally, we also autocomplete flags for any subcommand by running composer {cmd} -h --no-ansi and using tr and grep to limit the list to just words starting with a hyphen.

That's it. Now composer {tab} will autocomplete both built-in composer commands and also custom scripts!

Composer autocomplete

As you can see, in this example, in addition to the built-in commands like dump-autoload and show, you can also see my custom scripts, including apiary-fetch and .

This is very helpful for when my memory fails me!

Switching OpenWhisk environments

19 April 2017OpenWhiskRob

When developing with OpenWhisk, it's useful to use separate environments for working locally or on the cloud for development, staging and production of your application. In OpenWhisk terms, this means setting the host and the API key for your wsk command line application.

$ wsk property set --apihost $host --auth $key

1	$ wsk property set --apihost $host --auth $key

(Of course, for live and staging, ideally, you will be using a build server!)

For a Vagrant install of OpenWhisk, the host is 192.168.33.13 and the key can be found inside the ansible provisioning files. On Bluemix, the host is always openwhisk.ng.bluemix.net and separate environments is most easily done using separate "spaces" as each space has its own key.

To avoid having to keep looking up the correct keys, I wrote a simple Bash function in my .bash_profile file:

function owenv() {
    switches=""
    space=$1
    case "$space" in
        "dev")
            host=openwhisk.ng.bluemix.net
            key="aaa-bbb-ccc-ddd-eee:fff"
            ;;
        "uat")
            host=openwhisk.ng.bluemix.net
            key="bbb-ccc-ddd-eee-fff:ggg"
            ;;
        "live")
            host=openwhisk.ng.bluemix.net
            key="ccc-ddd-eee-fff-ggg:hhh"
            ;;
        "local")
            host=http://192.168.33.13:10001
            key=`cat ~/Projects/openwhisk/openwhisk/ansible/files/auth.guest`
            ;;
        "localssl")
            switches="-i" # insecure switch required as cert is self-signed
            host=192.168.33.13
            key=`cat ~/Projects/openwhisk/openwhisk/ansible/files/auth.guest`
            ;;
        *)
            echo "Error. No space set: choose dev, uat, live, local or localssl"
            return
    esac

    wsk $switches property set --apihost $host --auth $key > /dev/null && \
    wsk $switches property unset --namespace > /dev/null

    # display some information
    bold=$(tput bold)
    normal=$(tput sgr0)
    apihost=`wsk $switches property get --apihost | rev | cut -f1 | rev`
    namespace=`wsk $switches namespace list | tail -n1`
    echo "Host       ${bold}${apihost}${normal}"
    echo "Namespace ${bold}${namespace}${normal}"
}

function owenv() {

switches=""

space=$1

case "$space" in

"dev")

host=openwhisk.ng.bluemix.net

key="aaa-bbb-ccc-ddd-eee:fff"

;;

"uat")

host=openwhisk.ng.bluemix.net

key="bbb-ccc-ddd-eee-fff:ggg"

;;

"live")

host=openwhisk.ng.bluemix.net

key="ccc-ddd-eee-fff-ggg:hhh"

;;

"local")

host=http://192.168.33.13:10001

key=`cat ~/Projects/openwhisk/openwhisk/ansible/files/auth.guest`

;;

"localssl")

switches="-i" # insecure switch required as cert is self-signed

host=192.168.33.13

key=`cat ~/Projects/openwhisk/openwhisk/ansible/files/auth.guest`

;;

echo "Error. No space set: choose dev, uat, live, local or localssl"

return

esac

wsk $switches property set --apihost $host --auth $key > /dev/null && \

wsk $switches property unset --namespace > /dev/null

# display some information

bold=$(tput bold)

normal=$(tput sgr0)

apihost=`wsk $switches property get --apihost | rev | cut -f1 | rev`

namespace=`wsk $switches namespace list | tail -n1`

echo "Host ${bold}${apihost}${normal}"

echo "Namespace ${bold}${namespace}${normal}"

}

(Actual keys redacted!)

This code uses a case statement to set up the right host and key to use. I'm lazy, so just hardcode my Bluemix keys. There's probably a better way to that though. For the local Vagrant instance, I get the key directly from the file used by Ansible for provisioning. Again, due to laziness, I've hardcoded the Vagrant VM's IP address.

Lastly I set display the current host and namespace – tput is my new favourite bash command!

Switching environments

I can then use the function to switch to my Vagrant installation like this:

$ owenv local
Host      http://192.168.33.13:10001
Namespace guest

$ owenv local

Host http://192.168.33.13:10001

Namespace guest

Or, I can switch to my Cloud development environment using:

$ owenv dev
Host      openwhisk.ng.bluemix.net
Namespace 19FT_dev

$ owenv dev

Host openwhisk.ng.bluemix.net

Namespace 19FT_dev

Updating the CLI tool

I also have a script to update the CLI tool:

function owupdatewsk() {
    echo "Current version: `wsk property get --cliversion | rev | cut -f1 | rev`"
    pushd /tmp > /dev/null
    curl -s -o OpenWhisk_CLI-mac.zip https://openwhisk.ng.bluemix.net/cli/go/download/mac/amd64/OpenWhisk_CLI-mac.zip && \
    unzip OpenWhisk_CLI-mac.zip > /dev/null && \
    mv wsk /usr/local/bin/wsk && \
    rm OpenWhisk_CLI-mac.zip && \
    popd > /dev/null  && \
    echo "New version:     `wsk property get --cliversion | rev | cut -f1 | rev`"
}

function owupdatewsk() {

echo "Current version: `wsk property get --cliversion | rev | cut -f1 | rev`"

pushd /tmp > /dev/null

curl -s -o OpenWhisk_CLI-mac.zip https://openwhisk.ng.bluemix.net/cli/go/download/mac/amd64/OpenWhisk_CLI-mac.zip && \

unzip OpenWhisk_CLI-mac.zip > /dev/null && \

mv wsk /usr/local/bin/wsk && \

rm OpenWhisk_CLI-mac.zip && \

popd > /dev/null && \

echo "New version: `wsk property get --cliversion | rev | cut -f1 | rev`"

}

This is a quick way to collect the latest version of the wsk app.

POSTing data using KituraNet

10 April 2017OpenWhisk, SwiftRob

I had a need to send a POST request with a JSON body from Swift and as I had KituraNet and SwiftyJSON already around, it proved to reasonably easy.

To send a POST request using KituraNet, I wrote this code:

let url = "http://httpbin.org/post". 	 // Change to the actual URL
let dataToSend = ["foo": ["bar": "baz"]] // Change to the actual data

// body needs to be of type Data, use SwiftyJSON to convert
let body = try! JSON(dataToSend).rawData()

// HTTP.request() takes a callback that is called when the response is 
// ready. Set up result to extract the data from the callback
var result: [String:Any] = ["result": "No response"];


// create the request
var options: [ClientRequest.Options] = [
    .schema(""),
    .method("POST"),
    .hostname(url),
    .headers(["Content-Type": "application/json"])
]

let request = HTTP.request(options) { response in
    do {
        let str = try response!.readString()!
        if let jsonDictionary = JSON.parse(string: str).dictionaryObject {
            result = jsonDictionary // we got back JSON
        } else {
            result = ["error": "Unknown data format returned"]
        }
    } catch {
        print("Error \(error)") // error handling here
    }
}
request.write(from: body) // add body data to request
request.end() // send request

// result now has a dictionary with either an "error" key or the data returned
// by the server

let url = "http://httpbin.org/post". // Change to the actual URL

let dataToSend = ["foo": ["bar": "baz"]] // Change to the actual data

// body needs to be of type Data, use SwiftyJSON to convert

let body = try! JSON(dataToSend).rawData()

// HTTP.request() takes a callback that is called when the response is

// ready. Set up result to extract the data from the callback

var result: [String:Any] = ["result": "No response"];

// create the request

var options: [ClientRequest.Options] = [

.schema(""),

.method("POST"),

.hostname(url),

.headers(["Content-Type": "application/json"])

]

let request = HTTP.request(options) { response in

do {

let str = try response!.readString()!

if let jsonDictionary = JSON.parse(string: str).dictionaryObject {

result = jsonDictionary // we got back JSON

} else {

result = ["error": "Unknown data format returned"]

}

} catch {

print("Error \(error)") // error handling here

}

request.write(from: body) // add body data to request

request.end() // send request

// result now has a dictionary with either an "error" key or the data returned

// by the server

As you can see, I've liberally commented it, so it should be easy to follow. Let's look at some interesting bits.

SwiftJSON is convenient

SwiftyJSON does all the heavy lifting of converting dictionaries. As KituraNet requires a Data object for the body, we can do this in one line with SwiftyJSON:

let body = try! JSON(dataToSend).rawData()

1	let body = try! JSON(dataToSend).rawData()

(admittedly, this assumes a valid dictionary! In a real app, consider better error checking…)

Similarly, if we get a JSON string back from the server, converting it to a dictionary is as easy as:

if let jsonDictionary = JSON.parse(string: str).dictionaryObject {
    result = jsonDictionary // we got back JSON
}

if let jsonDictionary = JSON.parse(string: str).dictionaryObject {

result = jsonDictionary // we got back JSON

}

(This time, with some checking!)

Fin

Making a JSON-based POST request is easy enough with KituraNet and SwiftyJSON. Of course, the reason I chose this approach is that they are baked into OpenWhisk which is where this code is running.

I also refactored this code into the methods postTo() and postJsonTo() as you can see in this gist.

Using CircleCI for a PHP project

3 April 2017PHPRob

For a new client project, I've decided to use CircleCI to run my tests every time I push to GitHub.

This turned out to be quite easy; this is how I did it.

I started by creating a config file, .circleci/config.yml containing the following:

version: 2
jobs:
  build:
    working_directory: /var/www/html
    docker:
      - image: php:7.1-apache
        environment:
          APP_ENV: test
    steps:
      - run:
          name: Install system packages
          command: apt-get update && apt-get -y install git
      - run:
          name: Install PHP extensions
          command: docker-php-ext-install pdo
      - checkout
      - run:
          name: Install Composer
          command: |
            php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
            php -r "if (hash_file('SHA384', 'composer-setup.php') === trim(file_get_contents('https://composer.github.io/installer.sig'))) { echo 'Installer verified'; } else { echo 'Installer invalid'; unlink('composer-setup.php'); } echo PHP_EOL;"
            php composer-setup.php
            php -r "unlink('composer-setup.php');"
      - run:
          name: Display PHP information
          command: |
            php -v
            php composer.phar --version
      - run:
          name: Install project dependencies
          command: php composer.phar install
      - run:
          name: Run CS tests
          command: vendor/bin/phpcs
      - run:
          name: Run Unit tests
          command: vendor/bin/phpunit

version: 2

jobs:

build:

working_directory: /var/www/html

docker:

- image: php:7.1-apache

environment:

APP_ENV: test

steps:

- run:

command: apt-get update && apt-get -y install git

- run:

command: docker-php-ext-install pdo

- checkout

- run:

command: |

php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"

php -r "if (hash_file('SHA384', 'composer-setup.php') === trim(file_get_contents('https://composer.github.io/installer.sig'))) { echo 'Installer verified'; } else { echo 'Installer invalid'; unlink('composer-setup.php'); } echo PHP_EOL;"

php composer-setup.php

php -r "unlink('composer-setup.php');"

- run:

command: |

php -v

php composer.phar --version

- run:

command: php composer.phar install

- run:

command: vendor/bin/phpcs

- run:

command: vendor/bin/phpunit

The documentation is really good, but the file's organisation is pretty self-expanatory.

The config file has a list of jobs. The build job is run on a push to GitHub, so that's the one I've created. Inside the docker section contains a list of docker images that are required – in my case, I just need a single container running PHP 7.1. The other section in the job is the list of steps to run. Each step has a name and a command which is a bash command.

For my job, I need to install git and the PDO PHP extension. I then run the "magic" step called checkout which as per its name, checks out my source code. I then install composer, including verifying that it's valid and display the PHP version number and composer version number in case I ever need them to work out why a build failed.

I then turn my attention to the project itself and run composer install and then the tests themselves: phpcs and phpunit.

Running the build

To actually get builds to run, you log into CircleCI – usefully you authenticate via GitHub – and select the project to build. From now on, pushing to GitHub or a branch will result in the build running. On success, you get something like this:

Circle ci

Hooking up to Slack

Hooking up to Slack is done by going to Slack's Apps & Integrations section and searching for CircleCI. Add the configuration and follow the wizard. Once done, you get a URL that you add to the project's Chat Notifications settings in CircleCI. This is found by clicking on the "cog" next to the project's name in the builds list. There's also a setting callled "Fixed/Failed only" which I check.

Once that's done, you get Slack notifications on failures and then on the first success after a failure and you are now secure in the knowledge that your tests are being run reliably.

Automatically converting PDF to Keynote

30 March 2017ComputingRob

I use rst2pdf to create presentations which provides me with a PDF file. When it comes to presenting on stage, on Linux there are tools such as pdfpc and on Mac there's Keynote.

Keynote doesn't read PDF files by default, so we have to convert them and the tool I use for this is Melissa O'Neill's PDF to Keynote. This is a GUI tool, so I manually create the Keynote file when I need it which is tedious. Recently, with Melissa's prompting, I realised that I could automate the creation of the keynote file which makes life easier!

I use a Makefile for this and this is the target & relevant variables:

# Name of PDF to create
pdf = my-presentation.pdf

# Aspect ratio of Keynote file
aspect_ratio = 1280 x 720

# Determine the name of the Keynote file
key = $(pdf:%.pdf=%.key)


# "make keynote": creates the Keynote file using "PDF to Keynote"
keynote: $(pdf)
	rm -f $(key)
	defaults write net.clawpaws.PDFtoKeynote presentationSize "$(aspect_ratio)"
	defaults write net.clawpaws.PDFtoKeynote autoSaveAfterOpen 1
	defaults write net.clawpaws.PDFtoKeynote autoOpenAfterSave 0
	defaults write net.clawpaws.PDFtoKeynote autoCloseAfterSave 1
	open -a /Applications/PDF\ to\ Keynote.app/Contents/MacOS/PDF\ to\ Keynote "$(pdf)"
	osascript -e 'tell application "PDF to Keynote" to quit'

# Name of PDF to create

pdf = my-presentation.pdf

# Aspect ratio of Keynote file

aspect_ratio = 1280 x 720

# Determine the name of the Keynote file

key = $(pdf:%.pdf=%.key)

# "make keynote": creates the Keynote file using "PDF to Keynote"

keynote: $(pdf)

rm -f $(key)

defaults write net.clawpaws.PDFtoKeynote presentationSize "$(aspect_ratio)"

defaults write net.clawpaws.PDFtoKeynote autoSaveAfterOpen 1

defaults write net.clawpaws.PDFtoKeynote autoOpenAfterSave 0

defaults write net.clawpaws.PDFtoKeynote autoCloseAfterSave 1

open -a /Applications/PDF\ to\ Keynote.app/Contents/MacOS/PDF\ to\ Keynote "$(pdf)"

osascript -e 'tell application "PDF to Keynote" to quit'

The nice thing about PDF to Keynote is that it has preferences to automatically create the Keynote file after a PDF file opened and to automatically close the PDF file once saved. We can also programmatically set the aspect ratio. To do this, we use the defaults command line tool to set up PDF to Keynote the way that we want.

We then call open -a to open PDF to Keynote with the PDF file as the argument which then automatically creates the Keynote file and stores it into the same directory. The PDF file is automatically closed for us too.

Finally, we can use AppleScript via the osacript command to quite PDF to Keynote. I'm not sure if we need to wait for the conversion to happen before we quit, in which case, we can add sleep 3 if we need to.

That's it. Automatically creating the Keynote file vastly improves my workflow and I no longer have to think so much about it.

Update: Note that I changed the default write commands for the booleans as they need to be 1 and 0, not "YES" and "NO"…

Detecting OpenWhisk web actions

27 March 2017OpenWhisk, SwiftRob

I've already written about OpenWhisk web actions and how they allow you to send a status code and HTTP headers to the client by returning a dictionary with the keys status, headers and body from your main() method:

func main(args: [String:Any]) -> [String:Any] {
    return [
        "body": "<root>Hello world</root>",
        "code": 200,
        "headers": [
            "Content-Type": "text/xml",
        ],
    ]
}

func main(args: [String:Any]) -> [String:Any] {

return [

"body": "<root>Hello world</root>",

"code": 200,

"headers": [

"Content-Type": "text/xml",

]

}

If this test action is in the default namespace, then we create it with wsk action update test test.swift -a web-export true to enable web action support and access it via curl:

curl https://openwhisk.ng.bluemix.net/api/v1/experimental/web/19FT_dev/default/test.http

<root>Hello world</root>

curl https://openwhisk.ng.bluemix.net/api/v1/experimental/web/19FT_dev/default/test.http

<root>Hello world</root>

However, when you invoke this via the authenticated POST API (eg. Via curl or wsk action invoke) you get this:

$ AUTH=$(wsk property get --auth | awk '{printf("%s", $3)}' | openssl base64 | tr -d "\n")
$ curl -X POST -H "Authorization: Basic $AUTH" \
"https://openwhisk.ng.bluemix.net/api/v1/namespaces/19FT_dev/actions/test?blocking=true&result=true"

{
  "body": "<root>Hello world</root>",
  "code": 200,
  "headers": {
    "Content-Type": "text/xml"
  }
}

$ AUTH=$(wsk property get --auth | awk '{printf("%s", $3)}' | openssl base64 | tr -d "\n")

$ curl -X POST -H "Authorization: Basic $AUTH" \

"https://openwhisk.ng.bluemix.net/api/v1/namespaces/19FT_dev/actions/test?blocking=true&result=true"

{

"body": "<root>Hello world</root>",

"code": 200,

"headers": {

"Content-Type": "text/xml"

}

This could have been predicted as the authenticated POST API call just executes the action and sends back what it returned.

Additional arguments in a web action

When your action is called as a web action, then there are additional arguments, that don't appear otherwise. We can simply look for one of these. Specifically, I chose to look for __ow_meta_verb.

The simple way of doing this:

func main(args: [String:Any]) -> [String:Any] {

    if args["__ow_meta_verb"] == nil {
        return ["root": "Hello world"]
    }
    return [
        "body": "<root>Hello world</root>",
        "code": 200,
        "headers": [
            "Content-Type": "text/xml",
        ],
    ]
}

func main(args: [String:Any]) -> [String:Any] {

if args["__ow_meta_verb"] == nil {

return ["root": "Hello world"]

}

return [

"body": "<root>Hello world</root>",

"code": 200,

"headers": [

"Content-Type": "text/xml",

]

}

Note that we return a dictionary as an authenticated POST API call expects this. Calling internally via curl:

$ curl -X POST -H "Authorization: Basic $AUTH" \
"https://openwhisk.ng.bluemix.net/api/v1/namespaces/19FT_dev/actions/test?blocking=true&result=true"

{
  "root": "Hello world"
}

$ curl -X POST -H "Authorization: Basic $AUTH" \

"https://openwhisk.ng.bluemix.net/api/v1/namespaces/19FT_dev/actions/test?blocking=true&result=true"

{

"root": "Hello world"

}

(We can only get JSON back this way)

and of course calling the web action hasn't changed and we still get our XML.

We can call our function with whatever mechanism is appropriate and generate the right response.

Rob Allen's DevNotes

Developing software in the Real World

All posts by Rob

Simple way to add a filter to Zend-InputFilter

Extend Zend\InputFilter\Factory

Update your container's factory

Default route arguments in Slim

Slim's route cache file

Inserting binary data into SQL Server with ZF1 & PHP 7

Autocomplete Composer script names on the command line

Switching OpenWhisk environments

Switching environments

Updating the CLI tool

POSTing data using KituraNet

SwiftJSON is convenient

Fin

Using CircleCI for a PHP project

Running the build

Hooking up to Slack

Automatically converting PDF to Keynote

Detecting OpenWhisk web actions

Additional arguments in a web action