Registering Doctrine Type Mappings for standalone migrations

Shortly after starting to use Doctrine Migrations as a standalone tool in my project, I came across this error message:

Unknown database type bit requested, Doctrine\DBAL\Platforms\MySqlPlatform may not support it.

This means that I have a column in my database of type bit which is used for booleans in SQL Server, but confuses the MySQL platform as it's not a default mapping.

To support this, you need to modify the database connection's Platform object to know about the new mapping. However, with the setup that I'm using, I didn't have access to the connection object that's automatically created in the Migrations AbstractCommand object.

After poking around in the code for a bit, I discovered that the solution is to create the connection object myself and then attach it as a new helper to the Console\Application object.

This is how to do to it within the migrations.php file.

Firstly, create the database connection object using the information in migrations-db.php. This code is lifted from AbstractCommand:

if (file_exists('migrations-db.php')) {
    $params = include 'migrations-db.php';
    if (!is_array($params)) {
        throw new \InvalidArgumentException('The connection file has to return an array.');
    }
    $connection = \Doctrine\DBAL\DriverManager::getConnection($params);
} else {
    throw new \InvalidArgumentException('Missing "migrations-db.php" file. Alternatively use --db-configuration.');
}

We can now set up the database type mapping for bit:

$platform = $connection->getDatabasePlatform();
$platform->registerDoctrineTypeMapping('bit', 'boolean');

Now, all we need to do is add the connection to the Console\Application's HelperSet so that the AbstractCommand can use it. We are already adding the DialogHelper, so we add a ConnectionHelper

use Doctrine\DBAL\Tools\Console\Helper\ConnectionHelper;

$helperSet = new Console\Helper\HelperSet();
$helperSet->set(new Console\Helper\DialogHelper(), 'dialog');
$helperSet->set(new ConnectionHelper($connection), 'connection');
$cli->setHelperSet($helperSet);

That's it. Our migrations system know nows that a database column of type bit is a boolean on MySQL as well as on SQL Server.

Routing specific traffic to the VPN on OS X

I have a client that requires me to use a VPN when connecting to their servers. I use OS X's built in L2TP VPN to connect, but don't want all my traffic going that way.

To do this, I unchecked the Advanced VPN setting "Send all traffic over VPN connection" in the Network preferences and then created the file /etc/ppp/ip-up like this:

sudo touch /etc/ppp/ip-up
sudo chmod 755 /etc/ppp/ip-up

The file itself is a bash script that runs various /sbin/route commands and looks similar to this:

/etc/ppp/ip-up:

#!/bin/sh
/sbin/route add -host 192.168.1.0/16 -interface ppp0

Now, whenever I connect to the VPN, only traffic for hosts on 192.168.1.x is sent to the client's VPN and we're both happy.

Using Doctrine Migrations as a standalone tool

My current project has reached the point where a good migrations system is required. As I'm targeting two different database engines (MySQL and MS SQL Server) and we're already using DBAL, it made sense to use Migrations from the Doctrine project.

To do this, I updated my composer.json with the following require statements:

"doctrine/migrations": "1.0.*@dev",
"symfony/console": "~2.5"

in order to install Migrations and also Symfony console component so that I can run it from the command line.

The CLI script

Migrations doesn't come with it's own CLI script, so you have to create your own. Fortunately, we can tweak the supplied phar-cli-stub.php. I placed it in bin/migrations.php:

/**
 * Command line script to run Migrations
 * Inspired by phar-cli-stup.php
 */

use Symfony\Component\Console;
use Doctrine\DBAL\Migrations\MigrationsVersion;
use Doctrine\DBAL\Migrations\Tools\Console\Command as MigrationsCommand;

require (__DIR__ . '/../vendor/autoload.php');

// Set current directory to application root so we can find config files
chdir(__DIR__ . '/..');

// Instantiate console application
$cli = new Console\Application('Doctrine Migrations', MigrationsVersion::VERSION);
$cli->setCatchExceptions(true);

$helperSet = new Console\Helper\HelperSet();
$helperSet->set(new Console\Helper\DialogHelper(), 'dialog');
$cli->setHelperSet($helperSet);

// Add Migrations commands
$commands = array();
$commands[] = new MigrationsCommand\ExecuteCommand();
$commands[] = new MigrationsCommand\GenerateCommand();
$commands[] = new MigrationsCommand\LatestCommand();
$commands[] = new MigrationsCommand\MigrateCommand();
$commands[] = new MigrationsCommand\StatusCommand();
$commands[] = new MigrationsCommand\VersionCommand();

// remove the "migrations:" prefix on each command name
foreach ($commands as $command) {
    $command->setName(str_replace('migrations:', '', $command->getName()));
}
$cli->addCommands($commands);

// Run!
$cli->run();

Note that, by default, the console commands are all prefixed with 'migrations:'. This makes sense as usually they are integrated with the rest of the Doctrine ORM command line application and so it avoids any clashes. In this case, this script is only dealing with migrations and I don't want to type more than I have to, so I remove the prefixes!

Configuration

Migrations requires two separate configuration files to work: migrations.yml and migrations-db.php. These need to be in the current directory, so I used chdir() to ensure that it is in a known location – the application root in this case.

migrations.yml is the config file that tells Migrations the names of things:

migrations.yml:

name: DBAL Migrations
migrations_namespace: Migrations
table_name: migration_versions
migrations_directory: migrations

In my case, I want to use my migrations files to be in a directory called migrations and use the namespace Migrations. The database table name that Migrations uses to keep track of its internal status is called migration_versions. I'm not too imaginative when it comes to naming things!

migrations-db.php must return an array that is used to configure Doctrine\DBAL\DriverManager::getConnection()

migrations-db.php:

<?php
return array(
        'dbname'   => 'albums',
        'user'     => 'rob',
        'password' => '123456',
        'host'     => 'localhost',
        'driver'   => 'pdo_mysql',
);

The documentation will help you find out what can go in here.

We're now ready to go!

On the command line, php bin/migrations.php status should work:

$ php bin/migrations.php 
Doctrine Migrations version 2.0.0-DEV

Usage:
  [options] command [arguments]

Options:
  --help           -h Display this help message.
  --quiet          -q Do not output any message.
  --verbose        -v|vv|vvv Increase the verbosity of messages
  --version        -V Display this application version.
  --ansi              Force ANSI output.
  --no-ansi           Disable ANSI output.
  --no-interaction -n Do not ask any interactive question.

Available commands:
  execute    Execute a single migration version up or down manually.
  generate   Generate a blank migration class.
  help       Displays help for a command
  latest     Outputs the latest version number
  list       Lists commands
  migrate    Execute a migration to a specified version or the latest available version.
  status     View the status of a set of migrations.
  version    Manually add and delete migration versions from the version table.

Creating a migration

run php bin/migrations.php generate and a new migration class will be created in the migrations directory. It has two methods: up() and down(). Both have an instance of Doctrine\DBAL\Schema\Schema passed and an you can do whatever you like in the function to manipulate your database to its next state. Again, the documentation is helpful!

For instance:

<?php

namespace Migrations;

use Doctrine\DBAL\Migrations\AbstractMigration;
use Doctrine\DBAL\Schema\Schema;

class Version20141027161210 extends AbstractMigration
{
    public function up(Schema $schema)
    {
        $myTable = $schema->createTable('artists');
        $myTable->addColumn('id', 'integer',
            ['unsigned' => true, 'autoincrement'=>true]);
        $myTable->addColumn('name', 'string', ['length' => 60]);
        $myTable->setPrimaryKey(['id']);
    }

    public function down(Schema $schema)
    {
        $schema->dropTable('artists');
    }
}

Running the migration

Simply run php bin/migrations.php migrate and Migrations will the up() method for all migration classes that haven't be run yet. In this case it will create a table called artists and update the migration_versions table with the version number of '20141027161210'.

To back out of a migration, simple run php bin/migrations.php migration {version number} and Migrations will run the appropriate down() (or up()) methods to get to that version number.

Finally you can run php bin/migrations.php status to find out the current state.

On the whole, it turns out that it's quite easy to use Migrations as a stand-alone tool outside of Doctrine ORM or Symfony!

Two incidents

An online friend of mine once said this about a sexism incident at a conference:

"If I had been there, ZERO chance I would've sat idly by and let someone treat you like that"

I like to believe that I have the same policy: If I am there when some casual sexism occurs, I will call it out.

It turns out that this is untrue.

One

I have witnessed a number of incidents of sexism at conferences that I've been to recently and I want to talk about one particular situation that occurred.

Around lunchtime at a recent conference, I was with a group of developer friends that included one woman. A man joined our group and made a joke to the woman along the lines of asking if she was here with her boyfriend. I understand that they are friends and that he was being ironic and humorous and certainly meant no personal insult. She responded with a cutting remark and that was that.

Except that wasn't that.

I did not speak out.

It was just a joke. It worked by demeaning women in my industry. I should have said something, especially as I was in the company of friends. The fact that I didn't speak up is unacceptable because jokes like this are actively harmful by contributing to the background tolerance of discrimination against women.

I am ashamed that I didn't speak up. I let my friends and myself down and and I apologise to everyone in our community for not doing my bit when I had the chance.

Two

On Saturday, I took my son to one of his extracurricular activity sessions. Afterwards, my son and I were talking with his coach about the rest of our weekend and how I'd be helping him with the electrical bits of building his new Tamiya Hornet remote controlled car. The coach commented that he would like to do the same sort of things with his future grandchildren and hoped that he wouldn't get a granddaughter.

Maybe it was because my son was there and I'm trying to help teach him about these things, or maybe it was because of the incidents that I'd witnessed recently at conferences, but this time, I spoke up.

I talked about my friend who took an Arduino and some coloured LEDs to create a fuzzy clock where the colour of LEDs indicates the time to the nearest hour. I'm fully intending to copy her project as it would be especially useful to me at night when I'm not wearing my glasses. We then talked a little about how a granddaughter could certainly be interested in building and running remote control cars!

Fin

I want to help reduce sexism in this world. More than one recent conference incident has reminded me that I have a long long way to go, but maybe there's some hope for me yet.

Even though I regularly fail, sometimes I succeed and I won't let my failures stop me from getting it right next time.

Why I care about codes of conduct

tl;dr

I want every conference to enthusiastically champion their code of conduct in order to publicly reassure attendees (women in particular) that any issues will be dealt with. I want all of us to actually notice jokes and conduct that make conferences uncomfortable for women and call out or report issues if we see them. I'm still learning how to do this, but the rest of this post goes into detail on why I think this is subject is important.

The detail

NYCC's Anti-Harassment Policy

Following up on my last post about codes of conducts, I was recently asked why I'm interested in them. While I responded directly, it crosses my mind that I should probably write down my thoughts on this and then I can practice good DRY practices and just link back here!

I am becoming increasingly concerned about the lack of women and non-white people I see at the conferences I go to. It was particularly noticeable at one conference I went to fairly recently and so I’ve been asking why this is. Many of people who inspire me are women, but I hardly see any women at conferences. So, I've started with the issue of the lack of women at conferences and in development in general.

Conferences are one of the best ways I know to become a better developer. Not only do you get exposed to new technologies and techniques via the talks, you get to meet other developers & make contacts that last beyond the event.

We know from US statistics that around 20% of all developers are women. Looking at the male to female ratios at most of the conferences that I go to, I don't even see that many women developers there. I'm assuming that they are choosing not to attend.

That's sad.

So I've started reading. It's hard to read without judging based on your own experiences. I’ve also talked to my wife who has gone to events in a different space and other women friends in a variety of industries and it has been eye-opening.

From my learning so far there seemed to be two main topics that kept coming up: atmosphere and safety.

One major concern I have is that a woman’s initial base assumption seems to be that she expects that something “uncomfortable” is to going happen to them over the course of an event. This runs the full range of things from physical things like being touched “by accident”, through having someone stand very close in her personal space and monopolise her to the point that she is “trapped” talking to this person to men staring at her breasts when talking to her. There's also the words she will hear – from casual comments about women belonging in the kitchen, to overhearing jokes about how bad women are at logical thinking all the way to being asked if she has a boyfriend and if not, being propositioned.

As a man, it’s taken me ages to understand why a casual comment to a woman about whether she has a boyfriend is so damaging. It seemed harmless to me. Just a joke. However, no one ever talks to me a conference with an underlying subtext that they are mainly interested in having sex with me rather than geeking out on tech and programming. No one ever questions my authority in my subject area. Even now, I have been places where I hear conversations between one of my female friends and another man and they tell me afterwards that the man clearly didn't think she was competent by his choice of words and demeanour and I didn’t even notice :(

Hence, I can completely understand why a woman would rather not go to a conference than have to deal with this underlying unspoken assumption that she isn't good enough to be a programmer. Don’t even get me started on the message that's sent when you see pictures of women in lingerie or short skirts in slide decks and sponsor advertising material. I’ve seen all these things at conferences before.

To my wife, her personal safety is very much an active thought in her mind when she meets any man. It's her default position. Apparently this is as much of a worry in a public social setting as on a lonely street at night; maybe more so. By safety, I don’t mean violent rape. I mean being hugged when you don’t want to be; being touched "by accident"; being cornered by someone. These things don't just happen at evening socials, they also happen in the coffee break. These things are happening in public places and and no one else in the room notices.

How are we supposed to stop things that we don’t even see from happening?

The logical thing to me is that if any woman experiences anything she is uncomfortable with (or a man sees something happening to a woman), they should feel able to report it in private and be 100% sure that they will be believed and looked after, no matter how minor the organisers may perceive this incident to be.

I don't think I can emphasise enough that so many incidents appear to be minor and "just the way things are" to men. I'm finding this really difficult to internalise and I'm, notionally at least, aware of the issue. "Just a joke", "That's just how Bob is", "That's not against the law", "It's just a pat on the bottom" are all ways to enable us to believe that there's nothing in our community that needs improving.

Coming back to the subject at hand, I think that a published code of conduct is a clear way to say to the attendees that the organisers know that these things can happen and that (a) they are publicly saying that they are unacceptable and that (b) they are signalling that they have a plan, already in place, for dealing with any incident that may happen.

I appreciate that other people don't agree with me and think that their conference doesn't need a code of conduct because they work towards improving diversity in other ways.

I disagree.

Personally, I see a code of conduct that is championed by the organisers as a great way to improve the inclusivity of an event which can only make for a better conference. I want to see them at every conference.

I was impressed to see that NYCC put their code of conduct on billboards throughout their conference! This is a commercial venture, so certainly they thought about the pros and cons of doing it.

However, the best way we can make conferences more welcoming to women is if we, the attendees, call out our fellow attendees on what they say and what they do that's unwelcoming. It takes courage to report a joke that isn't suitable for a professional conference, but if we all do it, it becomes the norm.

Further reading

If you are interested in some further reading, on this topic then I can recommend Codes of Conduct 101 + FAQ, Sexual harassment at technical conferences: A big no-no and Your conference needs an anti-harassment policy.

For some real-world experiences, try My first OSCON, This is why we can't have nice things, The Creepy Librarian Stalker Hypothesis & Please do not pat me on my head. There are many many more examples out there; try asking any woman who's been to a conference as I haven't found one yet that did not have a story to tell.

Finally, the discussions on this reddit thread and on BoingBoing about the very visible NYCC banners are interesting too.

Provisioning with Ansible within the Vagrant guest

I've been setting up a Vagrant VM for use with some client projects and picked Ansible to do this. Firstly, I played with the Ansible provisioner, but found it a little slow and then I realised that Ansible doesn't run on Windows.

Rather than migrate what I'd done to Puppet, Evan recommended that I look into running Ansible on the guest instead and provided some hints. This turned out to be quite easy.

These are the steps when starting from the ubuntu/trusty64 base box.

Use the Shell provisioner

As we're running Ansible on the guest, we use the shell provisioner, so my Vagrantfile contains:

config.vm.provision :shell,
  :keep_color => true,
  :inline => "export PYTHONUNBUFFERED=1 && export ANSIBLE_FORCE_COLOR=1 && cd /vagrant/provisioning && ./init.sh"

This simply tells Vagrant to run init.sh which is stored in the provisioning directory of my project.

I immediately noticed a warning when running vagrant up: "stdin: is not a tty error". This is due to the way Ubuntu tries to echo a message to a shell that isn't interactive. To get rid of this, we need to configure Vagrant's ssh shell to be a non-login one in the Vagrantfile:

config.ssh.shell = "bash -c 'BASH_ENV=/etc/profile exec bash'"

init.sh

Our shell provisioner needs to do two things:

  1. Install Ansible
  2. Run our playbook

So, init.sh looks like this:

#!/usr/bin/env bash

if [ $(dpkg-query -W -f='${Status}' ansible 2>/dev/null | grep -c "ok installed") -eq 0 ];
then
    echo "Add APT repositories"
    export DEBIAN_FRONTEND=noninteractive
    apt-get install -qq software-properties-common &> /dev/null || exit 1
    apt-add-repository ppa:ansible/ansible &> /dev/null || exit 1

    apt-get update -qq

    echo "Installing Ansible"
    apt-get install -qq ansible &> /dev/null || exit 1
    echo "Ansible installed"
fi

cd /vagrant/provisioning
ansible-playbook setup.yml --connection=local

Obviously, we only want to install Ansible once, so we check the output of dpkg-query and only install if it's not already installed. Installation is easy enough: we just use apt-get to add install what we need (quietly!) from the ansible ppa as per the docs.

Once Ansible is installed, we can run ansible-playbook with the --connnection-local switch to run out playbook, setup.yml in my case.

At this point, it's all standard Ansible all the way for your provisioning. It's just faster and works with Windows.

substr_in_array

No matter what I want to do with an array, PHP usually has a first class method that does it. I was therefore surprised that in_array() didn't handle substring matches. (I was sure there was a flag, but apparently not!)

No doubt everyone has their own version of this, but here's mine so that I don't have to recreate it next time:

/**
 * A version of in_array() that does a sub string match on $needle
 *
 * @param  mixed   $needle    The searched value
 * @param  array   $haystack  The array to search in
 * @return boolean
 */
function substr_in_array($needle, array $haystack)
{
    $filtered = array_filter($haystack, function ($item) use ($needle) {
        return false !== strpos($item, $needle);
    });

    return !empty($filtered);
}

Is there a better way of doing this?

Setting up PHP & MySQL on OS X Yosemite

It's that time again; Apple has shipped a new version of OS X, 10.10 Yosemite. Apple ships PHP 5.5.14 with Yosemite and this is how to set it up from a clean install.

However, if you don't want to use the built-in PHP or want to use version 5.6, then these are some alternatives:

Let's get started… Continue reading

Git push to multiple repositories

I have a couple of projects where I need to push to more than one repo all the time.

I have been using this command line to do so:

git push origin && git push other-remote

However, I recently discovered that I can create a remote that points to more than one repository using these commands:

git remote add all git@github.com:akrabat/projectname.git
git remote set-url --add all ssh://example.com/path/to/projectname.git

I now have a remote called all that will push to both repositories!

There's no automatic way to go the other way and fetch from multiple repositories though as apparently it makes less sense to fetch the same branch identifier from multiple places.

Further details in this email by Linus in 2006.

If you want to script the creation of the all remote, then you could use this script which manipulates the remote configuration settings directly:

#!/bin/bash

if [ "`git remote| grep all`" == "all" ] ; then
    git remote remove all
fi

for r in `git remote`
do
    git config --add remote.all.url `git config remote.$r.url`
done

Create this as /usr/local/bin/git-add-push-all.sh and then you can just run it in the root of your project.

Kim

Today is Ada Lovelace day which celebrates the achievements of women in science, technology, engineering and maths. I have many role models in my technical life and many are women who inspire and encourage me to do better. There are too many to list, but I am thankful every one of them, both men and women.

I want to talk today about one person who inspires me: Kim. Kim is relatively new to development. Fortunately she works for an organisation that recognised her abilities and enabled her to to transition to a new position where she could work professionally as a developer for them. I've watched her grow from someone who knew very little to someone I now ask questions of.

She has three things going for her:

  • enthusiasm
  • eagerness to learn
  • willingness to ask questions

I feel these are her key strengths when it comes to growing so quickly into a competent developer. They work very well together.

Firstly, Kim is so enthusiastic to everything that she puts her mind to. It doesn't matter if it's updating HTML to ensure that information is displayed clearly for users, writing PHP or even testing Android code, Kim is excited to be doing the work.

Tied with this enthusiasm is an eagerness to learn. Just because she hasn't done it before doesn't seem to be a massive impediment. Just point her in the right direction and she'll go off and research the topic area and then comes back asking questions. The key thing here is that she has educated herself before she asks questions. As a result, the questions are nuanced, specific and answerable. If only everyone asked questions this way!

What do I want to learn from Kim? Most obviously, I have become lazy. I tend to ask questions first, so others do the "hard work" of research for me. This is disrespectful of their time and I am perfectly capable of doing enough research so then I can ask the specific questions to the people who freely give of their time to help me.