Category Archives: Software

installing XHGui via Ansible

I'm still using Ansible to provision Vagrant VMs. This is how I added the XHGui profiler to my standard setup.

Theres a number steps we need to do:

  • Install Composer
  • Install the uprofiler PHP extension
  • Install XHGui
  • Set up for profiling
  • Set up host for XHGui website

Install Composer

Installing Composer requires these tasks:

- name: Install Composer
  shell: curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin creates=/usr/local/bin/composer

- name: Rename composer.phar to composer
  shell: mv /usr/local/bin/composer.phar /usr/local/bin/composer creates=/usr/local/bin/composer

- name: Make composer executable
  file: path=/usr/local/bin/composer mode=a+x state=file

- name: Create global composer directory
  file: path=/usr/local/composer state=directory mode=0775

Firstly we download the Composer installer and run it to create composer.phar. We then rename to composer, make executable and then create a global directory for storing the packages that we download.

Install the uprofiler PHP extension

We install uprofiler via composer:

- name: Install uprofiler
  shell: export COMPOSER_HOME=/usr/local/composer && composer global require 'friendsofphp/uprofiler=dev-master' creates=/usr/local/composer/vendor/friendsofphp/uprofiler/composer.json

- name: Compile uprofiler
  shell: cd /usr/local/composer/vendor/friendsofphp/uprofiler/extension && phpize && ./configure && make && make install creates=/usr/lib/php5/20121212/uprofiler.so

- name: Configure PHP (cli)
  copy: src=uprofiler.ini dest=/etc/php5/cli/conf.d/21-uprofiler.ini mode=644

- name: Configure PHP (apache2)
  copy: src=uprofiler.ini dest=/etc/php5/apache2/conf.d/21-uprofiler.ini mode=644

The last two tasks copy uprofiler.ini to the relevant configuration directories. uprofiler.ini file is really simple:

[uprofiler]
extension=uprofiler.so

Install XHGui

Similarly, we install XHGui using composer:

- name: Install MongoDB
  apt: pkg={{ item }} state=latest
  with_items:
    - mongodb
    - php5-mongo

- name: Install XHGui
  shell: export COMPOSER_HOME=/usr/local/composer && composer global require --ignore-platform-reqs 'perftools/xhgui=dev-master' creates=/usr/local/composer/vendor/perftools/xhgui/composer.json

- name: Set XHGui permisssions
  file: path=/usr/local/composer/vendor/perftools/xhgui/cache group=www-data mode=775

- name: Configure XHGui
  template: src=xhgui_config.php dest=/usr/local/composer/vendor/perftools/xhgui/config/config.php owner=vagrant group=www-data mode=644

- name: Index mongo for XHGui
  script: xhgui_indexes.sh --some-arguments 1234 creates=/root/indexed_xhgui.txt

XHGi uses MongoDB for storage, so we install that install that first and then install XHGui via composer which pulls in all the dependencies. Note that XHGui has a extension dependency on xhprof, but we're using uprofiler, so we use the --ignore-platform-reqs flag to ignore.

XHGui requires a configuration file in it's config directory. I copied the default one and then changed it to profile every run. The minimum xhgui_config.php that you need is:

<?php
return [
    // Profile every request
    'profiler.enable' => function() {
        return true;
    },
]

This is the place where you could put in additional checks to decide whether to profile or not, such as checking for a GET variable of "profile", for instance.

Lastly, the XHGui README recommends that you add some indexes to MongoDB. I also wanted to automatically delete old records, which is also done via a MongoDB directive. This is done via the xhgui_indexes.sh shell script:

#!/bin/bash

# auto-remove records older than 2592000 seconds (30 days)
mongo xhprof --eval 'db.collection.ensureIndex( { "meta.request_ts" : 1 }, { expireAfterSeconds : 2592000 } )'

# indexes
mongo xhprof --eval  "db.collection.ensureIndex( { 'meta.SERVER.REQUEST_TIME' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'profile.main().wt' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'profile.main().mu' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'profile.main().cpu' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'meta.url' : 1 } )"

touch /root/indexed_xhgui.txt

Note that we create an empty file that is tested in the task as we only need to run this task once.

Set up for profiling

To profile a website, we just need to include /usr/local/composer/vendor/perftools/xhgui/external/header.php. This can be done by setting the auto_prepend_file PHP ini setting. As I use Apache, I can just add:

php_admin_value auto_prepend_file "/usr/local/composer/vendor/perftools/xhgui/external/header.php"

To my VirtualHost configuration.

Set up host for XHGui website

Finally, we need a VirtualHost for the XHGui website where we can view our profiles. I decided to use a separate subdomain, "profile", so my vhost looks like this:

<VirtualHost *:80>
  ServerName profiler.{{ server_name }}
  DocumentRoot /usr/local/composer/vendor/perftools/xhgui/webroot

  <Directory /usr/local/composer/vendor/perftools/xhgui/webroot>
      Options Indexes FollowSymLinks MultiViews
      AllowOverride All
      Order allow,deny
      Allow from all
      Require all granted
  </Directory>
</VirtualHost>

Where {{server_name}} is an Ansible variable that is the domain name of the site.

All done

That's it. Once I had worked out which pieces were required, putting them into Ansible tasks was remarkably obvious and now I can profile my website in development.

Shorter directory text in Bash prompt

Rather helpfully, David Goodwin left a comment about how he shortens the space taken up by the directory section of his terminal's PS1 prompt by using a Bash script to remove the middle portion.

This is a really good idea, so I ported it into my PS1 set up which resulted in some rearranging and thought I'd share here as I modified for OS X and I don't want to lose it!

The relevant portion of my .profile is:

# Git information for prompt
if [ -f $(brew --prefix)/etc/bash_completion.d/git-prompt.sh ]; then
    . $(brew --prefix)/etc/bash_completion.d/git-prompt.sh
fi
GIT_PS1_SHOWDIRTYSTATE=true
GIT_PS1_SHOWUNTRACKEDFILES=true

# Shorten current directory - Based on function by David Goodwin
function shorten_pwd()
{
    LENGTH="40"
    PART1="10"
    PART2="27"

    DIR=`echo "${PWD}" | sed "s/\\/home\\/$USER/~/" | sed "s/\\/Users\\/$USER/~/"`

    if [ ${#DIR} -gt $(($LENGTH)) ]; then
        echo "${DIR:0:$(($PART1))}...${DIR:$((${#DIR}-$PART2)):$PART2}"
    else
        echo "$DIR"
    fi
}


# Set prompt
prompt_cmd () {
    LAST_STATUS=$?

    local COLOUR_RESET='\[\e[0m\]'
    local BLACK='\[\e[0;30m\]'
    local RED='\[\e[0;31m\]'
    local GREEN='\[\e[0;32m\]'
    local YELLOW='\[\e[0;33m\]'
    local BLUE='\[\e[0;34m\]'
    local PURPLE='\[\e[0;35m\]'
    local CYAN='\[\e[0;36m\]'
    local WHITE='\[\e[0;37m\]'
    local BOLD_BLACK='\[\e[1;30m\]'
    local BOLD_RED='\[\e[1;31m\]'
    local BOLD_GREEN='\[\e[1;32m\]'
    local BOLD_YELLOW='\[\e[1;33m\]'
    local BOLD_BLUE='\[\e[1;34m\]'
    local BOLD_PURPLE='\[\e[1;35m\]'
    local BOLD_CYAN='\[\e[1;36m\]'
    local BOLD_WHITE='\[\e[1;37m\]'

    PS1="$BLACK\u@\h"     # user@host
    PS1+=" "
    PS1+="$BLUE"
    PS1+=$(shorten_pwd)   # current directory (usually \w)
    PS1+=" "
    PS1+="$RED"
    PS1+=$(__git_ps1)     # git status
    PS1+="$COLOUR_RESET"
    PS1+='\$ '
}

PROMPT_COMMAND='prompt_cmd && tab_title'

There are three sections here. Firstly we ensure that git-prompt.sh is loaded and configure a couple of settings for it. Then we write a function called shorten_cwd() based on David's script. The main changes here are that I also look for /Users/$USER as that's where OS X stores home directories and that I don't split in the middle. Finally we define prompt_cmd() to set PS1 in a way that I understand and assign it to PROMPT_COMMAND along with tab_title.

The end result looks like this:

Shorter prompt

Setting OS X's Terminal Tab to the current directory

I use many tabs in a Terminal window quite frequently, and while the window title will show the current directory name, the tab title doesn't. You can manually change it using shift+cmd+i, but who can be bothered?

Automating it so that the tab title always matches the current directory turns out to be really easy and just requires a few lines in ~/.profile.

Firstly, we need a function that sets the tab's title to the last segment of the current working directory:

function tab_title {
  echo -n -e "\033]0;${PWD##*/}\007"
}

We then just need to automate calling it whenever we change directory. The easiest way to do this is to change PROMPT_COMMAND:

PROMPT_COMMAND="tab_title ; $PROMPT_COMMAND"

That's it. Whenever I change directory, or open a new terminal tab, then the tab's title now matches the last segment of the directory for that tab. Much more useful!

Terminal tabs

Converting databases between MySQL and SQL Server

I regularly deal projects that target SQL Server, but mostly develop against MySQL to avoid having to run a full Windows stack locally all the time. One of the nice things about PHP with DBAL and Migrations is that the database is pretty well abstracted from my code. Of course, this means that I don't target any of the specialist features, but for these projects, this hasn't been an issue.

To convert data from SQL Server to MySQL, I've found Intelligent Converters's MSSQL to MySQL tool to work very well for my needs. It allows me to grab the test database from the client and convert it to a MySQL dump file that I can then import.

Jerry-rigging pygments to support new PHP keywords

I use rst2pdf to create my presentations and noticed that the syntax highlighter wasn't highlighting instanceof.

rst2pdf uses pygments for syntax highlighting, so I wondered what was going on. A short investigation led to me realise that the current stable version of pigments is 1.6 and they are working on 2.0. It seems that 2.0 has a number of changes to the PHP lexer, which aren't in 1.6.

While I'm waiting, I modified my local copy of pigments 1.6 directly!

On my Mac, the file I'm interested in is in the egg file at /Library/Python/2.7/site-packages/Pygments-1.6-py2.7.egg. The .egg file is simply a directly, so within there, I edited pygments/lexer/web.py which is where the PHP lexer is.

Open web.py and look for the PhpLexer class (it's around line 759 at the moment!). Scrolling further down, you come to the tokens section and then with the 'php' array, I found:

(r'(and|E_PARSE|old_function|E_ERROR|or|as|E_WARNING|parent|'
             r'eval|PHP_OS|break|exit|case|extends|PHP_VERSION|cfunction|'
             r'FALSE|print|for|require|continue|foreach|require_once|'
             r'declare|return|default|static|do|switch|die|stdClass|'
             r'echo|else|TRUE|elseif|var|empty|if|xor|enddeclare|include|'
             r'virtual|endfor|include_once|while|endforeach|global|__FILE__|'
             r'endif|list|__LINE__|endswitch|new|__sleep|endwhile|not|'
             r'array|__wakeup|E_ALL|NULL|final|php_user_filter|interface|'
             r'implements|instanceof|public|private|protected|abstract|clone|try|'
             r'catch|throw|this|use|namespace|trait)\b', Keyword),

This is the list of words that will be highlighted as keywords. Simply add the new ones that you need.

I then deleted the web.pyc file that was in the lexer directory (though I'm unsure if I needed to as it may auto-recreate itself) and then ran rst2pdf again to create my pdf with instanceof, yield & finally now correctly highlighted!

Obviously, when the next version of pygments is released, hopefully it'll be up to date, so jerry-rigging won't be required. Until then, this is working for me, at least.

View status of all Vagrant environments

I've just upgraded to Vagrant version 1.6, and vagrant global-status is possibly my favourite new feature.

This command lists all currently up Vagrant environments wherever they may be on your computer:

$ vagrant global-status

id       name    provider   state   directory                                         
--------------------------------------------------------------------------------------
dbc7770  joindin virtualbox running /Users/rob/www/thirdparty/joindin-vm              
0683c7a  default virtualbox running /Users/rob/www/thirdparty/joindin-zs7 
 
The above shows information about all known Vagrant environments
on this machine. This data is cached and may not be completely
up-to-date. To interact with any of the machines, you can go to
that directory and run Vagrant, or you can use the ID directly
with Vagrant commands from any directory. For example:
"vagrant destroy 1a2b3c4d"

As you can see by the helpful information message, you can then pass the id to any of the usual Vagrant commands in order to work with that Vagrant install rather than changing directory first.

I tend to use this to halt VMs that I've left running by accident! i.e.

$ vagrant halt 0683c7a

All in all, a very useful feature!

Z-Ray for Zend Server 7

I see that Zend Server 7 has now been released.

I've been running the beta for all my development work for a while now and the main reason is the new Z-Ray feature. Z-Ray is a bar that is injected into the bottom of your page showing lots of useful information.

This is what it looks like in its closed state when run on my development version of joind.in:

Z ray1

At a glance, I can see that this page's performance is acceptable, there's a notice that I need to look at and that 11 database queries were executed.

Continue reading

Perl syntax highlighting in Sublime Text 3

I'm currently writing a project in Perl for a client and have discovered that the default Perl syntax highlighting in Sublime Text is terrible.

Fortunately, the community has stepped up and Blaise Roth has created the ModerlPerl package. Install via Package Control.

To get all Perl files to open with the new syntax highlighter, use View > Syntax > Open all with current extension as… and select ModernPerl from the sub-menu.

Setting up Zend Server 6 on OS X for PHP development

I recently decided to upgrade my Mac's PHP to 5.4. One of the available options is Zend Server 6.1, Free edition. These are my notes how to set it up so that it works the way I develop.

Installation

Mount the disk image and follow the installation wizard. On OS X, Zend Server installs PHP, Apache and MySQL inside /usr/local/zend.

The Zend Server admin interface is at at http://localhost:10081. On first run you have to go through a wizard. I picked development for my Launch Option. You have to set an admin password – as it's my dev machine, I picked something simple :)

More information is available from Joe Stagner's guide on devzone. Ignore the bit about editing php.ini though as you can do that via Zend Server's admin system.

Apache

Like Joe, I also want the Apache running on port 80. Edit /usr/local/zend/apache2/conf/httpd.conf and replace Listen 10888 with Listen 80.

Note that the default localhost directory is /usr/local/zend/apache2/htdocs and the Apache processes run as user daemon with group daemon.

Zend Server admin

Once logged in, you can configure Zend Server:

Configuration -> PHP:
Set your PHP settings are required. Then press Save. To restart the server, click the button with two arrows in a circle on the toolbar.

You must set the date.timezone to stop the PHP warning. I also set sendmail_path as I like to redirect email on my development box.

Configuration -> Components:
Look down the list and turn off what you don't need. I turned off Zend Data Cache, Zend Job Queue and Zend Page Cache in addition to what is already off.

Command line control

Add the relevant Zend Server paths to your environment. Edit .bash_profile and add

export PATH=/usr/local/zend/bin:/usr/local/zend/mysql/bin:$PATH 

to the bottom. This will give you access to the php, mysql and other Zend Server related CLI tools.

zendctl.sh

The main CLI tool to control Zend Server is zendctl.sh. The most interesting commands are:

  • sudo zendctl.sh restart-apache to restart Apache
  • sudo zendctl.sh restart-mysql to restart MySQL

MySQL

Zend Server supplies MySQL version 5.5.27 rather than 5.6 for some reason. If you want 5.6, then grab it from MySQL's website or homebrew.

If you use your own MySQL, then stop Zend Server from starting its one by editing /etc/zcd.rc and changing MYSQL_EN=true to MYSQL_EN=false.

If you're using Zend Server's MySQL, then set a root password:

$ mysqladmin -u root password {new-password}
$ mysqladmin -u root -p{new-password} -h localhost password {new-password}
$ mysqladmin -u root -p{new-password} reload
$ history -c

I also had to link mysql.sock to /tmp so that some mysql CLI applications would work:

  • sudo ln -s /usr/local/zend/mysql/tmp/mysql.sock /tmp/mysql.sock

Apache vhosts

I couldn't work out how to configure Apache virtual hosts in the Zend Server admin unless you had an "application package", so I did them manually.

Apache is configured to look in /usr/local/zend/etc/sites.d/ for files names globals-*.conf and vhost_*.conf.

I created globals-vhost.conf:

NameVirtualHost *:80

LoadModule vhost_alias_module modules/mod_vhost_alias.so

And then I have vhost_dev.conf to configure my virtual hosts:

<virtualhost *:80>
    ServerAdmin webmaster@akrabat.com
    DocumentRoot "/www"
    ServerName localhost
    <directory "/www">
	    Options Indexes FollowSymLinks MultiViews
    	AllowOverride All
        Order allow,deny
	    Allow from all
    </directory>
</virtualhost>

<virtualhost *:80>
    VirtualDocumentRoot "/www/dev/%-2+/public"
    ServerName subdomains.dev
    ServerAlias *.dev
    UseCanonicalName Off
    LogFormat "%V %h %l %u %t "%r" %s %b" vcommon
    ErrorLog "/www/dev/vhosts-error_log"
    DirectoryIndex index.php index.html
    <directory "/www/dev/*">
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Order allow,deny
        Allow from all
        
        SetEnv APPLICATION_ENV development
                
        RewriteEngine On
        RewriteBase /
        RewriteCond %{REQUEST_FILENAME} -s [OR]
        RewriteCond %{REQUEST_FILENAME} -l [OR]
        RewriteCond %{REQUEST_FILENAME} -d
        RewriteRule ^.*$ - [NC,L]
        RewriteRule ^.*$ index.php [NC,L]        
    </directory>    
</virtualhost>

This allows me to put my development files in /www and sets up automatic virtual hosts for http://*.dev.

Fix sending email

Sending email doesn't work out of the box. To fix:

$ sudo mv /usr/local/zend/lib/libsasl2.2.dylib /usr/local/zend/lib/libsasl2.2.dylib.old
$ sudo ln -s /usr/lib/libsasl2.2.dylib /usr/local/zend/lib/libsasl2.2.dylib

See here for details.

Logs

The Zend Server admin displays your logs via Overview -> Logs. There is a list of log files down the left hand side. The interesting ones for me are:

  • error: Apache error log (/usr/local/zend/var/log/error.log)
  • access: Apache access log (/usr/local/zend/var/log/access.log)
  • php: PHP log (/usr/local/zend/var/log/php.log)

That's it. Zend Server now works as a development stack and you get the nice diagnostics features like code tracing and the events list.