Category Archives: Software

Styling rst2pdf tables

I currently use rst2pdf to create presentations slide decks from reStructured Text files. I like rST a lot as it's more expressive than Markdown and allows for extension.

Tables in rST are marked up like this:

+-----------+-----------+-----------+
| Heading 1 | Heading 2 | Heading 3 |
+===========+===========+===========+
| a         | b         | c         |
|           |           |           |
| aa        |           |           |
+-----------+-----------+-----------+
| d         | e         | f         |
+-----------+-----------+-----------+
| g         | h         | i         |
+-----------+-----------+-----------+
| j         | k         | l         |
+-----------+-----------+-----------+

We create a PDF file with the command rst2pdf test.rst which produces a table that looks like this:

Rst table standard

To style, this we create styles within a style file and then compile using rst2pdf test.rst -s my.style.

Let's start with the table element:

styles:
    table:
      commands: []
         [VALIGN, [ 0, 0 ], [ -1, -1 ], TOP ]
         [INNERGRID, [ 0, 0 ], [ -1, -1 ], 0.25, black ]
         [ROWBACKGROUNDS, [0, 0], [-1, -1], [white,#E0E0E0]]
         [BOX, [ 0, 0 ], [ -1, -1 ], 0.25, black ]

Behind the scenes, rst2pdf uses ReportLab to create the PDF. The commands style maps directly to ReportLab's TableStyle commands (section 7.4 of the current documentation)

Each command contains an identifier, the start and stop cell definition to which it applies and then the style to apply. The cell definition is defined as [X,Y] where [0,0] is the top left cell, [2,3] would be the cell with e in it in the definition above. Negative numbers count from the bottom right, so [-1,-1] is the bottom-right corner.

Key options:

VALIGN Vertical text alignment. Options: TOP, BOTTOM, MIDDLE.
INNERGRID, BOX, LINEBELOW, LINEABOVE, LINEBEFORE, LINEAFTER Style of the borders. First parameter is line thickness and second is colour.
BACKGROUND, ROWBACKGROUNDS, COLBACKGROUNDS Background colour. Parameter is an array of colours, used cyclically.
TOPPADDING, BOTTOMPADDING, LEFTPADDING, RIGHTPADDING Padding with cells. Parameter is a number.

The style for the table heading is called table-heading:

    table-heading:
        parent : heading
        backColor : beige
        alignment : TA_CENTER
        valign : BOTTOM
        borderPadding : 0

These settings should be obvious. I always override backColor!

The style for the table elements is table-body:

table-body:
      parent : normal

By default, it isn't styled, but if you want to change the textColor, this is where to do it.

Overriding on a per table basis

To override for a specific table, then set a class before the table:

.. class:: mytable

+-----------+-----------+-----------+
| Heading 1 | Heading 2 | Heading 3 |
+===========+===========+===========+
| a         | b         | c         |
|           |           |           |
| aa        |           |           |
+-----------+-----------+-----------+
| d         | e         | f         |
+-----------+-----------+-----------+

The most common reason to do this is to set up specific column widths:

    mytable:
        parent: table
        colWidths: [3cm, 6cm, 3cm]

This is mostly useful when the auto-sizing routine causes odd line breaks in heading text.

If you are targeting rst2pdf, then you can also set widths using the .. widths:: directive like this:

.. widths:: 20 20 60

+-----------+-----------+-----------+
| Heading 1 | Heading 2 | Heading 3 |
+===========+===========+===========+
| a         | b         | c         |
+-----------+-----------+-----------+
| d         | e         | f         |
+-----------+-----------+-----------+

The width numbers are percentages and it only works if you pass the command line option -e preprocess when compiling.

My defaults

My defaults currently are:

styles:
    table:
        commands: []
            [VALIGN, [ 0, 0 ], [ -1, -1 ], MIDDLE ]
            [ROWBACKGROUNDS, [0, 0], [-1, -1], [white]]
            [LINEBELOW, [0, 0], [-1, 0], 0.5, black]
            [BOTTOMPADDING, [0, 1], [-1, -1], 5]
            [TOPPADDING, [0, 1], [-1, -1], 5]
            [ALIGN, [ 0, 0 ], [ -1, -1 ], LEFT ]


    table-heading:
        parent : heading
        fontName: stdBold
        backColor : white
        alignment : TA_LEFT

This results in the very simple table style of:

Rst default

This suits me as a starting point.

installing XHGui via Ansible

I'm still using Ansible to provision Vagrant VMs. This is how I added the XHGui profiler to my standard setup.

Theres a number steps we need to do:

  • Install Composer
  • Install the uprofiler PHP extension
  • Install XHGui
  • Set up for profiling
  • Set up host for XHGui website

Install Composer

Installing Composer requires these tasks:

- name: Install Composer
  shell: curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin creates=/usr/local/bin/composer

- name: Rename composer.phar to composer
  shell: mv /usr/local/bin/composer.phar /usr/local/bin/composer creates=/usr/local/bin/composer

- name: Make composer executable
  file: path=/usr/local/bin/composer mode=a+x state=file

- name: Create global composer directory
  file: path=/usr/local/composer state=directory mode=0775

Firstly we download the Composer installer and run it to create composer.phar. We then rename to composer, make executable and then create a global directory for storing the packages that we download.

Install the uprofiler PHP extension

We install uprofiler via composer:

- name: Install uprofiler
  shell: export COMPOSER_HOME=/usr/local/composer && composer global require 'friendsofphp/uprofiler=dev-master' creates=/usr/local/composer/vendor/friendsofphp/uprofiler/composer.json

- name: Compile uprofiler
  shell: cd /usr/local/composer/vendor/friendsofphp/uprofiler/extension && phpize && ./configure && make && make install creates=/usr/lib/php5/20121212/uprofiler.so

- name: Configure PHP (cli)
  copy: src=uprofiler.ini dest=/etc/php5/cli/conf.d/21-uprofiler.ini mode=644

- name: Configure PHP (apache2)
  copy: src=uprofiler.ini dest=/etc/php5/apache2/conf.d/21-uprofiler.ini mode=644

The last two tasks copy uprofiler.ini to the relevant configuration directories. uprofiler.ini file is really simple:

[uprofiler]
extension=uprofiler.so

Install XHGui

Similarly, we install XHGui using composer:

- name: Install MongoDB
  apt: pkg={{ item }} state=latest
  with_items:
    - mongodb
    - php5-mongo

- name: Install XHGui
  shell: export COMPOSER_HOME=/usr/local/composer && composer global require --ignore-platform-reqs 'perftools/xhgui=dev-master' creates=/usr/local/composer/vendor/perftools/xhgui/composer.json

- name: Set XHGui permisssions
  file: path=/usr/local/composer/vendor/perftools/xhgui/cache group=www-data mode=775

- name: Configure XHGui
  template: src=xhgui_config.php dest=/usr/local/composer/vendor/perftools/xhgui/config/config.php owner=vagrant group=www-data mode=644

- name: Index mongo for XHGui
  script: xhgui_indexes.sh --some-arguments 1234 creates=/root/indexed_xhgui.txt

XHGi uses MongoDB for storage, so we install that install that first and then install XHGui via composer which pulls in all the dependencies. Note that XHGui has a extension dependency on xhprof, but we're using uprofiler, so we use the --ignore-platform-reqs flag to ignore.

XHGui requires a configuration file in it's config directory. I copied the default one and then changed it to profile every run. The minimum xhgui_config.php that you need is:

<?php
return [
    // Profile every request
    'profiler.enable' => function() {
        return true;
    },
]

This is the place where you could put in additional checks to decide whether to profile or not, such as checking for a GET variable of "profile", for instance.

Lastly, the XHGui README recommends that you add some indexes to MongoDB. I also wanted to automatically delete old records, which is also done via a MongoDB directive. This is done via the xhgui_indexes.sh shell script:

#!/bin/bash

# auto-remove records older than 2592000 seconds (30 days)
mongo xhprof --eval 'db.collection.ensureIndex( { "meta.request_ts" : 1 }, { expireAfterSeconds : 2592000 } )'

# indexes
mongo xhprof --eval  "db.collection.ensureIndex( { 'meta.SERVER.REQUEST_TIME' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'profile.main().wt' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'profile.main().mu' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'profile.main().cpu' : -1 } )"
mongo xhprof --eval  "db.collection.ensureIndex( { 'meta.url' : 1 } )"

touch /root/indexed_xhgui.txt

Note that we create an empty file that is tested in the task as we only need to run this task once.

Set up for profiling

To profile a website, we just need to include /usr/local/composer/vendor/perftools/xhgui/external/header.php. This can be done by setting the auto_prepend_file PHP ini setting. As I use Apache, I can just add:

php_admin_value auto_prepend_file "/usr/local/composer/vendor/perftools/xhgui/external/header.php"

To my VirtualHost configuration.

Set up host for XHGui website

Finally, we need a VirtualHost for the XHGui website where we can view our profiles. I decided to use a separate subdomain, "profile", so my vhost looks like this:

<VirtualHost *:80>
  ServerName profiler.{{ server_name }}
  DocumentRoot /usr/local/composer/vendor/perftools/xhgui/webroot

  <Directory /usr/local/composer/vendor/perftools/xhgui/webroot>
      Options Indexes FollowSymLinks MultiViews
      AllowOverride All
      Order allow,deny
      Allow from all
      Require all granted
  </Directory>
</VirtualHost>

Where {{server_name}} is an Ansible variable that is the domain name of the site.

All done

That's it. Once I had worked out which pieces were required, putting them into Ansible tasks was remarkably obvious and now I can profile my website in development.

Shorter directory text in Bash prompt

Rather helpfully, David Goodwin left a comment about how he shortens the space taken up by the directory section of his terminal's PS1 prompt by using a Bash script to remove the middle portion.

This is a really good idea, so I ported it into my PS1 set up which resulted in some rearranging and thought I'd share here as I modified for OS X and I don't want to lose it!

The relevant portion of my .profile is:

# Git information for prompt
if [ -f $(brew --prefix)/etc/bash_completion.d/git-prompt.sh ]; then
    . $(brew --prefix)/etc/bash_completion.d/git-prompt.sh
fi
GIT_PS1_SHOWDIRTYSTATE=true
GIT_PS1_SHOWUNTRACKEDFILES=true

# Shorten current directory - Based on function by David Goodwin
function shorten_pwd()
{
    LENGTH="40"
    PART1="10"
    PART2="27"

    DIR=`echo "${PWD}" | sed "s/\\/home\\/$USER/~/" | sed "s/\\/Users\\/$USER/~/"`

    if [ ${#DIR} -gt $(($LENGTH)) ]; then
        echo "${DIR:0:$(($PART1))}...${DIR:$((${#DIR}-$PART2)):$PART2}"
    else
        echo "$DIR"
    fi
}


# Set prompt
prompt_cmd () {
    LAST_STATUS=$?

    local COLOUR_RESET='\[\e[0m\]'
    local BLACK='\[\e[0;30m\]'
    local RED='\[\e[0;31m\]'
    local GREEN='\[\e[0;32m\]'
    local YELLOW='\[\e[0;33m\]'
    local BLUE='\[\e[0;34m\]'
    local PURPLE='\[\e[0;35m\]'
    local CYAN='\[\e[0;36m\]'
    local WHITE='\[\e[0;37m\]'
    local BOLD_BLACK='\[\e[1;30m\]'
    local BOLD_RED='\[\e[1;31m\]'
    local BOLD_GREEN='\[\e[1;32m\]'
    local BOLD_YELLOW='\[\e[1;33m\]'
    local BOLD_BLUE='\[\e[1;34m\]'
    local BOLD_PURPLE='\[\e[1;35m\]'
    local BOLD_CYAN='\[\e[1;36m\]'
    local BOLD_WHITE='\[\e[1;37m\]'

    PS1="$BLACK\u@\h"     # user@host
    PS1+=" "
    PS1+="$BLUE"
    PS1+=$(shorten_pwd)   # current directory (usually \w)
    PS1+=" "
    PS1+="$RED"
    PS1+=$(__git_ps1)     # git status
    PS1+="$COLOUR_RESET"
    PS1+='\$ '
}

PROMPT_COMMAND='prompt_cmd && tab_title'

There are three sections here. Firstly we ensure that git-prompt.sh is loaded and configure a couple of settings for it. Then we write a function called shorten_cwd() based on David's script. The main changes here are that I also look for /Users/$USER as that's where OS X stores home directories and that I don't split in the middle. Finally we define prompt_cmd() to set PS1 in a way that I understand and assign it to PROMPT_COMMAND along with tab_title.

The end result looks like this:

Shorter prompt

Setting OS X's Terminal Tab to the current directory

I use many tabs in a Terminal window quite frequently, and while the window title will show the current directory name, the tab title doesn't. You can manually change it using shift+cmd+i, but who can be bothered?

Automating it so that the tab title always matches the current directory turns out to be really easy and just requires a few lines in ~/.profile.

Firstly, we need a function that sets the tab's title to the last segment of the current working directory:

function tab_title {
  echo -n -e "\033]0;${PWD##*/}\007"
}

We then just need to automate calling it whenever we change directory. The easiest way to do this is to change PROMPT_COMMAND:

PROMPT_COMMAND="tab_title ; $PROMPT_COMMAND"

That's it. Whenever I change directory, or open a new terminal tab, then the tab's title now matches the last segment of the directory for that tab. Much more useful!

Terminal tabs

Converting databases between MySQL and SQL Server

I regularly deal projects that target SQL Server, but mostly develop against MySQL to avoid having to run a full Windows stack locally all the time. One of the nice things about PHP with DBAL and Migrations is that the database is pretty well abstracted from my code. Of course, this means that I don't target any of the specialist features, but for these projects, this hasn't been an issue.

To convert data from SQL Server to MySQL, I've found Intelligent Converters's MSSQL to MySQL tool to work very well for my needs. It allows me to grab the test database from the client and convert it to a MySQL dump file that I can then import.

Jerry-rigging pygments to support new PHP keywords

I use rst2pdf to create my presentations and noticed that the syntax highlighter wasn't highlighting instanceof.

rst2pdf uses pygments for syntax highlighting, so I wondered what was going on. A short investigation led to me realise that the current stable version of pigments is 1.6 and they are working on 2.0. It seems that 2.0 has a number of changes to the PHP lexer, which aren't in 1.6.

While I'm waiting, I modified my local copy of pigments 1.6 directly!

On my Mac, the file I'm interested in is in the egg file at /Library/Python/2.7/site-packages/Pygments-1.6-py2.7.egg. The .egg file is simply a directly, so within there, I edited pygments/lexer/web.py which is where the PHP lexer is.

Open web.py and look for the PhpLexer class (it's around line 759 at the moment!). Scrolling further down, you come to the tokens section and then with the 'php' array, I found:

(r'(and|E_PARSE|old_function|E_ERROR|or|as|E_WARNING|parent|'
             r'eval|PHP_OS|break|exit|case|extends|PHP_VERSION|cfunction|'
             r'FALSE|print|for|require|continue|foreach|require_once|'
             r'declare|return|default|static|do|switch|die|stdClass|'
             r'echo|else|TRUE|elseif|var|empty|if|xor|enddeclare|include|'
             r'virtual|endfor|include_once|while|endforeach|global|__FILE__|'
             r'endif|list|__LINE__|endswitch|new|__sleep|endwhile|not|'
             r'array|__wakeup|E_ALL|NULL|final|php_user_filter|interface|'
             r'implements|instanceof|public|private|protected|abstract|clone|try|'
             r'catch|throw|this|use|namespace|trait)\b', Keyword),

This is the list of words that will be highlighted as keywords. Simply add the new ones that you need.

I then deleted the web.pyc file that was in the lexer directory (though I'm unsure if I needed to as it may auto-recreate itself) and then ran rst2pdf again to create my pdf with instanceof, yield & finally now correctly highlighted!

Obviously, when the next version of pygments is released, hopefully it'll be up to date, so jerry-rigging won't be required. Until then, this is working for me, at least.

View status of all Vagrant environments

I've just upgraded to Vagrant version 1.6, and vagrant global-status is possibly my favourite new feature.

This command lists all currently up Vagrant environments wherever they may be on your computer:

$ vagrant global-status

id       name    provider   state   directory                                         
--------------------------------------------------------------------------------------
dbc7770  joindin virtualbox running /Users/rob/www/thirdparty/joindin-vm              
0683c7a  default virtualbox running /Users/rob/www/thirdparty/joindin-zs7 
 
The above shows information about all known Vagrant environments
on this machine. This data is cached and may not be completely
up-to-date. To interact with any of the machines, you can go to
that directory and run Vagrant, or you can use the ID directly
with Vagrant commands from any directory. For example:
"vagrant destroy 1a2b3c4d"

As you can see by the helpful information message, you can then pass the id to any of the usual Vagrant commands in order to work with that Vagrant install rather than changing directory first.

I tend to use this to halt VMs that I've left running by accident! i.e.

$ vagrant halt 0683c7a

All in all, a very useful feature!

Z-Ray for Zend Server 7

I see that Zend Server 7 has now been released.

I've been running the beta for all my development work for a while now and the main reason is the new Z-Ray feature. Z-Ray is a bar that is injected into the bottom of your page showing lots of useful information.

This is what it looks like in its closed state when run on my development version of joind.in:

Z ray1

At a glance, I can see that this page's performance is acceptable, there's a notice that I need to look at and that 11 database queries were executed.

Continue reading

Perl syntax highlighting in Sublime Text 3

I'm currently writing a project in Perl for a client and have discovered that the default Perl syntax highlighting in Sublime Text is terrible.

Fortunately, the community has stepped up and Blaise Roth has created the ModerlPerl package. Install via Package Control.

To get all Perl files to open with the new syntax highlighter, use View > Syntax > Open all with current extension as… and select ModernPerl from the sub-menu.