Category Archives: Computing

Git push to multiple repositories

I have a couple of projects where I need to push to more than one repo all the time.

I have been using this command line to do so:

git push origin && git push other-remote

However, I recently discovered that I can create a remote that points to more than one repository using these commands:

git remote add all git@github.com:akrabat/projectname.git
git remote set-url --add all ssh://example.com/path/to/projectname.git

I now have a remote called all that will push to both repositories!

There's no automatic way to go the other way and fetch from multiple repositories though as apparently it makes less sense to fetch the same branch identifier from multiple places.

Further details in this email by Linus in 2006.

If you want to script the creation of the all remote, then you could use this script which manipulates the remote configuration settings directly:

#!/bin/bash

if [ "`git remote| grep all`" == "all" ] ; then
    git remote remove all
fi

for r in `git remote`
do
    git config --add remote.all.url `git config remote.$r.url`
done

Create this as /usr/local/bin/git-add-push-all.sh and then you can just run it in the root of your project.

Context specific history at the bash prompt

One change I made recently to my .profile is this:

# up & down map to history search once a command has been started.
bind '"\e[A":history-search-backward'
bind '"\e[B":history-search-forward'

These two bind command change the way that the up and down arrow keys work once you start typing a command to only search the history for lines that start with what you've typed so far.

This means that I type, say, git and then press ↑ & ↓ to go through all the times I've typed a git command without having to go through all the other commands.

It's quite handy and I find it easier to use than ctrl+r.

Setting up mailcatcher as a service in Debian/Ubuntu

I've recently been changing joind.in's Vagrant system to use Debian and one issue I came across was getting Mailcatcher to start on boot and integrate property with the service command.

To do this, I created an init script which is based off the skeleton and then stored this in /etc/init.d and then ran update-rc.d mailcatcher defaults to set up the correct links in the various rc.d directories.

This is the init script:

/etc/init.d/mailcatcher:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          mailcatcher
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Example initscript
# Description:       This file should be used to construct scripts to be
#                    placed in /etc/init.d.
### END INIT INFO

# Do NOT "set -e"

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/rvm/gems/ruby-1.9.3-p547/bin
DESC="Super simple SMTP server"
NAME=mailcatcher
DAEMON=/usr/local/rvm/wrappers/default/$NAME
DAEMON_ARGS=" --http-ip 0.0.0.0"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
    # Return
    #   0 if daemon has been started
    #   1 if daemon was already running
    #   2 if daemon could not be started
    start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
        || return 1
    start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
        $DAEMON_ARGS \
        || return 2
    # Add code here, if necessary, that waits for the process to be ready
    # to handle requests from services started subsequently which depend
    # on this one.  As a last resort, sleep for some time.
     
    # Create the PIDFILE
    pidof mailcatcher >> $PIDFILE
}

#
# Function that stops the daemon/service
#
do_stop()
{
    # Return
    #   0 if daemon has been stopped
    #   1 if daemon was already stopped
    #   2 if daemon could not be stopped
    #   other if a failure occurred
    
    if [ -f "$PIDFILE" ]
    then
        kill `cat $PIDFILE`
        rm -f $PIDFILE
        return 0
    else
        return 1
    fi
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
    #
    # If the daemon can reload its configuration without
    # restarting (for example, when it is sent a SIGHUP),
    # then implement that here.
    #
    start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
    return 0
}

case "$1" in
  start)
    [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
    do_start
    case "$?" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
    esac
    ;;
  stop)
    [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
    do_stop
    case "$?" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
    esac
    ;;
  status)
    status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
    ;;
  #reload|force-reload)
    #
    # If do_reload() is not implemented then leave this commented out
    # and leave 'force-reload' as an alias for 'restart'.
    #
    #log_daemon_msg "Reloading $DESC" "$NAME"
    #do_reload
    #log_end_msg $?
    #;;
  restart|force-reload)
    #
    # If the "reload" option is implemented then remove the
    # 'force-reload' alias
    #
    log_daemon_msg "Restarting $DESC" "$NAME"
    do_stop
    case "$?" in
      0|1)
        do_start
        case "$?" in
            0) log_end_msg 0 ;;
            1) log_end_msg 1 ;; # Old process is still running
            *) log_end_msg 1 ;; # Failed to start
        esac
        ;;
      *)
        # Failed to stop
        log_end_msg 1
        ;;
    esac
    ;;
  *)
    #echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
    echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
    exit 3
    ;;
esac

:

As you can probably tell, it's very obviously a tweaked version of /etc/init.d/skeleton, but there are some important changes:

Path to binary
As Mailcatcher is a ruby app, the correct path to the binary is actually /usr/local/rvm/wrappers/default/mailcatcher which is not where which tells you it is. I suspect that my lack of knowledge about Ruby environments is showing…

do_start()
The do_start() function calls through to start-stop-daemon to start mailcatcher. However this doesn't create a pid file in /var/run for us, so we create it ourselves using:

pidof mailcatcher >> $PIDFILE

do_stop()
Mailcatcher is intended to be stopped by pressing the Quit button in the HTML interface, so the default code in skeleton doesn't work. I rewrote it to simply kill the process if the pid file exists:

    if [ -f "$PIDFILE" ]
    then
        kill `cat $PIDFILE`
        rm -f $PIDFILE
        return 0
    else
        return 1
    fi

That's it. The most important thing about these changes is that service mailcatcher status now works as expected and so Puppet's ensure => 'running' test actually works correctly.

Sharing host VPN with Vagrant

When moving a project into Vagrant, I realised that I needed the vagrant guest to share my OS X's VPN.

I'm using a separate IP address within a private network like this:

config.vm.network :private_network, ip: "192.168.101.101"

So, after some Googling, I added this provider configuration setting:

  config.vm.provider :virtualbox do |vb|
      vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
  end

and now my VM is sending data over the VPN. Note that this will not work when using the Vagrant's public_network setting though.

Changing the GitHub IRC hooks notification events

As joind.in uses GitHub to host its source code, we use the IRC hook to receive notifications to the IRC channel (#joind.in on freenode) when interesting things happen on the GitHub repositories.

We noticed recently that we were being notified about more types of things happening on some repositories compared to others, so I decided to investigate.

The code for this is here and a quick perusal of it shows that it will do something for these events:

  • push
  • commit_comment
  • pull_request
  • pull_request_review_comment
  • issues
  • issue_comment

However, when you go the administration page on the website, you cannot set which events to be notified on.

Fortunately, GitHub has an API and so you can quite easily manipulate the hooks using curl.

To read all the hooks attached to a project, you do this:

curl -u '{your github username}' -H "Accept: application/json" 
https://api.github.com/repos/{owner name}/{repository name}/hooks

so, for the joind.in API project, I do:

curl -u 'akrabat' -H "Accept: application/json" 

https://api.github.com/repos/joindin/joindin-api/hooks

This will return a list of hooks attached, including all the information about them. In this case, I'm interested in the IRC hook and there's two interesting keys in the results:

    "url": "https://api.github.com/repos/joindin/joindin-api/hooks/932001",
    "events": [
      "push",
      "pull_request"
    ],

The url provides the information on the end point to use to change the data and the events tell me what's set up. As you can see, only "push" and "pull_request" are set up for this repository.

To change this, we simply use CURL to POST a new set of events:

curl -u 'akrabat'  -H "Accept: application/json" -H "Content-type: application/json" -X PATCH 

https://api.github.com/repos/joindin/joindin-api/hooks/932001

-d '{"events":["push", "pull_request", "commit_comment", "pull_request_review_comment"]}'

Helpfully, the response includes a representation of the hook we have just modified, so we can see that the events list has changed. For more information, look at GitHub's API documentation for hooks.

I therefore went through all the joind.in repositories and set them all to the same set of events for consistency. We'll no doubt find out soon if that results in too many notifications to the channel!

rst2html does not support :start inline:

I'm currently writing some documentation in Restructured Text that I'm targeting at HTML and PDF using rst2html and rst2pdf.

For syntax highlighting, both rst2html and rst2pdf use Pygments, however rst2html doesn't support any Pygments options.

So a typical PHP snippet in rst targeting rst2pdf, would be written as:

.. code-block: php
    :startinline: true

    $this = str_replace('foo', 'bar', $that);

The startinline Pygments option is to allow it to highlight the snippet, even though the opening <?php is missing.

If you run rst2html, you get this error:

(ERROR/3) Error in "code-block" directive: unknown option: "startinline".

aargh!

To remove the error, I've simply run my rst file through see first, so I have a create-html.sh script that does this:

#!/bin/bash

sed '/startinline/d' ${1}.rst > temp.rst
rst2html.py --syntax-highlight=short --stylesheet=syntax.css temp.rst > ${1}.html
rm temp.rst

The sed command removes any lines that contain the word startinline and obviously, this means that the HTML version doesn't syntax highlight the snippet, but at least it doesn't generate an error!