All posts by Tom

Fedora 18 – non-graphical boot and ifconfig

After downloading Fedora 18 and creating a minimal install virtual machine I went straight to doing the default I always do.  As I was creating a test server I need to be able to see what’s happening during boot so I disable the graphical boot.

Previously (at-least up to Fedora 15) I’d used the plymouth commands to switch the graphical boot to details which had worked.

[root@localhost ~]# plymouth-set-default-plugin details
[root@localhost ~]# /usr/libexec/plymouth/plymouth-update-initrd

However, when I tried this with Fedora 18, nope, not working, the community to the rescue. Another individual  (Nigel Smith) had had the same issue and found the solution in editing the default grub configuration file. So if you need the same visit here:

http://nwsmith.blogspot.co.uk/2012/10/customizing-grub2-boot-options-in.html

While I was at it I also noticed that ifconfig produced a “Command not found”. Erm … wha? Where did that go? Isn’t that like a basic linux command? Turns out in Fedora 18 it’s been removed due to the alternative and replacement iproute package being in place for “many years”. So you’ve now got 2 choices:

1. use the ip command:

ip addr

2. install the net-tools package:

yum -y install net-tools

Formatting a Date in PHP using ISO Format

We all know how to format a date in PHP when using the date functions, but what happens if you’ve only got the ISO format? This doesn’t work with PHP’s date functions.

Well I got this exact situation whilst using Zend_Locale in Zend Framework. Because the locale data files utilised are sourced externally the format comes back in ISO format.  A conversion function is provided (Zend_Locale_Format::convertPhpToIsoFormat) however that’s converting the wrong way and won’t help in this situation.

I went ahead and wrote a quick function which will return a formatted date using a provided ISO format, rather than PHP format.  It works like the PHP function and accepts the same parameters.

You can find the function as a gist on github:

<?php

function date_iso($format, $timestamp = null)
{
	if ($timestamp === null) {
		$timestamp = time();
	}

	$convert = array(
		'a' => 'A' , 'B' => 'B', 'D' => 'z', 'ddd' => 't', 'dd' => 'd', 'd' => 'j',
		'EEEE' => 'l', 'EE' => 'D', 'eee' => 'N', 'e' => 'w', 'HH' => 'H', 'H' => 'G',
		'hh' => 'h', 'h' => 'g', 'I' => 'I', 'l' => 'L', 'MMMM' => 'F', 'MMM' => 'M',
		'MM' => 'm', 'M' => 'n', 'mm' => 'i', 'r' => 'r', 'SS' => 'S', 'ss' => 's',
		'U' => 'U', 'ww' => 'W', 'X' => 'Z', 'YYYY' => 'o', 'yyyy' => 'Y', 'yy' => 'y',
		'ZZZZ' => 'P', 'zzzz' => 'e', 'Z' => 'O', 'z' => 'T'
	);

	$values = preg_split(
		'/(a|B|D|d{1,3}|EEEE|EE|eee|e|HH|H|hh|h|I|l|M{1,4}|mm|r|SS|ss|U|ww|X|YYYY|yyyy|yy|ZZZZ|zzzz|Z|z|[^a-zA-Z]+)/',
		$format,
		-1,
		PREG_SPLIT_DELIM_CAPTURE | PREG_SPLIT_NO_EMPTY
	);
	foreach ($values as $key => $value) {
		if (isset($convert[$value])) {
			$values[$key] = date($convert[$value], $timestamp);
		}
	}
	return implode($values);
}

Installing MongoDB on a Linux distro using SystemD instead of inittab

While trying to get MongoDB working on a Fedora 15 test server I found that there was no way of installing the latest version with SystemD control due to a pre-existing known bug.

To Mongo’s credit there is a file provided in the source under rpm/init.d-mongod, however when it comes to this being used by SystemD, well it just errors.

This was frustrating, but as I was doing it on a VM, I did what any logical person would do. Installed Mongo from the package manager, took a copy of the required SystemD files, reverted to a snapshot, then installed the latest version of Mongo from source.

After this I adapted the mongod.service file to use the correct locations, ensured all the required directories, files and users were present on the system then started the service et voila, working latest version of MongoDB on Fedora 15 with SystemD

the two required files are:
/lib/systemd/system/mongod.service

[Unit]
Description=High-performance, schema-free document-oriented database
After=syslog.target network.target
 
[Service]
Type=forking
User=mongod
Group=mongod
PIDFile=/var/run/mongodb/mongod.pid
EnvironmentFile=/etc/sysconfig/mongod
ExecStart=/usr/local/bin/mongod $OPTIONS run
 
[Install]
WantedBy=multi-user.target

/etc/sysconfig/mongod

Or you can just use the RedHat distro based install script I created:

#!/bin/sh

# MongoDB Version
MONGODB_VER='2.2.2'

# Get all the dependencies up to date
yum -y update
yum -y install scons gcc-c++ glibc-devel

# Get the source
cd /usr/local/src/
wget http://downloads.mongodb.org/src/mongodb-src-r$MONGODB_VER.tar.gz
tar xfz mongodb-src-r$MONGODB_VER.tar.gz
cd mongodb-src-r$MONGODB_VER

# Compile and Install
scons all
scons install

# Create the SystemD dependant files
echo '[Unit]
Description=High-performance, schema-free document-oriented database
After=syslog.target network.target
 
[Service]
Type=forking
User=mongod
Group=mongod
PIDFile=/var/run/mongodb/mongod.pid
EnvironmentFile=/etc/sysconfig/mongod
ExecStart=/usr/local/bin/mongod $OPTIONS run
 
[Install]
WantedBy=multi-user.target' > /lib/systemd/system/mongod.service

echo 'OPTIONS="--quiet -f /etc/mongod.conf"' > /etc/sysconfig/mongod

# Setup the required user and group
useradd -r -U mongod

# Setup the required directories
mkdir -p /var/run/mongodb/
mkdir -p /var/log/mongo/
mkdir -p /var/lib/mongo/
chown mongod:mongod /var/run/mongodb/
chown mongod:mongod /var/log/mongo/
chown mongod:mongod /var/lib/mongo/
chmod 0755 /var/log/mongo/
chmod 0755 /var/run/mongodb/
chmod 0755 /var/lib/mongo

# Start the new service and enable it on boot
systemctl --system daemon-reload
systemctl start mongod.service
systemctl enable mongod.service

New Job

Today was my last day working at my current employers. I’ve been there since May 2007 and had some enjoyable times with a group of people really worth working with, hopefully having made some good friends along the way. I’ve learnt a lot while here both with the corporate environment, being my first long-term employers, and a lot in terms of coding, having written everything from back end systems to the company’s web-sites.

After 5+ years though it was time to up-sticks and find myself a fresh challenge and it’s moving sub-counties in my beloved Yorkshire from South back to West. It was a hard decision to move on after so long, but I feel it’s the right one, and time for a change.

I start at my new employers in just over a week, giving me a nice relaxing week off in-between.

Here’s to the future, different challenges, and a load of new colleagues to get to know.

Installing the default Linux Kernel on a Linode CentOS 6 box

While creating the new web-server for my employers, to replace a Fedora 10 box which gets no security updates, I needed to compile some software from source, meaning I needed the kernel sources.

Since I couldn’t easily obtain these I needed to install the Kernel provided by the distribution rather than the more recent kernel provided by Linode themselves.

The Linode Library provided a way of doing this for CentOS 5 but not for CentOS 6, thus I adapted the provided script for v5 into one that works with CentOS 6 et voila, distro provided kernel.

Here’s the full source available as a gist on github:

### Starting from a fresh CentOS 6 or newer Linode
### Enable the native kernel to boot from pvgrub
### It will autoconfigure itself with each yum update.
### This is adapted from a previous script for CentOS 5.5 found here:
### http://www.linode.com/docs/assets/542-centos5-native-kernel-selinux-enforcing.sh
### Provided via the linode wiki
### https://www.linode.com/docs/tools-reference/custom-kernels-distros/run-a-distributionsupplied-kernel-with-pvgrub#centos-5
### Provided without warranty, although since it should only be run
### on first box build if your box gets broken simply rebuild it

mkdir /boot/grub/

DISTRO_PLATFORM=`uname -p`
AWK_VERSION_MATCH="{if(\$1==\"kernel.$DISTRO_PLATFORM\") print \$2}"
KERNEL_VERSION=`yum -q list kernel | awk "$AWK_VERSION_MATCH"`

### Write template grub.conf
cat > /boot/grub/grub.conf << EOF
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initramfs paths are relative to /boot/, eg.
#          root (hd0)
#          kernel /boot/vmlinuz-version ro root=/dev/xvda
#          initrd /boot/initramfs-version.img
#boot=/dev/xvda
default=0
timeout=3
title CentOS ($KERNEL_VERSION.$DISTRO_PLATFORM)
        root (hd0)
        kernel /boot/vmlinuz-$KERNEL_VERSION.$DISTRO_PLATFORM root=/dev/xvda
        initrd /boot/initramfs-$KERNEL_VERSION.$DISTRO_PLATFORM.img
EOF

ln -s /boot/grub/grub.conf /boot/grub/menu.lst
yum -y install kernel
if [ $? -ne 0 ]; then
    echo "ERROR aborting..."
    exit 1
fi

Tidying HTML source from PHP

In June 2011 I decided the site famous to Leeds United fans leedsfans.org.uk had been down for WAY too long.  It proved an invaluable resource to Leeds United fans on the history of the club over the years, however the sites host and admin Jabba (Jon) had given up on the project for whatever reason with no immediate intention of reviving it.
I found the last copy taken of the site on archive.org and promptly wrote a simple script to scrape all of the content archive.org had stored into a folder so that I could at-least re-host the static content. I did try to contact Jabba to see if I could take over the original domain, but unfortunately I’ve had no response. I then registered the most similar yet cheap domain name I could find (leeds-fans.org.uk) and put all the content I’d obtained back on the web so people could find it again.

Anyway after looking through the source I decided to try and at-least make the HTML valid until I get a chance to eventually re-design and re-launch the site. This is relatively easy with libtidy installed on your system and php compiled with it available.

I wrote a PHP script to go though a folder of static X/HTML pages, run them through libtidy, keep the doctype and save this back to the file.

If such a script will be useful to you here’s the source also available as a gist on github:

<?php
	defined('TIDYDIR_EXTENSION') || define('TIDYDIR_EXTENSION', 'html');
	function tidyDir($directory) {
		$htmlFiles = glob($directory.DIRECTORY_SEPARATOR.'*.'.TIDYDIR_EXTENSION);
		$filenameRegEx = '#^(.+?)\.([^\.]+?)$#';
		$htmlTidy = new tidy();
		foreach ($htmlFiles as $entry) {
			if (preg_match($filenameRegEx, $entry, $matches)) {
				$filename = $matches[1];
				$extension = $matches[2];
				$htmlContents = file_get_contents($entry);
				$doctype = (preg_match('#\A\s*(\<[\s\S]+?\>)[\s\S]*#', $htmlContents, $matches))
							? $matches[1]."\n"
							: '';
				$htmlTidy->parseString($htmlContents);
				if (0 < $htmlTidy->getStatus()) {
					if ($htmlTidy->cleanRepair()) {
						$correctedHTML = $doctype.$htmlTidy->html()->value;
						echo 'saving ',$filename,'.',$extension,"\n";
						if (!file_put_contents($filename.'.'.$extension, $correctedHTML)) {
							echo 'failed saving ',$entry,"\n";
						}
					} else {
						echo 'FAILED TO CLEAN UP ',$entry,"\n";
						die;
					}
				} else {
					echo 'Goody, ',$entry,' is valid html ',"\n";
				}
			}
		}
		$d = dir($directory);
		while (false !== ($entry = $d->read())) {
			if (0 !== strpos($entry, '.') && is_dir($directory.DIRECTORY_SEPARATOR.$entry)) {
				echo 'calling tidyDir on ',$directory,DIRECTORY_SEPARATOR,$entry,"\n";
				tidyDir($directory.DIRECTORY_SEPARATOR.$entry);
			}
		}
	}
	tidyDir(__DIR__);

HTML5 Storage and Objects

Yep, HTML5’s new localStorage and sessionStorage functionality is great, you can store information you want client side, meaning you don’t have to transfer the data via XHRs to the server, or store it in cookies, or in flash storage to keep it between page views.

There is one downside however. Objects, you can’t store them. It only accepts strings, thus I did what everyone’s doing, used JSON to store the object in storage. This however is a bit, well, annoying. You’ve got to remember all the time to run a JSON.parse or JSON.stringify on the data your using or you’ll simply loose it without any errors being thrown

This lead me to simply write a wrapper for both types of storage so I didn’t have to remember wherever I was using them.

Here’s the source also available in a gist on github:

if (typeof HTML5 == 'undefined') {
	var HTML5 = {};
}
/**
 * Wrapper class to deal with easily storing values in local storage
 * without having to constantly use JSON.parse and JSON.stringify everywhere
 * you want to save an object.
 *
 * @param {String} index the base index to use in the localStorage global object
 * @author Tom Chapman
 */
HTML5.localStorage = function(index)
{
	/**
	 * @type {Mixed}
	 * @private
	 */
	var localValues;

	/**
	 * @type {String}
	 * @private
	 */
	var localIndex;

	/**
	 * Class level constructor
	 *
	 * @constructor
	 * @param {String} index
	 * @private
	 */
	var __init = function(index) {
		if (typeof index != 'string' || index === null) {
			throw new Error('A string index must be provided to HTML5.localStorage');
		}
		localIndex = index;
		var currentLocalValue = localStorage.getItem(index);
		if (typeof currentLocalValue != 'undefined' && currentLocalValue !== null) {
			try {
				localValues = JSON.parse(currentLocalValue);
			} catch (err) {
				localValues = currentLocalValue;
			}
		} else {
			localValues = {};
		}
	}(index);

	return {
		/**
		 * Returns all vars or index from the localValues
		 *
		 * @param {String} [index] the index inside the object in use
		 * @return {Mixed}
		 */
		get: function(index) {
			return (typeof index == 'undefined')
					? localValues
					: ((typeof localValues[index] != 'undefined')
							? localValues[index]
							: null);
		},

		/**
		 * Set localValues or an index in localValues
		 *
		 * @param {Mixed} value the value to assign to the object, or if index provided the index inside the object
		 * @param {String} [index] the index inside the object in use
		 * @return {Mixed}
		 */
		set: function(value, index) {
			if (typeof index == 'undefined' || index === null) {
				localValues = value;
			} else {
				if (typeof localValues != 'object') {
					throw new Error();
				}
				localValues[index] = value;
			}
			localStorage.setItem(localIndex, (typeof localValues != 'string' && typeof localValues != 'number')
													? JSON.stringify(localValues)
													: localValues);
		},

		/**
		 * Removes either the whole object from the localStorage or the index provided
		 *
		 * @param {String} [index] the index inside the object in use
		 */
		remove: function(index) {
			if (typeof index == 'undefined') {
				localStorage.removeItem(localIndex);
			} else if (typeof localValues[index] != 'undefined') {
				delete localValues[index];
				localStorage.setItem(localIndex, (typeof localValues != 'string' && typeof localValues != 'number')
													? JSON.stringify(localValues)
													: localValues);
			}
		}
	};
};
if (typeof HTML5 == 'undefined') {
	var HTML5 = {};
}
/**
 * Wrapper class to deal with easily storing values in session storage
 * without having to constantly use JSON.parse and JSON.stringify everywhere
 * you want to save an object.
 *
 * @param {String} index the base index to use in the localStorage global object
 * @author Tom Chapman
 */
HTML5.sessionStorage = function(index)
{
	/**
	 * @type {Mixed}
	 * @private
	 */
	var sessionValues;

	/**
	 * @type {String}
	 * @private
	 */
	var sessionIndex;

	/**
	 * Class level constructor
	 *
	 * @constructor
	 * @param {String} index
	 * @private
	 */
	var __init = function(index) {
		if (typeof index != 'string' || index === null) {
			throw new Error('A string index must be provided to HTML5.sessionStorage');
		}
		sessionIndex = index;
		var currentLocalValue = sessionStorage.getItem(index);
		if (typeof currentLocalValue != 'undefined' && currentLocalValue !== null) {
			try {
				sessionValues = JSON.parse(currentLocalValue);
			} catch (err) {
				sessionValues = currentLocalValue;
			}
		} else {
			sessionValues = {};
		}
	}(index);

	return {

		/**
		 * Returns all vars or index from the sessionValues
		 *
		 * @param {String} [index] the index inside the object in use
		 * @return {Mixed}
		 */
		get: function(index) {
			return (typeof index == 'undefined')
					? sessionValues
					: ((typeof sessionValues[index] != 'undefined')
							? sessionValues[index]
							: null);
		},

		/**
		 * Set sessionValues or an index in sessionValues
		 *
		 * @param {Mixed} value the value to assign to the object, or if index provided the index inside the object
		 * @param {String} [index] the index inside the object in use
		 * @return {Mixed}
		 */
		set: function(value, index) {
			if (typeof index == 'undefined' || index === null) {
				sessionValues = value;
			} else {
				if (typeof sessionValues != 'object') {
					throw new Error();
				}
				sessionValues[index] = value;
			}
			sessionStorage.setItem(sessionIndex, (typeof sessionValues != 'string' && typeof sessionValues != 'number')
													? JSON.stringify(sessionValues)
													: sessionValues);
		},

		/**
		 * Removes either the whole object from the sessionStorage or the index provided
		 *
		 * @param {String} [index] the index inside the object in use
		 */
		remove: function(index) {
			if (typeof index == 'undefined') {
				sessionStorage.removeItem(sessionIndex);
			} else if (typeof sessionValues[index] != 'undefined') {
				delete sessionValues[index];
				sessionStorage.setItem(sessionIndex, (typeof sessionValues != 'string' && typeof sessionValues != 'number')
													? JSON.stringify(sessionValues)
													: sessionValues);
			}
		}
	};
};