Second Drawer

Webfonts with Nginx

2010-08-21T17:00:00Z

I just made some changes on my site to use webfonts for the main menu and any Lahu language sections of the site. I have chosen to serve the font files directly from my server as most of the other webfont servers don’t provide a good Unicode 5 version of any font required for Lahu display.

This does mean that I needed to update my Nginx mime types to serve the font files with the correct mime type. I added the following to my mime.types file in the Nginx config directory:

application/x-font-ttf                ttf;
font/opentype                         otf;
application/vnd.ms-fontobject         eot;

and reloaded Nginx. Now they are served correctly instead of ‘application/octet-stream’.

Logwatch, socklog and svlogd

2010-08-20T17:00:00Z

Runit is a great package that provides process supervision and can be used as a replacement for ‘sysvinit’. The author of the package has also made a syslogd replacement called socklog which works well with runit.

Logwatch is a collection of scripts that can be used to notify you of changes to your logs. Out of the box on a Debian machine, though, logwatch will not be able to read all the logs spat out by socklog. The default setup produces logs that have the log facility and level appended to the log entry and the hostname removed:

authpriv.info: Aug 21 20:45:44 sshd: pam_unix(sshd...

I recently had to modify Logwatch for some other reasons and thought I would post my existing scripts here for others to see also. These should be placed in the /etc/logwatch directory so they are not overwritten on package upgrades.

conf/logfiles/messages.conf

LogFile = 
Archive = 
LogFile = socklog/*/current
Archive = socklog/*/@*

*RemoveFacility
*ExpandRepeats
*RemoveService = talkd,telnetd,inetd,nfsd,/sbin/mingetty
*ApplyStdDate

conf/logfiles/syslog.conf

Logfile = 
Archive = 
LogFile = socklog/*/current
Archive = socklog/*/@*

*RemoveFacility
*ExpandRepeats
*RemoveService = talkd,telnetd,inetd,nfsd,/sbin/mingetty
*ApplyStdDate

conf/logfiles/secure.conf

LogFile = 
Archive = 
LogFile = socklog/auth/current
Archive = socklog/auth/@*

*RemoveFacility
*ExpandRepeats
*ApplyStdDate

conf/logfiles/maillog.conf

LogFile =
LogFile = socklog/mail/current
Archive =
Archive = socklog/mail/@*

*RemoveFacility

conf/logfiles/kernel.conf

LogFile = socklog/kern/current
Archive = socklog/kern/@*
*RemoveFacility
*ExpandRepeats
*ApplyStdDate

There may be others needed for your own system. All these configurations reference the script RemoveFacility does the bit of removing the facility and level from each log entry.

scripts/shared/removefacility

#!/usr/bin/perl

# removes the facility and level from logfiles generated by socklog
if ( $ENV{'LOGWATCH_DEBUG'} > 4 ) {
   print STDERR "DEBUG: Inside RemoveFacility\n";
}

while (defined($ThisLine = <STDIN>)) {
   $ThisLine =~ s/^[a-z]+.[a-z]+: //i;
   print $ThisLine;
}

and also the onlyhost script which is used to restrict entries by host but since the hostname is not even there we just return them all:

scripts/shared/onlyhost

#!/usr/bin/perl

use strict;

my $line;
while (defined($line = <STDIN>)) {
    print $line;
}

Rack map and subdomains

2010-07-03T17:00:00Z

The situation

You are using subdomains for locales such as en.example.com and th.example.com. Your app is Rack based and is behind Nginx with a ‘wildcard’ server_name directive such as:

server {
  server_name .example.com;
  ...
}

You are also using Rack’s map abilities to have two different applications mounted on different paths on your domain. This could be done like this in your config.ru file:

map '/users' do
  run Example::Admin
end

map '/' do
  run Example::Public
end

The problem

If the URL http://th.example.com is hit, Nginx will set ‘HTTP_HOST’ as ‘th.example.com’ but ‘SERVER_NAME’ will be ‘example.com’. What his means is that tests within Rack fail and result in neither of the above ‘maps’ matching so a 404 is returned.

A solution

In Rack apps you are able to modify the environment. This means that one way to fix this is to simply set the environment’s’ SERVER_NAME’ to equal ‘HTTP_HOST’ by including something like this in your config.ru file, before your map calls:

module Rack
  class Blah
    def initialize app
      @app = app
    end

    def call env
      env['SERVER_NAME'] = env['HTTP_HOST']
      @app.call env
    end
  end
end
use Rack::Blah

map '/users' do
...etc.

Optimise your images for performance and consistency

2010-05-20T17:00:00Z

In an effort to reduce your website’s page size, whether it be to just provide a better experience for the user or to work inline with Google’s plans to factor page speed into rankings, you will inevitably get to the point of dealing with your site’s images.

In many sites the images take up a large proportion of the site’s weight (in bytes). So getting the most out of them is very important. One tool that is essential in this regard is pngcrush.

pngcrush

Pngcrush is a tool that lossless reduces the size of PNG files. PNG are used mainly for styling the website and so are vital to the look (and often the readability) of the site itself. JPEGs are great for photographs which need no restriction on the number of colours. Compression of JPEGs are dealt with differently. Few modern designers use GIFs anymore now that IE supports alpha transparency in PNGs, so we will assume that we are only dealing with PNGs.

Pngcrush uses a number of techniques to reduce the size of your PNGs such as reducing the palette or applying different compression algorithms to the image data.

An example

A good example of the size difference is the menu image used on this site. An initial save of the PNG image was 159Kb. This same image when run through pngcrush resulted in a size of 14.5Kb, with no reduction in quality. It is a simple as this:

pngcrush infile.png outfile.png

or trying more methods to reduce size:

pngcrush -brute infile.png outfile.png

For my site, I am already using the sprite technique to combine images for my main menu so together I am able to reduce, not only the total amount to be downloaded, but also the number of HTTP requests needed to create my menu.

Alternatives

There are several other alternatives to pngcrush such as its fork, optiPNG. This does pretty much the same job.

Colour consistency

One thing that pngcrush does that others do not yet seem to do is reduce the file size by removing the colour correction (or colour profile) data from the file. This can sometimes reduce the size by up to 40%. What this also means is that your images will appear more consistently across browsers. How so?

Many image manipulation programs will place a colour profile within the image to enable different systems to display the image as close as possible to the original. Firefox, for instance, will use these profiles to adjust the colours before displaying them. Google Chrome, on the other hand, will not. What this means is that some images on your site will appear differently in Firefox and Chrome. It could also mean that colour values in CSS style sheets will not match those colours, while using identical codes, in PNG files.

The command line to remove all colour profiles from your PNG is as follows:

pngcrush -rem gAMA -rem cHRM -rem iCCP -rem sRGB infile.png outfile.png

Conclusion

By removing the colour profile data from your images, you not only reduce the file size but you force browsers (such as Firefox) to make do without the profile and so render them more consistently, with other systems and with CSS colour values. A faster and more colour accurate site.

Rack locale setter middleware

2010-04-22T17:00:00Z

For a project I am currently working on I am using the nice i18n library r18n which provides i18n for Rails, Sinatra and most general Ruby projects. It also is reasonably ORM agnostic and so I am using it quite easily with DataMapper.

I have decided to use a subdomain to specify the locale rather than having it be part of the path. So instead of this:

http://example.com/en/foo/

I have this:

http://en.example.com/foo/

I find this makes it easier by being able to specify relative paths everywhere that don’t need to have the locale played with. Switching the locale for the site is also just a matter of changing the domain.

That said, r18n expects the locale to be placed in the ‘params’ array for Sinatra and I thought a simple Rack middleware would do this well. So here it is:

module Rack
  class LocaleSetter
    def initialize(app)
      @app = app
    end

    def call(env)
      req = Rack::Request.new(env)
      if m = req.host.match(/^(?:www\.)?([a-z]{2})\./)
        locale = m[1]
      else
        locale = 'en'
      end
      req.params['locale'] ||= locale
      @app.call env
    end
  end
end

Due to many peoples’ habit of putting ‘www’ at the start of the domain, this should account for en.example.com and www.en.example.com. It also allows for the locale to be specified in the query string. So en.example.com/foo?locale=th will set the locale to ‘th’.

There are probably others out there but due to the simplicity of Rack based apps it is very easy to write, and adjust, your own.