Kategoriarkiv: ruby rails

Make testing possible with Rails and PostGIS

Inspired by this SO question:

http://stackoverflow.com/questions/8718112/rails-not-fully-compatible-with-postgresqlpostgis-geometry-data-type

I’ve had a lot of trouble recently with Rails and PostGIS. One of the main problem arised with testing. When you run Rails tests (or RSpec for that matter) the test database is always dropped and a new is created. The problem with this is, that the PostGIS extension is not created in this new database. Therefore your schema-load will fail, if you have any postgis-specific fields/indices.

The solution to this is to use a PostgreSQL database template – and of course the activerecord-postgis-adapter found here: https://github.com/dazuma/activerecord-postgis-adapter

First the template:

You need a PostgreSQL template with PostGIS functions support.

Create a template database:

$ psql -U postgres
> CREATE DATABASE template_postgis WITH TEMPLATE=template1 ENCODING='UTF8';
> \c template_postgis;
> CREATE LANGUAGE plpgsql;

Load necessary PostGIS functions into template (your SQL files may be located in a different path, but they come with the PostGIS installation):

$ psql -U postgres -f  /usr/share/postgresql/9.1/contrib/postgis-2.1/postgis.sql template_postgis
$ psql -U postgres -f /usr/share/postgresql/9.1/contrib/postgis-2.1/spatial_ref_sys.sql template_postgis
$ psql -U postgres -f  /usr/share/postgresql/9.1/contrib/postgis-2.1/topology.sql template_postgis

Set database as template and grant permissions:

$ psql -U postgres template_postgis
> UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
> GRANT ALL ON geometry_columns TO PUBLIC;
> GRANT ALL ON spatial_ref_sys TO PUBLIC;
> GRANT ALL ON topology TO PUBLIC;

Now the adapter:

Add this gem to your Gemfile and run bundle:

gem 'activerecord-postgis-adapter'

After that add the adapter and the template to your config/database.yml like this:

development:
  adapter: postgis
  template: template_postgis
  database: mydb_development

test:
  adapter: postgis
  template: template_postgis
  database: mydb_test

And that should do the trick! Test it by running rake db:schema:dump and rake db:test:prepare.

Let me know, if you get any additional problems regarding this.

Bookmark and Share

Make Rails routing case insensitive with route_downcaser gem

A long time ago I solved a nagging issue concerning Rails’ handling of url casing (or rather: non-handling). I posted the resulting code here.

I’ve realised that a lot of people are actually using this solution, so I finally took the time to make it into a fully-tested gem.

The gem is released on RubyGems here.

All documentation (such as is needed) can be found on the gem’s github page.

All you actually need to do is adding the gem to your gemfile, run bundle, and the gem takes care of the rest. No config or initialization code needed, it’s all baked into the gem.

Enjoy! :-)

Bookmark and Share

My favorite Apache setup for a Rails application

You think 1 site equals 1 Apache Virtual Host? Think again, and read on to figure out why not.

For every Rails application that I deploy into production mode, I usually creates 3 Virtual Host entries in Apache. To illustrate, here is the setup for our dog-site www.gipote.dk:

Note: This setup is using Passenger (aka. mod_rails). If you are not already using it for your Rails production environment, you really should do so. It’s the best thing happened since Rails itself.


    ServerName www.gipote.dk

    DocumentRoot /var/www/rails/gipote_production/public

    RailsEnv production



    ServerName  asset0.gipote.dk
    ServerAlias asset1.gipote.dk
    ServerAlias asset2.gipote.dk
    ServerAlias asset3.gipote.dk

    DocumentRoot /var/www/rails/gipote_production/public

    PassengerEnabled off



    ServerName gi-pote.dk
    ServerAlias *.gi-pote.dk
    ServerAlias *.gipote.dk

    RewriteEngine on
    RewriteRule ^(.*) http://www.gipote.dk$1 [R=301]

    PassengerEnabled off

The *=”" inside my virtual host declations is the WordPress SyntaxHighlighter playing tricks on me. It should just be an *

Some explanation is required…

First of all, I want one and only one entry into our site. This originally came from making it easier to setup Google Analytics (you need to tweak the tracking code to support multiple domains). But it seems that it also improves our SEO rankings – so more power to us.

So the first virtual host only accepts www.gipote.dk. The last virtual host accepts all other variations on gipote.dk (except a few – I’ll come to that) as well as an entirely different domain gi-pote.dk. The last virtual host redirects all these requests to www.gipote.dk.

The last virtual host relly must be placed at the bottom, since it contains wildcards. These wildcards – *.gipote.dk – will catch all hostnames not specified in any virtual host above it. Position is vital.

Now to the second virtual host. It requires a bit more explaining…

Internet Explorer and Firefox has a limit when fetching a webpage concerning concurrent connections to the same host. They will only have 2 open connections to a single host at a time. So if you have a page with many external files – or assets (css, js, images), you may find, that it is slow to load, even if you have a powerful server. The bottleneck is at the client this time.

Rails have a fix for this: Every asset specified in the ERB by the asset helpers image_tag(), stylesheet_link_tag(), etc. Can automatically prepend the URL with an asset host. You need to define the asset host in your config/environments/production.rb file:

config.action_controller.asset_host = "http://asset%d.gipote.dk"

The %d is important. This will make Rails’ asset helpers prepend either of


http://asset0.gipote.dk


http://asset1.gipote.dk


http://asset2.gipote.dk


http://asset3.gipote.dk

to any asset url. It will use exactly these 4 variants, and they will be distributed (almost) evenly out on your assets.

So image_tag(‘logo.png’) will become http://asset2.gipote.dk/images/logo.png (or 0, 1, 3).

The cool thing about this is, that the aforementioned browsers will open 2 connections for each host. So instead of fetching 2 assets concurrently, your browser will now fetch 8 at the same time.

So we need our Apache to deal with this. We cannot rely on the last virtual host to do this, because it will simply redirect to www.gipote.dk. And we don’t want to add these 4 hostnames as alias’ to the top virtual host, because we don’t want our users to be able to access our Rails application by any other hostname than www.gipote.dk.

So I create the second virtual host. Put in all 4 hostnames, point it to my Rails public folder. Oh – and for good performance measure, I disable Passenger (PassengerEnabled off). Otherwise the users would still be able to use the hostnames to access our application.

Bookmark and Share

How to make Rails routing case-insensitive

Update 2013-01-06: I have now converted this into a gem: Read more

Update 2011-03-10: Brian Marquis pointed out, that I’ve missed showing a good way to actually load the .rb file containing the middleware. This has now been added to the article.

Update 2010-11-06: Rails 3 has changed its routing mechanism. One of the changes involves which environment variable is used as the source for routing. In Rails 2.3.x it was REQUEST_URI, in Rails 3 it is PATH_INFO. I have changed the middleware code to take care of both versions.

At our dog-site www.gipote.dk we have a shop selling dog-tags. The URL for this shop is www.gipote.dk/hundetegn (hundetegn = dog-tag in danish). However one of our marketing-guys have a tendency to capitalize the word “hundetegn”, so that URL reads www.gipote.dk/Hundetegn (with a capital H).

Rails will yield a 404 NOT FOUND on this, simply because it is case sensitive. The Rails core team doesn’t seem to want this changed, but I found a neat way of doing it myself: Using the new Rack Middleware framework.

First the solution:

Create a file called downcase_route_middleware.rb and put it in RAILS_ROOT/lib or wherever you think middleware files ought to go. Fill it with this piece of code:

class DowncaseRouteMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    # Rails 2.3.x routing use REQUEST_URI
    uri_items = env['REQUEST_URI'].split('?')
    uri_items[0].downcase!
    env['REQUEST_URI'] = uri_items.join('?')

    # Rails 3.x routing use PATH_INFO
    env['PATH_INFO'] = env['PATH_INFO'].downcase

    @app.call(env)
  end
end

Now to telling Rails to use this new middleware class. This differs a bit in Rails 2 and 3.

Adding the middleware in Rails 3.x:

Open config/application.rb and find the lines

module YourAppName
  class Application < Rails::Application
    ..bunch of code..
  end
end

Add the middleware code like this:

module YourAppName
  class Application < Rails::Application
    config.autoload_paths << "#{config.root}/app/middleware"
    config.middleware.use "DowncaseRouteMiddleware"

    ..bunch of code..
  end
end

Adding the middleware in Rails 2.3.x:

Open up config/environment.rb, and find the line

Rails::Initializer.run do |config|

  ...a lot of config code...
end

Add the middleware code, so it looks like this afterwards

Rails::Initializer.run do |config|
  config.autoload_paths << "#{config.root}/app/middleware"
  config.middleware.use "DowncaseRouteMiddleware"

  ...a lot of config code...
end

Update: The line:
config.autoload_paths << "#{config.root}/app/middleware"
is my general way of using middleware. I always places all my middleware classes in a folder called app/middleware. The line above enables Rails to automatically search the autoload-path for a file that matches the middleware class used (i.e. "downcase_route_middleware.rb").

If this is the only middleware class that you use, you can simply do a
require "downcase_route_middleware.rb""
somewhere in your config initializers. I just find the above solution more elegant.

Restart your rails application, and it will now accept all kinds of casing on the routing part.

Note: This code only downcases the part of the URI containing namespace, controller, action, and id. It does not touch the querystring parameters, and for a good reason too: The parameter values could contain some context in their casing.

Now for the explanation:

Rails 2.3 introduced Rack middleware. A lot of people have already explained this concept, so instead of going into details, I suggest that you read Pratik's explanation on Rails Guides and Ryan's Railscast on the subject.

Basically, my middleware class gets access to the environment hash and passes it on to the next middleware class on the stack. In between I get to modify the nessecary data. In this case it's the REQUEST_URI (In Rails 3 it's PATH_INFO), since ActionController::Dispatcher uses the URI from here to determine the correct route. By converting the URI to lowercase, ActionController::Dispatcher gets the correct path no matter the case entered by the user.

Bookmark and Share

The woes of libodbc-ruby1.8 and Debian + Ubuntu

Update 05-05-2010: This issue still exists on the new Ubuntu 10.04 LTS (Lucid Lynx). The dist-upgrade will override any version locks that you have made on libodbc-ruby1.8 and install version 0.9997-2 (and lock it). So after a dist-upgrade you will have to remove libodbc-ruby1.8 and install the older version again.

I have the dubious honor of developing on a Rails application, that runs on top of Microsoft SQL Server. This has given some headaches at our most recent system upgrades.

My development machine runs Ubuntu 9.10 (Karmic Koala). Our servers run Debian 5.0 (Lenny).

If you run any of these OS versions and encounter an error like the following:

dbd-odbc-0.2.4/lib/dbd/odbc/driver.rb:36:in `connect':DBI::DatabaseError: INTERN (0) [RubyODBC]Cannot allocate SQLHENV

then help can be found here.

The problem on both OS’es lies in the package libodbc-ruby1.8. But it is actually stranger than you might expect, which I will explain below.

Ubuntu first

We’ll start off with Ubuntu. Fire up Synaptic Package Manger and search for libodbc-ruby1.8. You will see, that the distribution package is versioned 0.9997-2. You need to uninstall this and install version 0.9995-1 instead. You can download version 0.9995-1 here: http://packages.ubuntu.com/hu/hardy/i386/libodbc-ruby1.8/download

Now install it with apt-get and… problem solved. Please note however that your automatic package update will revert to version 0.9997-2 unless you uncheck it before running the update.

Update 19-02-2010: This is also an issue on the 64-bit version of Ubuntu 9.10. Here’s a link where you can download the 0.9995-1 .deb package for both 32-bit AND 64-bit. Thanks Goran.
http://mirrors.kernel.org/ubuntu/pool/universe/libo/libodbc-ruby

Debian next

On Debian it’s the other way around. Debian comes installed with libodbc-ruby1.8 version 0.9995-1 and you need to upgrade this to 0.9997-2. This package is currently only found in unstable, but you can download it here: http://packages.debian.org/sid/i386/libodbc-ruby1.8/download

Install it with apt-get, overriding the previous package.

Hope this saves you from the days of work that I invested in this particular oddity. :-)

Bookmark and Share

Reclaim control of your application with Cucumber

As I’ve written earlier(danish), I’ve arrived rather late to the test-driven development-train. As a result I have a large application, developed in Rails, but without utilizing the test framework. Granted I have lately added some unit-tests, but in the whole, I would deem the application uncovered by tests.

Such a beast is difficult to handle. To upgrade it to support the latest Rails version (2.3.5 in this time of writing) is like jumping of a plane and hope to catch a floating parachute on your way down. On the other hand, writing unit-tests and functional tests to cover all current functionality is a huge task, that is discouraging simply by its presence. To put it short: The application has grown beyond my control.

So how to tackle this situation? I want the entire application (or most of at least) covered by test, but I will not be able to spare the next 3 months doing nothing but writing unit and functional tests.

Enter Cucumber.

With Cucumber I am able to create happy paths for all the use cases in my application. I know how each area of the application is supposed to be used, and by outlining these uses in Cucumber features and scenarios, I get a basic coverage of all functionality.

This is a very top-down approach. I don’t get to test each small functionality separately. And even the Cucumber guys encourage coders to write more than just happy paths (adding also scenarios that are supposed to test for error messages). But at least, if something breaks, I will know about it. So when I change something in my application, I run all my cucumber features. If one of the scenarios fail, I can now focus on writing more detailed tests for this particular part of the application.

A big thanks to Aslak Hellesøy and friends for helping me reclaim control of my application!

Bookmark and Share

Simple Amazon S3 file synchronize for Rails application (without the need of Capistrano)

So you have a nice Rails-app running and you want all your static assets to reside on Amazon S3?

There are other good solutions on the web, but most of what I’ve seen requires that you also use Capistrano for deploying your app. If you have not yet mastered that beast (just like me), you will like this little rake task, that I’ve made.

Just copy/paste this code into your favorite editor, change the 4 parameters in the top, and save it in lib/tasks/aws.rake.

Then you can do this:

# rake aws:sync

This will upload all folders and files from your public folder into your Amazon S3 bucket.

The script creates a file called .aws_cache in the root of your application folder. This is used so that additional calls to rake aws:sync only uploads new and modified files.

If you have already uploaded your files before applying this script, you can start by calling

# rake aws:build_cache

This will build the local cachefiles without uploading anything to Amazon S3.

Here’s the code:

require 'find'
require 'digest/md5'
require 'yaml'

#########################################################
# Configuration - these lines are all you need to edit
ACCESS_KEY_ID     = "your-own-aws-access-key"
SECRET_ACCESS_KEY = "and-the-secret-key"
BUCKET             = "name-of-your-bucket"

# If you have any subfolders inside "public", that you do not want to place on AWS, list them here
IGNORE_FOLDERS    = %w(upload UserFiles videos)
#########################################################

class AwsCache
  def initialize
    @filename = ".aws_cache"
    load
  end

  def clear
    @cache = {}
  end

  def load
    clear
    if File.exists?(@filename)
      @cache = YAML::load_file(@filename)
    end
  end

  def save
    File.open(@filename, "w") do |f|
      f.puts @cache.to_yaml
    end
  end

  def add(path)
    puts "add to cache: #{path}"
    @cache[path] = checksum(path)
  end

  def equal?(path)
    c = checksum(path)
    @cache[path] == c
  end

  protected
  def checksum(path)
    Digest::MD5.file(path).hexdigest
  end
end

namespace :aws do
  desc "Synchronize public folder"
  task :sync => :environment do
    init

    AWS::S3::Base.establish_connection!(
      :access_key_id     => ACCESS_KEY_ID,
      :secret_access_key => SECRET_ACCESS_KEY
    )

    bucket = AWS::S3::Bucket.find(@bucket)

    loop_folder()

    @aws_cache.save
  end

  desc "Rebuild local cache file"
  task :build_cache => :environment do
    init

    absolute_folder = File.join(@base_folder, "")

    Find.find(absolute_folder) do |path|
      if FileTest.directory?(path)
        if File.basename(path)[0] == ?.
          Find.prune
        elsif IGNORE_FOLDERS.include?(File.basename(path))
          Find.prune
        end
      else
        @aws_cache.add(path)
      end
    end

    @aws_cache.save
  end
end

def init
  @base_folder = File.join(RAILS_ROOT, "public")
  @bucket      = BUCKET
  @aws_cache   = AwsCache.new
end

def loop_folder(folder = "")
  absolute_folder = File.join(@base_folder, folder)

  total_size = 0

  Find.find(absolute_folder) do |path|
    if FileTest.directory?(path)
      if File.basename(path)[0] == ?.
        Find.prune
      elsif IGNORE_FOLDERS.include?(File.basename(path))
        Find.prune
      else
        next
      end
    else
      s3_store(path)
    end
  end
end

def s3_store(path)
  s3_path = path.gsub("#{@base_folder}/", '')

  if transfer?(path)
    AWS::S3::S3Object.store(
      s3_path,
      open(path),
      @bucket,
      :access => :public_read)

    puts "Stored into AWS: #{AWS::S3::S3Object.url_for(s3_path, @bucket)[/[^?]+/]}"
  end
end

def transfer?(path)
  return false if @aws_cache.equal?(path)

  @aws_cache.add(path)
  return true
end

Feel free to contact me, if you have any questions, problems, suggestions for improvement.

Bookmark and Share

Thinking about searching in your app?

I’ll cut to the chase. There is really only one good search engine for Rails applications at the moment of this writing.

It’s called ThinkingSphinx, it’s fast, it’s reliable, and it has the easiest Rails implementation, that I have seen so far.

There is really no need for me to do a tutorial here, since Pat Allan has already done a great one himself. Read it here.

I use ThinkingSphinx in production on my Gipote site. Before that I’ve been with Ferret (not good in a multi-mongrel production site), and UltraSphinx (gave me great problems when indexing multiple models). After using ThinkingSphinx as my search engine, all pain has gone.

Bookmark and Share

Hooray – now full-time on Rails

I have decided to sell my company and get a job at a company developing solely in Ruby on Rails.

This is really the best decision, that I’ve done for myself AND my family for a very long time. The career as self-employed was really starting to tear me down.

So now – almost 2 years since I complained about the desire to work only with Rails but having the economical need to do PHP – I have finally shedded the PHP load and am now a full-time Rails developer.

Dear PHP: Thank you for 9 great years. It has been fun, but now I really must move on.

At the same time, I decided to change the name of my blog. I will be having a lot of travelling by train each day, and although I will do a lot of coding on these trips, it is also a perfect place, to write down the thoughts and ideas coming up during the day.

Bookmark and Share