Reclaim control of your application with Cucumber

As I’ve written earlier(danish), I’ve arrived rather late to the test-driven development-train. As a result I have a large application, developed in Rails, but without utilizing the test framework. Granted I have lately added some unit-tests, but in the whole, I would deem the application uncovered by tests.

Such a beast is difficult to handle. To upgrade it to support the latest Rails version (2.3.5 in this time of writing) is like jumping of a plane and hope to catch a floating parachute on your way down. On the other hand, writing unit-tests and functional tests to cover all current functionality is a huge task, that is discouraging simply by its presence. To put it short: The application has grown beyond my control.

So how to tackle this situation? I want the entire application (or most of at least) covered by test, but I will not be able to spare the next 3 months doing nothing but writing unit and functional tests.

Enter Cucumber.

With Cucumber I am able to create happy paths for all the use cases in my application. I know how each area of the application is supposed to be used, and by outlining these uses in Cucumber features and scenarios, I get a basic coverage of all functionality.

This is a very top-down approach. I don’t get to test each small functionality separately. And even the Cucumber guys encourage coders to write more than just happy paths (adding also scenarios that are supposed to test for error messages). But at least, if something breaks, I will know about it. So when I change something in my application, I run all my cucumber features. If one of the scenarios fail, I can now focus on writing more detailed tests for this particular part of the application.

A big thanks to Aslak Hellesøy and friends for helping me reclaim control of my application!

Simple Amazon S3 file synchronize for Rails application (without the need of Capistrano)

So you have a nice Rails-app running and you want all your static assets to reside on Amazon S3?

There are other good solutions on the web, but most of what I’ve seen requires that you also use Capistrano for deploying your app. If you have not yet mastered that beast (just like me), you will like this little rake task, that I’ve made.

Just copy/paste this code into your favorite editor, change the 4 parameters in the top, and save it in lib/tasks/aws.rake.

Then you can do this:

# rake aws:sync

This will upload all folders and files from your public folder into your Amazon S3 bucket.

The script creates a file called .aws_cache in the root of your application folder. This is used so that additional calls to rake aws:sync only uploads new and modified files.

If you have already uploaded your files before applying this script, you can start by calling

# rake aws:build_cache

This will build the local cachefiles without uploading anything to Amazon S3.

Here’s the code:

require 'find'
require 'digest/md5'
require 'yaml'

#########################################################
# Configuration - these lines are all you need to edit
ACCESS_KEY_ID     = "your-own-aws-access-key"
SECRET_ACCESS_KEY = "and-the-secret-key"
BUCKET             = "name-of-your-bucket"

# If you have any subfolders inside "public", that you do not want to place on AWS, list them here
IGNORE_FOLDERS    = %w(upload UserFiles videos)
#########################################################

class AwsCache
  def initialize
    @filename = ".aws_cache"
    load
  end

  def clear
    @cache = {}
  end

  def load
    clear
    if File.exists?(@filename)
      @cache = YAML::load_file(@filename)
    end
  end

  def save
    File.open(@filename, "w") do |f|
      f.puts @cache.to_yaml
    end
  end

  def add(path)
    puts "add to cache: #{path}"
    @cache[path] = checksum(path)
  end

  def equal?(path)
    c = checksum(path)
    @cache[path] == c
  end

  protected
  def checksum(path)
    Digest::MD5.file(path).hexdigest
  end
end

namespace :aws do
  desc "Synchronize public folder"
  task :sync => :environment do
    init

    AWS::S3::Base.establish_connection!(
      :access_key_id     => ACCESS_KEY_ID,
      :secret_access_key => SECRET_ACCESS_KEY
    )

    bucket = AWS::S3::Bucket.find(@bucket)

    loop_folder()

    @aws_cache.save
  end

  desc "Rebuild local cache file"
  task :build_cache => :environment do
    init

    absolute_folder = File.join(@base_folder, "")

    Find.find(absolute_folder) do |path|
      if FileTest.directory?(path)
        if File.basename(path)[0] == ?.
          Find.prune
        elsif IGNORE_FOLDERS.include?(File.basename(path))
          Find.prune
        end
      else
        @aws_cache.add(path)
      end
    end

    @aws_cache.save
  end
end

def init
  @base_folder = File.join(RAILS_ROOT, "public")
  @bucket      = BUCKET
  @aws_cache   = AwsCache.new
end

def loop_folder(folder = "")
  absolute_folder = File.join(@base_folder, folder)

  total_size = 0

  Find.find(absolute_folder) do |path|
    if FileTest.directory?(path)
      if File.basename(path)[0] == ?.
        Find.prune
      elsif IGNORE_FOLDERS.include?(File.basename(path))
        Find.prune
      else
        next
      end
    else
      s3_store(path)
    end
  end
end

def s3_store(path)
  s3_path = path.gsub("#{@base_folder}/", '')

  if transfer?(path)
    AWS::S3::S3Object.store(
      s3_path,
      open(path),
      @bucket,
      :access => :public_read)

    puts "Stored into AWS: #{AWS::S3::S3Object.url_for(s3_path, @bucket)[/[^?]+/]}"
  end
end

def transfer?(path)
  return false if @aws_cache.equal?(path)

  @aws_cache.add(path)
  return true
end

Feel free to contact me, if you have any questions, problems, suggestions for improvement.

Thinking about searching in your app?

I’ll cut to the chase. There is really only one good search engine for Rails applications at the moment of this writing.

It’s called ThinkingSphinx, it’s fast, it’s reliable, and it has the easiest Rails implementation, that I have seen so far.

There is really no need for me to do a tutorial here, since Pat Allan has already done a great one himself. Read it here.

I use ThinkingSphinx in production on my Gipote site. Before that I’ve been with Ferret (not good in a multi-mongrel production site), and UltraSphinx (gave me great problems when indexing multiple models). After using ThinkingSphinx as my search engine, all pain has gone.

Hooray – now full-time on Rails

I have decided to sell my company and get a job at a company developing solely in Ruby on Rails.

This is really the best decision, that I’ve done for myself AND my family for a very long time. The career as self-employed was really starting to tear me down.

So now – almost 2 years since I complained about the desire to work only with Rails but having the economical need to do PHP – I have finally shedded the PHP load and am now a full-time Rails developer.

Dear PHP: Thank you for 9 great years. It has been fun, but now I really must move on.

At the same time, I decided to change the name of my blog. I will be having a lot of travelling by train each day, and although I will do a lot of coding on these trips, it is also a perfect place, to write down the thoughts and ideas coming up during the day.

Tabs be damned

– at least that seems to be a clear oppinion of the Rails community.

Okay so I lied to you in the last post. I’ve been working on my first real Rails project for quite some time now, so instead of a realtime diary, I will be reflecting upon the major issues, that I went through (eg. “how to make a multiple-page wizard-like create controller”). The reason being, that I haven’t been allowed to speak in public about the project until now – we will release it in the coming week.

But I will break the cronology by starting with one of my latest issues:

Ruby on Rails does not allow tab characters for indentation in RHTML templates

Have you ever tried to get a wierd “SyntaxError Exception” when calling a page in your Rails application? There is nothing wrong with the inlined Ruby code.

It turns out, that if you indent your RHTML code with tab characters instead of spaces, the ActionViewer will report a compile error when evaluating your template.

I know, accept, and also appreciate, that Rails is “oppinionated” software. But this is really ridiculous. IMHO tabs are the only way to go – combined of course with spaces for alignment.

This has been an issue at least since Rails 1.1 (I haven’t been on Rails before that), and I found somebody mentioning a workaround then.

So I patched Rails and worked happily ever after – that is until yesterday, when I decided I wanted to release the app with Rails 1.2 instead of 1.1 (better to work out the knots before going live).

I was slammed with RHTML Syntax Errors again. And I had completely forgotten about this little patch, which was, of course, now overruled by the new version of Rails.

It turned out (after much debugging), that the ActionView has been recoded quite a lot, so the previous patch could not be applied in the same place.

But here is what you should do:

1) In Ruby’s lib folder, find the file:
rubygems1.8gemsactionpack-1.13.1libaction_viewbase.rb

2) From line 504:

def compile_template(extension, template, file_name, local_assigns)
render_symbol = assign_method_name(extension, template, file_name)
render_source = create_template_source(extension, template, render_symbol, local_assigns.keys)

add these two lines just in the top of the method:

def compile_template(extension, template, file_name, local_assigns)
# convert all tab-characters to space-indentation (4 spaces)
template = template.gsub(/t/, ” “) unless template.nil?

render_symbol = assign_method_name(extension, template, file_name)
render_source = create_template_source(extension, template, render_symbol, local_assigns.keys)

Now I only need to find out, how I can format my code in Blogger… 😉

Why I must use Rails but cannot abandon PHP

I think I’m an ordinary guy.

At least in programming terms, I’m pretty ordinary. Over the last year, a lot of people have been talking about, how they would like to switch permanently to Ruby on Rails, but they can still only use it for spare-time projects, since their company/customers/etc. are very deep into PHP/.NET/J2EE/whatever technology in existing projects.

I definetely fall into this category, which then – through the philosophy that “ordinary” is made up by the most people – makes me just like everyone else.

I have my own programming company in Denmark. We used to call ourselves a “webbureau”, but skipped the last part, since now only 20% of our projects are plain homepages with or without a CMS. The rest are real application-programming, although mostly with a webbased interface.

Most of our projects are developed in PHP. That means we have a lot of finished solutions for our customers in PHP, which again explains why it will be quite costly to abandon this language alltogether. Besides, I’ve come to like PHP, since it saved me from the IMHO hideous ASP/VBScript, that I used before (we’re back in ’98).

I’ve been a long way around. As a genuine C/C++ hacker, I’ve always had this urge to experiment with (for me) new things. This have included (in the past) Delphi, Java, and lately for web-development: Perl and Python (and some of its web-frameworks, inclding Zope and ). But always seeking back to PHP when it came to web-development. And my customerbase grew, and so did the software made in PHP.

So now when I’ve found Rails, I am in a position where I simply cannot just switch away from PHP, although I sure would like to. Rails is simply geniously made.

But I have just started a “spare-time” project, which I am making in Rails. And I will try to use this blog as a diary for this project. In the hope, that others (if any are listening) can learn from my mistakes.