Ruby block and Regex

Open source is fantastic! I learnt something new from reading Rails source code today again.

Today’s surprise was brough to me by the power of regular expression and Ruby blocks.

Rails 3 Net::HTTPHeader module has the following method

def urlencode(str)
  str.dup.force_encoding('ASCII-8BIT').gsub(/[^a-zA-Z0-9_\.\-]/){'%%%02x' % $&.ord}
end

1st, I learnt the gsub method could take a block as its second argument.

2nd, inside the block, all regular expression back references could be used, e.g. $& for referencing all matching elements.

Published: 2011-12-06

Android screen grab

I needed to publish an Android app for one of our clients today. Screenshots of the app is required for publishing.

To do it, extremely simple.

  • Get application onto testing Android device
  • Connect Android device to computer, make sure device’s debug mode is enabled.
  • From the computer, start the DDMS program that’s shipped with the Android SDK
  • From DDMS, highlight the connected Android device, and go to menu “Device” - “Screen capture”

Not too hard. Have to say Apple’s native screen capture feature on iOS devices is a lot simpler.

Published: 2011-11-29

RVM update

RVM is super easy to use, and it manages Ruby Gems well. Most of the Ruby/Rails projects I work on are using its Gemset feature.

I was running RVM version 1.6 and decided to perfom and upgrade.

rvm get latest
rvm upgrade 1.9.2-p180 1.9.2-p290

Second command from above is extremely useful. As per example above, it updates my local Ruby 1.9.2-p180 to 1.9.2-p290. All gemsets associated with the 1.9.2-p180 are also relinked to the new 1.9.2-p290. Handy!

Published: 2011-11-05

Create SQLite DB from CSV

Recently a project I’m working on requires me to populate an SQLite database with CSV contents. A very straight forward task that I have done so many times before in PHP.

This round Ruby is the pick. Everything just feels a bit nicer.

Published: 2011-10-22

Ruby string interpolation

We store configuration strings in YAML files. It’s common that we need to substitute part of the string with runtime values. Hence we introduce some kind of placeholders in the config strings. Given an example that our application may need to hit an external URL to retrieve some remote resources at runtime. To do this, we could make a config string in our application’s config YAML file like this

remote_url_template: "http://remote.domain.com/give/me/RESOURCE1/and/RESOURCE2"

Then in our code, we substitute “RESOURCE1” and “RESOURCE2” with the real resource names we require. In my early days of programming in Ruby, I used some very ordinary string substitution method like this

# remote_url_template is read from the config yaml
resource_one = "coke"
resource_two = "chips"
remote_url = remote_url_template.gsub("RESOURCE1", resource_one).gsub("RESOURCE2", resource_two)

It works, but I don’t really like it… So I found something better, that is to use Ruby’s string interpolation methods

# in yaml file
remote_url_template: "http://remote.domain.com/give/me/%s/and/%s"

# in ruby code
# remote_url_template is read from the config yaml
resource_one = "coke"
resource_two = "chips"
remote_url = sprintf(remote_url_template, resource_one, resource_two)

A lot nicer, but still can be better. Thanks to a colleague of mine, Aleksey Gureiev, another more elegant way can be used

# in yaml file
remote_url_template: "http://remote.domain.com/give/me/%s/and/%s"

# in ruby code
# remote_url_template is read from the config yaml
resource_one = "coke"
resource_two = "chips"
remote_url = remote_url_template % [resource_one, resource_two]

That is satisfactory!

Published: 2011-08-20

Manage Rails app configurations

Most of applications require application configuration files. The Rails applications I worked on pretty much all require an initialiser (or should it be the Rails convention initialiZer) to load a configuration YAML file. I’m aware that there are gems out there providing this kind of configuration solutions, such as SettingsLogic. For a simple problem like this one, I think we could roll our own too (oh, the not invented here syndrome).

The codebase I worked on typically has an initialiser called load_config.rb. It’s got something like this

# config/initializers/load_config.rb

AppConfig = YAML.load_file(File.join(RAILS_ROOT, 'config', 'config.yml'))

# Override config options by correct environment
env_options = AppConfig.delete(RAILS_ENV)
AppConfig.merge!(env_options) unless env_options.nil?

Easy and good. There’s one more thing I’d love to add to this. I’d love to have a local config file, config.local.yml, that can overwrite config values defined in the config.yml. And this config.local.yml is not source controlled, means every developer can make tweak the configs to tailor to their own development environment without needing to touch the source controlled configuration file, config.yml.

To do this, just add the following block of code to the load_config.rb

local_config_filepath = File.join(RAILS_ROOT, 'config', 'config.local.yml')
if File.exist?(local_config_filepath)
  AppConfig.merge!(YAML.load_file(local_config_filepath))
end
Published: 2011-07-23

CoffeeScript Basics

Rails 3.1 will support CoffeeScript out of the box. I gave the new boy in town a little test. I think it’ll be a great tool for web developers writing javascript code. CoffeeScript is short and precise, generated JavaScript variables are properly scoped, class inheritance is made super easy, love the syntax sugar. Oh and yes, mixing in jQuery is easy too.

Here’s a little play script I wrote to illustrate how a JavaScript class can be written in CoffeeScript

class Person
  ### private var ###
  my_name = null

  ### constructor ###
  constructor: (name) ->;
    my_name = name

  ### public instance method ###
  speak: (saying)->;
    alert "#{saying} #{fancy_name()}!"
 
  ### private instance method ###
  fancy_name = ->;
    if my_name == "Jim"
      "Jimbob"
    else if my_name == "Dan"
      "DanDaMan"
    else
      my_name

  ### class method ###
  @feature: ->;
    alert "1 Person has only 1 head"

Person.feature()
jim = new Person("Jim")
jim.speak("Hi")

The above CoffeeScript will be translated into the following JavaScript

var Person, jim;
Person = (function() {
  /* private var */  var fancy_name, my_name;
  my_name = null;
  /* constructor */
  function Person(name) {
    my_name = name;
  }
  /* public instance method */
  Person.prototype.speak = function(saying) {
    return alert("" + saying + " " + (fancy_name()) + "!");
  };
  /* private instance method */
  fancy_name = function() {
    if (my_name === "Jim") {
      return "Jimbob";
    } else if (my_name === "Dan") {
      return "DanDaMan";
    } else {
      return my_name;
    }
  };
  /* class method */
  Person.feature = function() {
    return alert("1 Person has only 1 head");
  };
  return Person;
})();
Person.feature();
jim = new Person("Jim");
jim.speak("Hi");
Published: 2011-07-20

Background jobs on Heroku

I’m recently asked what my recent technology excitement is, of course in the context of programming. The name of Heroku came into my mind instantly. I’ve been using Heroku for all my Rails based applications recently. I’m amazed how easy it is to deploy and scale (to a degree) applications on to Heroku.

Heroku’s default way of running background jobs is using the delayed_job Ruby gem, which essentially records background jobs into a database table. And on Heroku, you can start a worker to monitor and process the job queue. This involves a lot of database polling as you could imagine.

So, in one of the applications I’m working on, I tried something more elegant (in my opinion). Github uses a Ruby gem called Resque for their background jobs.

  • Resque uses a Redis based backend to store queued jobs, so it's fast in job logging and job retrieving.
  • Resque allows you to separate jobs into different queues, so everything becomes better categorised.
  • Resque also provides a Sinatra based web interface to monitor all jobs  including failures, so better monitoring and better debugging.

All good. I also opted to use a Resque plugin called resque-heroku-autoscaler. This allows me to dynamically scale the number of Heroku workers up and down base on the jobs recorded in the queue. Since Heroku charges on worker usage, so with this auto scaler, I only pay for what’s used.

As for the implementation, it’s scattered across many files. Here’s my implementation, which handles the delivery of welcome emails upon membership creation.

Gemfile

gem 'resque', :require => "resque/server"
gem 'resque-heroku-autoscaler'

config/resque.yml

development: 127.0.0.1:6379
test: 127.0.0.1:6379

config/initializers/resque.rb

require 'resque'

heroku_environments = ["staging", "production"]
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'development'

unless heroku_environments.include?(rails_env)
  resque_config = YAML.load_file(rails_root + '/config/resque.yml')
  Resque.redis = resque_config[rails_env]
else
  uri = URI.parse(ENV["REDISTOGO_URL"])
  Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
end

config/initializers/resque_heroku_autoscaler_setup.rb

require 'resque/plugins/resque_heroku_autoscaler'

Resque::Plugins::HerokuAutoscaler.config do |c|
  c.heroku_user = 'heroku_user_name'
  c.heroku_pass = ENV['HEROKU_PASSWORD']
  c.heroku_app  = ENV['HEROKU_APP']

  if Rails.env.production?
    c.new_worker_count do |pending|
      (pending/5).ceil.to_i
    end
  end

  if Rails.env.development?
    c.scaling_disabled = true
  end
end

app/models/membership_observer.rb

class MembershipObserver < ActiveRecord::Observer
  def after_create(membership)
    Resque.enqueue(MailerCallback, "MyMailer", :welcome_email, membership.id)
  end
end

app/models/mailer_callback.rb

require 'resque/plugins/resque_heroku_autoscaler'

class MailerCallback
  extend Resque::Plugins::HerokuAutoscaler

  # resque queue name
  def self.queue
    :email_queue
  end

  # resque callback method
  def self.perform(mailer, email_type, *args)
    mailer.constantize.send(email_type, args).deliver
  end
end

Reference

Published: 2011-06-30

Ruby on Rails, 2 months in

I’ve started my “senior” developer role in doing Ruby on Rails projects for a bit over 2 months. Reason I quoted the “senior” bit is that I’m really a newbie to it. I guess the seniority is more about general web based application development rather than RoR. Having said that, RoR isn’t that foreign to me at all. I started studying it a year ago. However my self-study doesn’t compensate my lack of real world working experience. In the past 2 months, lots of ideas and knowledges are exchanged between me and my peers. I learnt a lot of Rails of course. And I believe I have also contributed some non-rails thinking into the Rails project.

2 months in, I learnt that Rails is moden and rapid in doing web development (I absolutely love it). However it’s convention over configuration nature sometimes dictates the way developers think too much. Somehow, developers’ creativities become canvased by Rails. You find yourself battling with routes, ActiveRecord models, DRY controllers a lot. The application architectural deliberating stage is somewhat overlooked, if not bypassed. Most of the time, this will lead to future potential problems, which could be security issues, scaling issues, performance issues or just tedious labour work in doing repetitive tasks.

As I’m still learning, what I believe now is that don’t let Rails control how you author software,

  • write non AR models to describe your domain if required
  • use service layer classes to encapsulate business logic if it makes sense
  • off load long running, non critical tasks to a job queue (loving the resque gem now) when it's appropriate
  • the list goes on

Let Rails help you instead,

  • use its powerful command line tool
  • utilise its ecosystem, all those wonderful gems
  • make a use of its RESTful controllers, exposing API is never as easy as before
  • look into it's respond_to feature, making your RESTful API respond to XML, json, or whatever is made super easy
  • the list goes on

To summarise, the basic software engineering practices apply to Rails or non Rails applications the same way.

Bend the rails to let your train reach it’s destination. Expect more Rails related posts from now on.

Published: 2011-04-24