It's a trap!

Using POSIX signals in Ruby is pretty easy. Use the trap method and pass it a block that will execute when the trap is received.

trap("TERM") { puts "TERM signal received."}
sleep(50)

If you run this and send it a TERM signal (using kill <pid> for example), it will print "TERM signal received." One thing to note, this will override Ruby's built in handlers. So, kill will no longer cause the program to exit.

trap will return a reference to the current trap handler. You can use this to replace the previous handler when you are finished.

previous_handler = trap("TERM") { puts "TERM signal received."}

and to restore the previous handler:

trap("TERM", previous_handler)

One poorly documented gotcha is that trap handlers always run in the main thread and not in the thread that the trap was defined. This can cause hard to track down bugs.

On a previous project we were running into weird issues where open database transactions were being committed when the process received a TERM signal. We were using an old version of Rails 1.2 and tracked it down to the transaction method in ActiveRecord::Transactions (condensed to show relevant code):

previous_handler = trap('TERM') { raise TransactionError, "Transaction aborted" }

begin
  ...
  
  result = connection.transaction(Thread.current['start_db_transaction'], &block)
  
    objects.each { |o| o.commit_transaction }
    return result
  rescue Exception => object_transaction_rollback
    objects.each { |o| o.abort_transaction }
    raise
  ensure
    ...
end

This is supposed to raise a TransactionError if a TERM is received, but if Rails is not running on the main thread (which it isn't if you are using Mongrel), the raise will not occur in the transaction thread. So, the rescue will not be executed.

We found a workaround by changing the trap line to this:

transaction_thread = Thread.current
previous_handler = trap('TERM') { transaction_thread.raise TransactionError, "Transaction aborted" }

This will cause the exception to always be raised in transaction_thread.

This doesn't seem to be a problem on the newer version of Rails, but its something to watch out for when using trap.

Railsconf 2008

I had a great time at Railsconf. I got to meet up with a lot of coworkers I haven't seen in a while. The buzz around the conference was energizing. It definitely feels like the community is maturing. There were a lot of people there using rails for their day to day business. I also was happy to see how a lot of the sessions were very ruby focused. The Rails and Ruby communities felt like they were coming together.

On Thursday I went to see Neal Ford and Pat Farley's metaprogramming tutorial. Now I may be a bit biased but I thought their talk was great. I also heard good things from random people I talked to. After that I went to the Refactotum talk put on by Relevance. Thanks to them, I submitted my first patch to CruiseControl.rb!

I enjoyed Nathaniel Talbott's 23 Hacks talk. Its easy to forget to keep programming fun! Justin Gehtland's Small Things, Loosely Written and Written fast was also a great talk. Great ideas and an entertaining presentation. Philippe Hanrigou's talk on troubleshooting Mongrel processing was great, too. It was pretty hardcore and narrowly focused, but the people that were there seemed to get a lot out of it.

All the upcoming ruby implementations are pretty exciting as well. They all look to be progressing well and each has their own niche. The Rubinius demo of browseable backtraces was awesome. After hearing Kent Beck's comment on using Ruby tools to develop Ruby, I think Rubinius in particular has great promise as a Ruby development platform. I could see some powerful IDE's taking advantage of all the internals exposed by Rubinius. A lot of people have commented on it already, but Maglev looks interesting. As a consultant, a platform like that could allow us to use Ruby in places beyond traditional web apps.

Its great to be around a large group in the Rails community. It looks like the excitement isn't dying down anytime soon!

iPhone SDK first impressions

I've been working with the iPhone SDK lately. I'm working on an application to input receipt information for expense reports. It's a fairly simple app, but its useful to me and a good starting point.

Here's the things I like so far:

• Apps are written in Objective-C and use the same Foundation framework as OS X apps. Things like NSArray, NSDictionary, NSDate are all the same. If you have any experience writing Cocoa apps, you can get started on iPhone apps fairly easily.


• iPhone development is well integrated into XCode. You just pick an iPhone app template and go. The "Build and Go" button will build and deploy your app to the built in simulator.


• The Cocoa Touch UI widgets are nice. Apple did the smart thing and built a new UI framework from the ground up. The iPhone interface is quite a bit different, so trying to shoehorn desktop UI widgets would have been a mistake. UITableView is easy to use and extend. I'm using it on just about every screen.


• There is a lot of good documentation so far. I know its still in beta, but there is a lot of sample code, videos and API docs.

The SDK is still in beta and there are a few rough edges. Some problems I've run into:

• Interface Builder is not usable. The second release of the SDK included Interface Builder for the iPhone. There are no project templates available to set up an iPhone project with Interface Builder. I searched around and found out how to add the IB xib file to the build and show it on the screen. It is not as seamless as it is in the desktop version. I'm sure this will get fixed by the time the SDK leaves beta. Fortunately, the SDK is still quite usable without Interface Builder.


• OpenGL, CoreLocation and other frameworks are not useable in the simulator. I can understand the technical problems in getting these to work, but Apple is not accepting developers into the iPhone Dev Program very quickly. So, its impossible to work on any apps using the frameworks that won't run in the simulator.


• I can't figure out how to get OCUnit to work in the SDK. After being immersed in TDD while working at ThoughtWorks, it would be nice to be able to unit test my iPhone code. I'm still not familiar with how OCUnit works, so I haven't figured out if there is a way to have tests run in the simulator.

It's been fun getting away for web apps for a bit and working on mobile devices again. I think the iPhone has great potential, and the SDK is quite powerful. It will be interesting to see all the applications that will come out for it. I think the iPhone also has potential in the enterprise space. It looks like Apple has thought about that, too. I'm eager to see how their enterprise developer program turns out.

Fun with Enumerable::Enumerator

Ruby has a nice class in the core library for turning objects with iterator methods into Enumerable objects: Enumerable::Enumerator. I discovered this class a while ago, but hadn't thought of a need for it at the time.

Recently, I was integrating with some web services and needed to parse some XML. REXML's XPath is a great tool for this. XPath has a method #each that will allow you to iterate over all the elements that match the XPath. So, to collect all the text nodes for an XPath, you could do something like this:

items = []
REXML::XPath.each(@document.root, "//list/items").each do |element|
  items << element.text
end

XPath implements the each method, but isn't an Enumerable so it doesn't support methods like collect. Using Enumerable::Enumerator and Rails' to_proc makes this code simpler:

Enumerable::Enumerator.new(REXML::XPath, :each, @document.root,  "//list/items").collect(&:text)

Spiking with reckless abandon

A couple months ago, we were trying to improve our build process to better reflect our production environment. Our application runs on Oracle and the database schema is owned by one Oracle user, and our application logs in as another. The application user is granted rights to the tables it needs to use. There are also synonyms defined for all the tables our app uses.

Our builds only used a single database user for all our tests. If we forgot to update our database scripts to add new grants or synonyms, we wouldn't know it until we deployed to our QA environment. So, our goal was to have our build run with the same user split that the production environment did.

One problem with that scenario is our functional tests inserted data into tables that the app user didn't have insert rights to. We decided to spike switching to the schema owner for insertion of test data and go back to the regular app user for the rest of the test. We came up with this crazy bit of code to hijack our ActiveRecord models and reuse them to insert test data:

module TestData
  def self.const_missing(name)
    klass = Kernel.const_get(name)
    new_klass = Class.new(klass)
    const_set(name, new_klass)

    new_klass.class_eval do
      establish_connection "test_db_owner"

      def ==(comparison_object)
        comparison_object.equal?(self) ||
          (comparison_object.instance_of?(self.class.superclass) &&
            comparison_object.id == id &&
            !comparison_object.new_record?)
      end

      def instance_of?(klass)
        self.class == klass || self.class.superclass == klass
      end
    end

    new_klass
  end
end

This bit of code defines a new module called TestData. It overrides const_missing? and looks for an existing constant in the root namespace. It creates a new class in the TestData module that is a subclass of the original class. It then tells ActiveRecord to connect as the schema owner. There is some stuff in there to patch == and instance_of? to allow the new object to be compared to the original one.

Now, code like this is quite dangerous and is nothing I'd ever use in real production code. The real point of the spike however, was not figuring out how to connect as a different user, but to see how feasible it would be to modify our tests to use two database users. This little hack only took about 20 minutes to write and allowed us to attack our main problem. Since this was a spike, we knew we were going to throw it all away after we learned what we needed to know.

It turned out that we ran we couldn't use two seperate users easily, because all our test methods ran in transactions. We relied on the transaction rollback to clean up all our inserted test data. We could work around this, but it would take a lot more rework than we had originally planned. Our little TestData module was promptly deleted from our code base.

I am a big fan of any language that gives power to the developer. I don't like to be shackled because some language designer thinks that developers should be protected from themselves. Features like Ruby's const_missing? can be horribly misused and can cause code that is extremely difficult to debug, but it can also be really useful. Our little experiment only took a couple of hours, because we were able to use a powerful, but possibly dangerous feature of Ruby. That module was the quickest way for us to get to the meat of the problem without wasting a lot of time on technical details that we could figure out once we committed to doing the work.

I believe in smart tools for smart developers. Making a language overly safe doesn't really enforce code quality. Bad programmers will write bad code no matter the language. All it does is keep the good ones from accomplishing what they need to do.

Synchronizing using Active Record

Recently, I was working on implementing a simple job queue. I needed multiple processes to pull jobs from the queue, and I needed to synchronize access to the job queue. ActiveRecord allows you to pass the :lock option to #find.

def find_new_job
  Job.transaction do
    job = Job.find(:first, :conditions => "status = 'new'", :lock => true)
    if job
      job.status = "processing"
      job.save!
    end
    job
  end
end

ActiveRecord will use a SELECT ... FOR UPDATE, which locks the rows in the result set. When another query selects any of those rows, it will block until the first transaction is complete. This works well on Postgres and MySQL 5, but Oracle gave me an error:

ActiveRecord::StatementInvalid: OCIError: ORA-02014: cannot select FOR UPDATE from view with DISTINCT, GROUP BY, etc.: select * from (select raw_sql_.*, rownum raw_rnum_ from (SELECT * FROM jobs WHERE (status = 'new') ) raw_sql_ where rownum <= 1) where raw_rnum_ > 0 FOR UPDATE
 from ./script/../config/../config/../vendor/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:128:in `log'
 from ./script/../config/../config/../vendor/rails/activerecord/lib/active_record/connection_adapters/oracle_adapter.rb:222:in
`execute'
etc...

The Oracle adapter generates the above SQL for a find(:first), and SELECT .. FOR UPDATE won't work on a subselect. So, changing the method to something like the following will allow it to work in Oracle:

def find_new_job
  Job.transaction do
    jobs = Job.find(:all, :conditions => "status = 'new'", :lock => true)
    job = nil
    if jobs.size > 0
      job = jobs.first
      job.status = "processing"
      job.save!
    end
    job
  end
end

I've used this sort of job queue in Rails apps to allow for asynchronous calls to external services. Instead of calling the services synchronously and making the user wait (and use up a mongrel instance), create an entry in the job queue. External ruby processes can poll the job queue looking for work to do. If the user needs to see the results of the job (such as placing an order or authorizing a credit card), the request that created the job redirects to a controller action that polls the status of the job. When the job is completed, the action will redirect to a new action that will display the results of the job.

The same database locking mechanism can also be used as a simple way to synchronize access to resources that are shared across multiple processes on multiple servers. Create a simple ActiveRecord model for a Lock object:

class Lock < ActiveRecord::Base
  def self.synchronize(name)
    self.transaction do
      Lock.find(:first, :conditions => ["name = ?", name], :lock => true)
      yield
    end
  end
end

Create a lock:

Lock.create!(:name => "TestLock")

Then use it like this:

Lock.synchronize("TestLock") do
  # access resource here
end

The downside to this is that all locks you want to use need to be created beforehand. If there are a few predetermined locks needed for the application, this approach works fine.

Photo: Gatun locks at the Panama Canal

Image borrowed from Flickr user: Donna62