Monday, February 9, 2009

It's a trap!

Using POSIX signals in Ruby is pretty easy. Use the trap method and pass it a block that will execute when the trap is received.

trap("TERM") { puts "TERM signal received."}
sleep(50)


If you run this and send it a TERM signal (using kill <pid> for example), it will print "TERM signal received." One thing to note, this will override Ruby's built in handlers. So, kill will no longer cause the program to exit.

trap will return a reference to the current trap handler. You can use this to replace the previous handler when you are finished.

previous_handler = trap("TERM") { puts "TERM signal received."}


and to restore the previous handler:

trap("TERM", previous_handler)


One poorly documented gotcha is that trap handlers always run in the main thread and not in the thread that the trap was defined. This can cause hard to track down bugs.

On a previous project we were running into weird issues where open database transactions were being committed when the process received a TERM signal. We were using an old version of Rails 1.2 and tracked it down to the transaction method in ActiveRecord::Transactions (condensed to show relevant code):

previous_handler = trap('TERM') { raise TransactionError, "Transaction aborted" }

begin
...

result = connection.transaction(Thread.current['start_db_transaction'], &block)

objects.each { |o| o.commit_transaction }
return result
rescue Exception => object_transaction_rollback
objects.each { |o| o.abort_transaction }
raise
ensure
...
end


This is supposed to raise a TransactionError if a TERM is received, but if Rails is not running on the main thread (which it isn't if you are using Mongrel), the raise will not occur in the transaction thread. So, the rescue will not be executed.

We found a workaround by changing the trap line to this:

transaction_thread = Thread.current
previous_handler = trap('TERM') { transaction_thread.raise TransactionError, "Transaction aborted" }


This will cause the exception to always be raised in transaction_thread.

This doesn't seem to be a problem on the newer version of Rails, but its something to watch out for when using trap.

No comments: