Tuesday, February 26, 2008

Testing MIME response types

I feel like I might have covered this before, but I was looking for a way to test respond_to. I had found this post on how to test it, but after looking at it for a while, I found myself rewriting it. Mainly, I took out parts that convert the Mime types, and inserted Rail's own Mime type objects. You can use it like this:


request_mime(:fbml) do
get :list
assert_response_mime(:fbml)
end

request_mime("text/xml") do
get :list
assert_response_mime("text/xml")
end


Just include it in your test_helper.rb file in test/

class Test::Unit::TestCase
include Threecglabs::MimeTestHelpers
end


Here's "mime_test_helpers.rb". Just throw it in lib/

module Threecglabs
module MimeTestHelpers

def self.included(mod)
mod.class_eval do
include MimeRequest
include MimeAssertions
end
end

module MimeRequest
# changes the mime type of the request within the block
#
# request_mime(:fbml) do
# get :list
# assert_response_mime(:fbml)
# end
def request_mime(mime_type_name_or_extension)
if mime_type_name_or_extension.kind_of?(String)
mime_type = Mime::Type.lookup(mime_type_name_or_extension)
elsif mime_type_name_or_extension.kind_of?(Symbol)
mime_type = Mime::Type.lookup_by_extension(mime_type_name_or_extension.to_s)
else
raise ArgumentError.new("mime type must be string or symbol")
end
old_mime_type = @request.accepts
@request.accept = mime_type.to_s
yield
@request.accept = old_mime_type
end
end

# These are assertions to test respond_to, whether they return a specific mime type
# as a response to a request
module MimeAssertions

# Helps out with response testing, by letting to assert that the most recently-made
# request responded with a particular MIME extension, like :html, :fbml, :xml, etc.
def assert_response_mime(expected_mime_type_ext)
expected_mime_type = Mime::Type.lookup_by_extension(expected_mime_type_ext.to_s)

# Mime::Type.parse doesn't parse accept parameters correctly, therefore
# we account for having multiple types in the accept
response_mime_types = @response.headers['type'].split(/,\s*/).map do |accept_type|
mime_type_name = accept_type.split(/;\s*/).first
Mime::Type.parse(mime_type_name)
end
assert_block("Responded with #{response_mime_types.map(&:to_s).inspect} when expecting #{expected_mime_type}") {
response_mime_types.any? { |response_mime_type| expected_mime_type == response_mime_type }
}
end

end
end
end

Snippet!

irb console tricks

irb Mix Tape — err.the_blog

Not a big entry, but I think worth posting because it pays to learn your tools. I've got no insights of my own to add. Someone else's tip!

Wednesday, February 20, 2008

Render_to_string only counts if failed

render_to_string followed by render issues - Ruby Forum

I was puzzled by this yesterday, and good thing I went to do something else instead of tearing my hair out. Came back fresh today and found this. Apparently, render_to_string normally doesn't count as a render--given that it succeeds. If it throws an exception for some reason, then it'll get count as a double render if you rendered elsewhere. Small tip!

Friday, February 15, 2008

Energy over Space

Annotation (Harper's Magazine): Keyword: Evil

It's an annotated blueprint of one of Google's datacenters. While it's all good and well to point these criticisms out, that doesn't mean nothing's being done about it. Google itself has announced that it was creating a renewable energy R&D group. In addition, it's also in the semiconductor's interest to build cooler chips. Hot chips won't sell as well now that datacenters are concerned with gigaflop per watt.

I have a gut feeling that if the next president is in tune with what's going on in tech, he/she'd challenge Americans to rise to the occasion to innovate through the energy problems, just as JFK challenged Americans to fly to the moon half a century ago. I remember in the 90's as a kid, there was a stint of environmentalism. Captain Planet. Save the Walrus tshirts. And then it kinda faded away until now, a whole decade later. I'm hoping that it's not a fad, like it was in the 90's. There are plenty of interesting problems in this space. And I hope we figure them out before we're able to easily colonize other planets. If space tech supercedes energy tech, then it would end up to be more economical to just ditch a planet once we dirtied it up for another one. And pity the natives that happened to be living there. Humans would be the tyrants looking to subjugate other hospitable planets, and not the victims, like so many science fiction stories would have us believe. It would look like empirialism all over again. So I hope we figure out this renewable energy stuff before we figure out how to easily colonize outer space.

Wednesday, February 13, 2008

MIME responder filter for Rails

I didn't think I had to do this, but I ended up writing a filter that acts like a switch statement for different MIME types. Let me explain. Normally, in Rails, you can respond to different requests for different content with something like this:

class PostController < ActionController::Base
def list
@posts = Post.find(:all)
respond_to do |format|
format.html
format.fbml
format.xml { render :xml => @people.to_xml }
end
end
end

When you have something like this, the browser (or whatever client) can ask for different MIME types. Here, we can return html to a browser, xml to maybe a data importer, and fbml to facebook.

I spent last week integrating Mobtropolis with facebook. Mobtropolis doesn't require a facebook account to use it, so like other websites, it has its own authentication mechanism, something like:

class PostController < ActionController::Base
before_filter :website_authenticate_filter, :except => [:index, :list]
end

When I started using facebooker library, it already came with an authentication before_filter. That means we have two authentication filters, one native, and one for facebook. Mobtropolis users don't have to be in facebook to use it, and facebookers don't have to sign up again in mobtropolis to use it.

However, since before_filters are executed in succession, it leads to a case where the facebook authentication would be called if html was requested, and vice versa. The alternative was to take apart both authentication filters, and create a monolithic filter to handle the two different cases. Instead, I did this:

class PostController < ActionController::Base
before_respond_to_filter :except => [ :index, :list ] do |format|
format.html :website_authentication_filter
format.fbml :facebook_authentication_filter
end
end

That way, I didn't have to mix together the guts of each authentication filter, and it solved the problem of the wrong authentication filter being run. You can also use it like:

class PostController < ActionController::Base
before_responds_to_filter :only => :home do |format|
format.html do |controller|
return if controller.logged_in?
controller.send(:redirect_to, :controller => :home)
end
format.fbml :ensure_application_is_installed_by_facebook_user
end
end

By the way, I tried to alter the filter_chain as a request came in. Filter chains are copied and passed around the filters, so you can't write a filter that alters the filter chain. So don't waste your time crawling around in the guts of Rails to do this like I did. It's just as well, as that'd be a nightmare to maintain.

It does have some weaknesses though. You can only assign the filters to the same set of :except and :only options in the filters.

It ended up the code for this sort of magic was fairly easy. I'm not sure if there's an easier way to do what I wanted, but I'll see if Rails core people would find it useful (or not). In the meanwhile, for those of you Rubyists that have written plugins before that want to play with it. As with the usual mumbo jumbo, it's provided as is, I'm not maintaining it, and do whatever you want with it:


module Threecglabs
module Filters

# MimeResponderFilter
module MimeResponderFilter

def self.included(mod)
mod.extend(ClassMethods)
end

# Filters can respond to different mime types, so that you can use
# different filters depending on which mime type is being requested
#
# before_responds_to_filter :except => [:login, :signup, :forgot, :invite_request, :profile] do |format|
# format.html :authentication_filter
# format.fbml :ensure_application_is_installed_by_facebook_user
# end
#
# This way, one can take the appropriate actions in setting up authentication
# from different mime types, and still separate the implemenation of the different
# kinds of implementations
#
# The formats also take blocks, like regular filters
#
# before_responds_to_filter :only => :home do |format|
# format.html do |controller|
# return if controller.logged_in?
# controller.send(:redirect_to, :controller => :home)
# end
# format.fbml :ensure_application_is_installed_by_facebook_user
# end
#
# NOTE: an :all format defaults to :html, therefore, a format.html is required
module ClassMethods
def before_respond_to_filter(options = {}, &block)
before_filter MimeResponderFilter.new(&block), options
end

private
# This is a call that implements a MIME responder filter
class MimeResponderFilter#:nodoc:
attr_reader :filters

def initialize(&block)
@filters = {}
block.call(self)
end

def filter(controller)
filter = @filters[controller.request.format.to_sym] || @filters[:html]
if filter.kind_of?(Proc)
filter.call(controller)
else
controller.send!(filter)
end
end

# implements the "format.#{mime_type}" part of the filter
def method_missing(mime_type, method_name = nil, &block)
if block_given?
@filters[mime_type.to_sym] = block
else
@filters[mime_type.to_sym] = method_name.to_sym
end
end
end
end

end
end
end


Snippet!

Friday, February 08, 2008

Erlang Advocacy and the class of problems it solves

I spend more time than I should reading hacker.news. Granted, I sometimes feel like it's the US Weekly or Maxim of the tech world, when stories like "5 ways you know you're failing at your start up" and the general hubbub over Facebook back in June or the recent Microsoft and Yahoo buyout. However, the quality of the comments there is generally high, and on occasion, I'll start replying to a comment, and before I know it, two hours have passed, and it's better off as a blog post.

The challenge of the Erlang advocate is not to convince me, over and over, that Erlang wins its class; the challenge is to convince me that Erlang's class of problem is so important to my life that I should study Erlang rather than vascular surgery, or television repair, or other obscure technical skills that I don't know that much about.


The follow-up question then would be how many existing problems can be converted to the type that Erlang can solve well? And how many problems previously impractical are now practical to solve? So if it ends up to be a bigger class than you thought, you may well be limiting the number problems that you can solve easily and practically out of the ones that will be important in the future.

I like Erlang (aside from some syntax ugliness), so I'll give it a shot. I think it's important because it allows a program to easier to scale out rather than scale up. If it was running an algorithm that are parallelize-able, then you can just technically add more cheap processors to it for a speed up, rather than designing a faster processor. We'd want speedups in this way because CPUs are becoming multi-cores, and to take advantage of them one will have to write some type of parallel program, since it's proven difficult to fully automate parallelizing serial programs. In addition, with bandwidth pipes getting bigger and the internet more and more pervasive, it is possible that you can leverage other computers you don't own (but given permission) for computational or storage resources in the future most of whom don't belong to a single entity. (think more SETI@home than Amazon)

In addition, parallelized systems (not just erlang) can be more fault tolerant and can fail more gracefully (or hobble along, if you'd like to call it that). Sensor networks are one example. Instead of a single radar to detect the environment, you throw a bunch of sensors out there, and they network themselves and report what they see/hear back to you. If some of them gets destroyed that's ok, because the system's still functioning with less sensors.

A swarm of UVAs doing surveillance in an area is another example. If you have a single computer commanding all the UVAs, it's actually quite hard, because you don't want them to crash into each other, so that's N^2 comparisons (less if you do oct-trees, probably). And if a target comes into the area, it becomes a non-trivial allocation problem: how do you decide which UVA to assign to monitoring it, and when do you switch them out when their fuel runs low? It ends up that doing it in with an actor model, where each UVA decides what to do at any given moment (local interactions) might not be optimal, but it's redundant and fault-tolerant.

Biological systems work this way as well. Gene expression is actually a network of genes being turned on and off by proteins expressed by other genes being turned on and off recursively. If one gene can't activate another, it might not always be detrimental, because there might be other pathways to express that gene. Since I'm not as familiar with the intricacies of biological systems, I'll refrain from saying anymore.

I was surprised when I was watching a video of Alan Kay talking about OOP that the OOP C++/Java I learned in college wasn't what he had in mind. Rather, he meant OOP to be more like biological systems and more process orientated. Encapsulation was only meant so that objects (analogous to actors in erlang) would have to pass messages to each other (method calls).

So what concrete examples of problems fall into this class Erlang is good at? The obvious ones are the embarrassingly parallelized algorithms, like genetic algorithms, neural networks, 3d rendering, and if I'm not mistaken, FFTs. Indexing web pages is another. But then again there are other algorithms that are inherently serial like protocol handshaking or newton's method.

I don't know what the ratio is between embarrassingly parallel and Serial problems are, but my gut is that with the advent of multi-cores and availability of the internet, I think there will be plenty of parallel problems to go around.

Ruby and Haxe language writers are both implementing the actor model like Erlang, if that's any indication of how important they think it is. While I don't think Erlang will be the 100 year language, the ideas by which it's a poster child will reverberate in the descendant languages for a long time to come.