Wednesday, January 30, 2008

How to find multiple file types using linux's "find"

I've always found *nix's "find" and "grep" rather hard to use. Not only are there different flavors of regular expressions to use, but mainly different syntax. For find, the directory you're searching for comes first. For grep, it comes last. To find the negation of something, you'd use "-not" and for grep it's "-v". pain.

Anyway, been trying to learn my tools better, and I found out how to grep and replace expressions across multiple files through emacs. Since rails uses all sorts of file extensions, naturally, I wanted to grep for find different files types. I had thought the -name options took regexs (it doesn't), so I had tried it in regex (no go)...only to find that it's something like this:

find . -name "*.rb" -o -name "*.rhtml"

the -o is the equivalent of a boolean "or". small tip...

Tuesday, January 29, 2008

2D barcodes rebirth

Google just announced 2Dbarcodes for print ads, which prompted Joel to talk about the :CueCat, which was a large failure in the late 1990's. Wired Magazine sent you these barcode readers shaped like cats with this one issue. The idea was that you could scan barcodes in print ads, and given the URL encoded in the barcode, it'll take you there in your browser. I can hear wails of people going, "now why would I want to do that?" Of course, that's from a lack of imagination.

Now, ideas are brilliant or stupid only in the right context. Joel is correct about his assessment of the :CueCat. However, there are instances where it works and is in widespread use.

Japanese print ads have 2D barcodes on them. Even blogs and webpages have 2D barcodes, so you can access information (URL or otherwise) from your phone.

There are a couple things different there. Most Japanese peoples' access to the internet has been through their cell phones, rather than through their computers. Landlines are much more expensive than having a cell phone. In addition, the majority of Japanese are in urban areas where they use public transportation. That gives them a lot of down time to play with their phones. The print ads in the train have the 2D bar codes on them, so people can check out the ad while they're riding the train. Given that unless you have a full keypad (real or virtual), it's still harder typing in a phone, than using a 2D barcode.

That said, I don't think Americans (Don't know about europe) will find as much use for 2D barcodes for print ads, as we drive everywhere. The tech world has changed a lot since the :Cuecat was around, so I think most of the right players are in place for 2D barcodes. I don't think having 2Dbarcodes in newspapers makes sense. It makes much more sense for print ads, you can see on the street in strictly urban areas, like New York City.

If I'm walking around town, I'd find it very handy to be able to check out how many tickets are left for a show and being able to buy tickets for the show from the URL in a 2D barcode on the print ad for a show across town. It would also be useful if sewn onto tags of pillows, clothing, other products, etc, so that would provide product information, or at least a URL that has the manual, or specs. That way, if some piece were broken, it'd be an easy way to order replacement parts.

I think it's a tiny, tiny step into tying the real world with the virtual, and part of the move to form a Clickable Earth.

Sunday, January 27, 2008

Making sure extra tasks get run when installing plugins with Piston

When I graduated college, I was a EE major that could code. However, in hindsight, I don't think I was as good because I had no idea how to put together moderate sized systems or above. One of the things that I like about Rails is that they have a system for plugins. I was too uncouth to consider it before. Once you extract a plugin out, you essentially have to treat it as a separate library or package. This forces you to encapsulate, because it's a bit of a pain changing plugins. I know you can extract code out to DLLs or Gems, but I never did it because it seemed like I didn't need it--plus it was extra steps.

I eventually would want a programming language that lets me extract out libraries inside the language, and make its public interface RESTful, and auto-extern its path in SVN and whatever the equivalent is in distributed version control like git/darcs.

Recently, I've started using Piston. It's a ruby plugin manager that's essentially a wrapper around svn:externals. It's been pretty easy to use. In addition, my plugins won't get stale. However, when you install a plugin with piston, you only import the files. It doesn't actually execute the install.rb file, like /script/plugin install would. I'll try to see if I can submit a patch to piston later, but for now, simply run "install.rb" in the root of the plugin, and it should do the install for you. small tip!

Friday, January 25, 2008

Interfacing and distributing code as a language feature

Recently, I've been looking to send email from erlang. Despite all the cool things it does, its INets library doesn't have an SMTP client in it. Trapexit happened to be down this week, so I ended up hunting around on the web (next time, I should just ask on newsgroups).

I found ErlMail as part of an Erlang Software Framework suite, and it had a simple SMTP client in it. I was glad I didn't have to write one myself, but I found it didn't do authentication. it was written cleanly enough that I was able to patch in authentication without much of a problem, and submitted it to the maintainer.

The short experience made me pretty thankful that SMTP protocol is in plain text, rather than bit-packed. Since much of our general purpose languages are pretty good at manipulating text, it's comparatively easy to interface with it and write a wrapper for it.

However, it's a pity that for every new language that comes out, a new wrapper must be written for an SMTP client. Same goes for REST/SOAP interfaces for any number of APIs that we see out there, from Google maps to facebook apps.

Part of the problem is the mismatch in syntax of the protocol/interface and the language the programmer is working in. Simply looking at REST interfaces, there's no programming language that makes native method calls like /post?title=32?body=f1rst%20post. Not that would make sense to do so since there are other RPC (remote procedure call) protocols too.

The only way I can see out of it is for a language(or library) dynamically generate code that maps a native call syntax to a RPC syntax. For every new type of RPC interface (REST, SOAP, JSON, etc.), we'd have a single file that describes the mapping. Then, when a new service or platform comes out with a REST interface, you don't need to write a new wrapper for that interface.

However, lots of languages have different method invocation capabilities. Some need types to arguments, others can take first class functions or blocks, and still others can take an unknown number of arguments. I don't think it would be easy, but it'd be nice to have so we stop wasting our time writing interface wrappers. Maybe Lisp was on to something when it said code is data and data is code.

Thursday, January 24, 2008

XMPP for machines

Jive Talks: XMPP (a.k.a. Jabber) is the future for cloud services

I found this great because it gave me a way to think about something familiar in a different way. I'm use to thinking about XMPP as just an IM and presence protocol, used by applications that let humans to communicate with other humans. But I didn't take it one step further and think of it as a messaging service between machines, mostly because I was under the impression that polling problem was solved (by the likes of Comet).

If this is possible, then by the same token, one should be able to run a "IMsite" over XMPP, analogous to a "web site" over HTTP. It's just that there currently is no "browser" for XMPP. If there were, you can technically send DOM updates or javascript (or whatever the browser can interpret) over XMPP. I imagine one should be able to take the mozilla engine and tack XMPP instead of HTTP in front (probably easier said than done).

That way we should be able to build browser apps that need near-real-time updates. The obvious one is chat. In fact, most of our XMPP clients are specialized to do that. Other applications are collaboration software, like a shared whiteboard (if sending SVG over XMPP would not be a bandwidth hog). Video lectures with auto advancing slides might be another one. Fleet tracking might be another. MMORPGs would also be easier to write on such a platform. It'd be interesting to see where this goes.

Update: Looks like people already tacked XMPP onto Mozilla

Setting time in your fixtures

In your rails fixtures, you should be able to set the time dynamically. If you set your database to default to UTC, make sure you use:

id: 1
body: "This is my witty post"
created_on: <%= 3.days.ago.utc.xmlschema %>

Unless you want to wreck havoc in your test cases. tip!

Monday, January 14, 2008

Getting inline-block working across browsers

In mobtropolis, there's a gallery of pictures that I have to show. While, normally, it's not too hard showing them as inline elements, if you just have pictures, it gets a bit tougher if you have a bunch of stuff that you have to inline correctly. Sites like facebook solve the problem by using a fixed layout, so you know the width of the area you can work in, and can thus use a table.

When you have a stretchable layout, however, that doesn't work. And what you need to do is use CSS, "display: inline-block" The problem is, it doesn't have consistent support across browsers. Only Opera and Safari use "display: inline-block" and "display: inline-table" correctly. IE6 and Firefox both use "display: inline" only and don't recognize inline-table and inline-block.

So thank goodness for design blogs out there: Align List Items Horizontally with CSS . His solution is a bit of a work around, but it prevents me from writing any javascript when I didn't need to.

.ib-fix li { display:-moz-inline-box; -moz-box-orient:vertical;
display:inline-block; vertical-align:top; word-wrap:break-word; }
* html .ib-fix li { display:inline; }
* + html .ib-fix li { display:inline; }
.ib-fix li > * { display:table; table-layout:fixed; overflow:hidden; }

I should add that you need to also add this to get it to work in IE6, and get rid of the weird padding on the left:

list-style: none inside none;
padding: 0px;

Yay. small tip! As a bonus, I also found out about ie7.js, a javascript library that fixes incompatibilities in ie7 and ie6.

The possibility of a reshapable keyboard

I have a bunch of hard drives that failed. I suspect that it's the controller card that is broken, but either way, I can't use it. What to do with old hard drives? That's when I started looking around, and it ends up that there are rare-earth magnets inside. (as well as a voice actuator that you can hook up to an amp to get hard drive speakers.)

And thus, I found a long article on rare earth magnets. Magnets have long fascinated people, as it makes all sorts of things possible, like speakers, hard drives, motors, and generators. But I didn't know about magnetic braking and that you can buy ferrofluids (also described in the article).

Ferrofluids are liquids that responds to a magnetic field. When you put a magnetic field near it, it responds by getting spiky. The stronger the field, the more dense the spikes. I was able to play with some in a enclosed sac once. It's kinda weird. You can actually feel resistance in the liquid when you put a magnet by it, like something's in the liquid.

While the optimus keyboard lets you re-display the keys in any way you wish, I've always wanted a keyboard that I can reshape. I'd rather have the keyboard actually be a membrane stretched over a flat rectangular plate. And depending on the application, the membrane would be able to take on different shapes. So instead of having keys when I'm looking at a map, the "keyboard" would be in the shape of the terrain I'm manipulating. Then I can pan and tilt. If I'm flying a plane, I'd rather have a joystick I can manipulate. I suppose you can make a rudimentary one with ferrofluids in an enclosed membrane. Not only can you reshape the liquid with controlled electromagnetic fields, but you should also be able to detect human interaction with the membrane by how it changes the magnetic field.

If a reshapable keyboard were to exist, you can also hook up two together through the internet. That way, you can interact with other people through touch, and not just text. If I put my hand on the reshapable keyboard and push down, the connected keyboard at the other end should have an imprint of my hand, pushing up out of the membrane. I'd also be able to augment my interactions so that my hand can appear to be holding something that it might not really be on my end. An inane thing would be to play paper, rock, scissors, where instead of the hand gestures, you'd actually see a sheet of paper, a chunk of rock, or pair of scissors rise out of the reshapable keyboard. A more useful application might be to keep family members or loved ones in touch--literally.

And if the membrane were embedded with OLEDs then it can be possible to add color to the membrane, so the interface would be something you can directly manipulate.

When I dreamt this up, I was thinking of gaming applications or remote surgery. Imagine the kind of fun and good you can do with the technology! However, after a bit of thought, I think a more likely scenario is that geeks adopt it for that, and then the porn industry makes it widespread. Just wait and see.

Thursday, January 10, 2008

Mobtropolis bruhaha: an interview

Chicago, unlike many homogenized cities in the US, has character. And Chicagoans are pretty proud of the uniqueness of their city, from the losing streak of their Cubs to how you should never put mustard on a hot dog. As far back as I can remember, Chicago TV stations would have shows that cover the various local unique flavors of Chicago and places you can visit.

Thatcher Kamin, producer at the local TV station, channel 26 WCIU does a show called the Chicago Insider. The show not only covers venues, but also local Chicago people doing different things, mostly artists, musicians, and the occasional business. Thatcher found me and wanted to do an interview about Mobtropolis, and as far as I know, I'm the only programmer on there.

Since the powers that be at the TV station don't quite understand the web yet (through no fault of Thatcher), they didn't provide an embeddable video, nor a link to the video itself. Thus, I'm forced to just provide you with instructions.

Simply visit, and click on "play" at the little flash thing up top.

It was an experience getting interviewed, and I can now understand why actors don't like watching their own films. You notice all sorts of stuff you never knew about yourself before--like twitching your eyebrows. I can't watch it more than once myself. There was a lot of stuff cut out, and I had no idea what was going to make it and what wasn't. All in all, it was fun, but I can see I need to work on my elevator pitch much more. Well, if anything, I hope you're entertained by the funky beats in the background, and give Mobtropolis a shot if you were amused by the interview.

Wednesday, January 09, 2008

Nerd time Issue 14

I use to work at a research lab, and while they do know what's going on with military tech, they don't really get news of consumer web technologies. I read up on it alot while I was there, and I still do now. I started a little mailing list after I left to keep them updated, and that's what became Nerd time. Here is the latest for your enjoyment.

Hey all,

Happy 2008! It's been over a month since the last nerd time. At first, it was because not much was happening. I've found that Silicon Valley(especially the software/web side of things) is much like Hollywood in the sense that there are fads and fashions that people make ruckus about and then ditch after a month. And it's like sensationalism news when nothing noteworthy is happening, but people still find lots of trifle things to make noise about. So even though we attempt to filter it all through our favorite feeds and social news sites, it's still a chore.

I met up with Ray the first time since College, and he said he feels out of touch with the tech world, even though he's a chip designer. All he has to do is read nerd time! You get my own biased, hand-picked version of what's noteworthy.

Mike and Nick, check out Johnny Lee and Buglabs, being hardware freaks that you are.

As usual, the easy stuff is up top. If you want to stretch your brain, go further down.

Johnny Lee's wonderful world of HCI
Wiiremote VR tracking, Wii multitouch, Automatic registration of projection screens. It's this researcher's venture into HCI. more melding of computing into everyday lives. The videos are more self explanatory. If you skip everything else, I suggest you don't skip these.

Facebook, Google, and Plaxo join DataPortability Workgroup
I strongly believe that a users's data belongs to the user, and they should be able to take it around with them. I was pretty excited about Google's announcement about OpenSocial platform around October or so. I haven't personally taken a look at it myself, but according to pips here and there, it's been said it's incomplete and there was a case of a security hole. But this announcement of fb joining a data portability workgroup is good news, as making your data portable with you across applications will go a long way in making using computers a lot easier.

This is a startup that you redirect all your snail mail to, and it opens and scans it in for you. You basically can read snail mail like you do email. You can even choose to recycle junk mail (good for the earth), and shred important documents. Neat idea, though of course, it all has to rely on trust of the company. As the social-political climate of the US over foreign energy dependence gains awareness with the push for 'green', we might see more companies like this that tries to move things into digital to save on green.

Google's philanthropic arm going to develop clean energy cheaper than coal.
Just because something is digital doesn't necessarily mean it has a small environmental footprint. The servers it's running on might be burning coal. I think it's rather big, actually. It remains to be seen whether Google will democratize energy production, to shake up a very traditional energy industry, or simply join as another player. Regardless, this venture makes sense for them, as they run gigantic server farms.

Prediction Markets at GooglePlex study organization information flow
Google uses internal prediction markets to gather data on what's going on. Prediction markets are basically using the mechanics of stock markets to predict things like, "is google stock over valued?", "Will the U of I win the Rose bowl?", "Will Comet win mindshare with developers?" This particular article talks about mapping information flow in organizations. I think as the world moves faster and faster, mid-sized companies are exploring more flexible organization structures to be able to respond quicker to changing markets. You'll probably see more studies into company organization in the future.

This is interesting because it's another indication of how real-life is merging with digital. Any website monitors its traffic to know which pages are more popular than others. It uses that as implicit feedback to what customers like. This does the same for physical storefronts. Using bluetooth on customer's cell phones to track foot traffic, they can see which racks and displays are selling more. As more and more sensors enter into the world, we'll be able to do analysis and better understand our lives. I won't talk about privacy concerns, cuz that's just a can of worms.

Amazon SimpleDB and computation on tap
I wish I brought Amazon stock. The more I find out about Jeff Bezos, the more I understand that it's not really a bookstore that he built, but a philosophy. Anyway, Amazon, in this past year has been releasing "computation on tap" They basically sell distributed computing and storage services to developers and startups. Lots of people have taken advantage of it, for good reasons you can read elsewhere. In the past month, they released simple db, which is really just a fast index db. With computing on tap, it enables small teams to do large things. If computing and storage is a commodity, I suppose the value becomes what you can build with it.

WiTricity wireless power
Wireless power has been in development for some time now. I had mentioned it in one of my earlier nerd times. This one, although just experimentally demo'd, is rather neat. We probably won't see wireless power more ubiquitous until maybe 5 years later. I don't quite understand it myself (should have paid attention in PHYS 112), but apparently, it's not radiation like we're familiar with. Instead, it uses magnetic resonance, so that only two electromagnetic devices with the same resonance will transfer energy efficiently, everything else, like humans, won't.

This is a startup making mobile devices that you can hack. They plug together like lego pieces and run linux. People can prototype and experiment with different mobile form factors and fulfill their niche needs.

Quagmire 2D programming language
This is a 2D virtual machine, and you can watch it as it uses its memory visually on a 2D surface. What's interesting is that the latter examples looks like chaos, which reminds me of the pictures you see in "A New Kind of Science" In it Wolfram claims that basic computers nowadays just have simple loop patterns rather than the complex ones you'd see in class 3 CAs. Problem is, we don't know yet exactly what class 3 CAs compute better, and how to harness it, other than CA120 is turing complete.

Comet - AJAX's cousin
Comet is using AJAX to push information to browsers, rather than having browser poll the server. The term was coined about two years ago, in reference to the bathroom cleaner. So you can get stuff like chat working in browsers, or like a stock market feed.

Sunspider - javascript benchmarking
Javascript is getting faster and faster with work being donw on the underlying engines underneath popular browsers. I've seen "lemmings" implemented completely in javascript. You'll continue to see this sort of thing. However, flash, silverlight, and flex are still far ahead of the curve when it comes to interactivity of the web.

Version control in emacs
The new emacs 22 has svn version control built it! *high fives Ian*

Tumble logs are the half cousin of twitter and blogs. They represent a flow of conciousness digital media tidbits from the poster's daily life. We might start to see more of this if it catches on. However, other than personal expression, I don't exactly see a future for it. But then again, I was wrong about Friendster when it first came out.

Next gen ANNs
A talk on next generation Artificial Neural Networks. I haven't watched it yet, but will this weekend. It promises to be pretty exciting.

Ycombinator in Ruby
I finally figured out what ycombinators are. If I had to take a stab at it, it's a way to implement recursion when you don't have named recursions, loops, iterators, etc, and all you have are functions and a couple substitution rules. It's pretty neat, and will stretch your brain a bit to understand it. Traditionally, you only do this in lambda calculus, but having the code in Ruby made it easier to understand.

Wednesday, January 02, 2008

DRYing up your views with a TableBuilder

In the last two months, when I was adding more features to mobtropolis, I found it painful to try and lay things out all over again from scratch. As a result, it sucked to see ugly layouts on the new pages juxtaposed with all the styling I had done before. It wasn't until a week ago that I said to myself, "Stop the madness!" and started refactoring my views--something I never thought of doing much of until now. When you don't, the barrage of angle brackets blows out of proportion and it starts to look pretty damn fugly with complex views.

What I should be able to do is take common mini-layouts in my views and make them available as helpers so that I can think in terms of bigger chunks to piece together pages, rather than in divs and spans. In addition, it makes your interface more consistent for your users.

Some good resources were presentations from the 2007 RailsConf, like V is for Vexing and Keeping your views DRY. While a lot of view DRYing talks about form builders, I didn't see any on table builders, so I decided to take a stab at it. Personally, I don't like to overuse tables for layouts. But as certain elements in my page layouts have been repeated, I refactored them into first helpers, and then when I did more than one, I extracted it out into a simple table builder. This is how you'd use it:

For example, I have a mini-layout where I show simple stats:

Here's how I used a simple table builder to display the above:

<% score_card do |sc| -%>
<% sc.placard("Votes", "", num_voters, "") -%>
<% sc.placard("Sceneshots", "", num_sceneshots, "")-%>
<% sc.placard("Participants", "", num_participants, "") -%>
<% sc.placard("Challenges","", num_challenges, "") -%>
<% end -%>

And I find that I started using the same sort of thing in other places, like in a user's profile:

<% score_card do |sc| -%>
<% sc.placard("Submitted Scenes", "", num_scenes, "") -%>
<% sc.placard("Submitted Sceneshots", "", num_sceneshots, "") -%>
<% end -%>

I cut out some details so you can see that it's just a block that gets passed a ScoreCard object, from which you call placard to add another score to the score_card. It sure beats writing <table> and <td> over and over again.

To declare the helper, we create a helper with the structure of the table inside the declaration of a ScoreCard object. We have a ScoreCard object to hold the contents of the placards. When they're called in the block above in the template, they get stored in the ScoreCard object, and not written out to erb immediately. That way, we can place them wherever in the table we please, by making the call to card.display(:placards):

module ScorecardHelper
def score_card(html_options = {}, &block)
options = { :class => :scorecard, :width => "100%" }.merge(html_options) do |xm, card|
xm.table(options) do => :top) do
xm << card.display(:placards)

So then what's ScoreCard look like? Pretty simple. It has a call to each cell that can be filled in the mini-layout. It's kinda analogous to how form_for passes in a form object, on which you can call text_field, etc.

require 'lib/table_builder'

# Used to create a scorecard helper
class ScoreCard < TableBuilder
cells :placards

# a placard is placed into scorecards
def placard(text, text_id, score, widget)
xm =
@placards[:html] += do
xm.span(:style => "font-size: 1.2em") { xm << "#{text}" }
xm.em(:id => text_id, :class => "primary_number") { xm << "#{score}" }
xm << "#{widget}"


Notice that there's a call to cells() to initialize the type of cell, and a method of the same name that builds the html for that cell. If you have other types of cells, you simply put it in the list of cells, and then create a method for it that is called in the template. By convention, you'd stick the html of the cell contents in "@#{name_of_cell}"[:html], and if you wanted, pass in the html_options, and stick it in "@#{name_of_cell}"[:options]. Then, you can access those in the helper wherever you want.

Let's try another one. I have a mini_layout with a picture, and some caption underneath it, like a polaroid.

<% polaroid_card do |p_card| -%>
<% p_card.picture do -%>
<%= sceneshot_for scene -%>
<% end -%>
<% p_card.caption do -%>
<%= pluralize(scene.num_of_sceneshots, "sceneshot") %>
<% end -%>
<% end -%>

The associated helper and PolaroidCard object are pretty simple:

module PolaroidCardHelper
# a polaroid card is used to show a picture with a caption below it.
def polaroid_card(html_options = {}, &block)
options = { :class => :polaroidcard, :style => "display: inline;" } do |xm, card|
xm.table(options) do { {
xm << card.display(:picture)
}} { => "caption")) {
xm << card.display(:caption)

require 'lib/table_builder'

class PolaroidCard < TableBuilder
cells :picture, :caption

def picture(html_options = {}, &block)
@picture[:html] = capture(&block)
@picture[:options] = html_options

def caption(html_options = {}, &block)
@caption[:html] = capture(&block)
@caption[:options] = html_options

I've tried to pull all the plumbing out into TableBuilder (dropped it into lib/), and only leave the flexibility of creating the table structure in the helper, and the format of the cell in the object. It ends up TableBuilder isn't too complex either. It uses some metaprogramming to create some instance variables. I know it pollutes the object instance variable namespace, but I wanted to be able to say @caption[:html], rather than @cells[:caption][:html].

class TableBuilder < ActionView::Base
class << self
# used in the child class declaration to name and initialize cells
def cells(*names)
define_method(:initialize_cells) do
@cell_names = { |n| "@#{n}".to_sym }
@cell_names.each do |name|
if instance_variable_defined?(name)
raise"name clash with ActionView::Base instance variables")
instance_variable_set(name, { :html => "", :options => {} })

def initialize(decor_block, &table_block)
html =, self)
concat(html, decor_block.binding)

def display(cell_name)

def html_options(cell_name)


I've found have these helpers cleans up my views significantly, though I have to admit, it's not exactly easiest to use yet. In addition, I'm not exactly thrilled about having TableBuilder inherit from ActionView::Base, but it was the only way I could figure out to get the call to concat() to work. In any case, the point is to show you that refactoring your views into helpers is a good idea, and even something like a table builder goes a long way, even if you don't do it the way I did it. Lemme know if this helps or hinders you. snippet!