Mushkin - Get More
If you follow my blog, it's really not often that I give plugs for products or services. But sometimes, when your experience is just overwhelming positive, you just have to give credit where credit is due.
Three years ago, I brought a Dell Dimension 2400. It's a 2.66GHz P4 computer--paltry by even by yesterday's standards, but it was cheap...about $500. I brought it on a whim, and since then, it's become my main development machine. However, I had only brought it with 512MBs of RAM...which I thought was more than enough at the time.
Enter Firefox 2 stage left. The elephant that easily eats up 230MBs.
Like I lamented before, I've been having productivity slow-downs due to applications using an extraordinary amount of memory. I was actually watching screens redraw itself because the swap partition was being exercised like crazy. I had sworn off Eclipse because I felt that for what I used it for, it was eating up too much memory. But at the end of the day, what room was left by Eclipse was taken up by Firefox 2.
In the end, I succumbed and went out to buy more RAM for my machine while waiting for Firefox 3. But it's been so long (2 years) since I brought hardware, I didn't know what to get. Kudos to Dell for providing easy ways to look up what RAM you need by its model number. I use to revel in being able to look up specs to find the lowest price, but I simply don't have the time anymore. Dell was offering 1GB RAM for $109. Seemed reasonable. But I decided to check Mushkin.
I've used Mushkin since my sophomore year in college (8 years ago!), and none of the RAM I've ever purchased from them have ever failed so far. Believe me, RAM fails. Probably not as fast as hard drives, but RAM fails. And when it does, it's the last thing you assume that has failed. As a result, you waste so much time in your dorm room debugging it instead of out there mixing it up with the co-eds.
I revisited mushkins for the first time in a long time, and I have to say that while they have a dizzying array of RAM from hardcore hardware enthusiasts (what I use to be), they also have easy-to-find components for owners of stock computer models (like for me now). Not only that, they offer the same type of RAM for $30's less. Tres awesome.
Normally, if I just saw Mushkin out of the blue, I don't know if I'd trust them. But having used them all these years, and given that their RAM keeps on ticking, they stand by their quality, however they do it. Not only is their stuff quality, but their shopping experience is a breeze.
So I'd recommend their stuff. If I get Mobtropolis to a point where I need huge servers, I know where I'm getting my RAM.
Ok, next time, will be more coding related stuff, as I have some interesting things to post that have been gathering in the queue. Stay tuned!
Saturday, September 29, 2007
Wednesday, September 26, 2007
[115, 117, 109, 109, 97, 114, 105, 122, 97, 116, 105, 111, 110].map{|c| c.chr}.join
Time does fly by when you don't work on the weekend. It's been almost a week since my last post--so you know I've been busy coding. Today, I got stuck on caching, so here I am, blogging.
In Nerd Time 8, I had mentioned the algorithm on content-aware image resizing. For those of you that didn't hear about it a couple weeks ago, watch the movie. It seems pretty magical at first. It basically computes an energy function for the image to decide which part it can cut out if it needed to.
I don't know if the rest of you had the same thought, but content-aware image resizing is essentially image summarization. You're throwing out less important information in the picture in favor of preserving informational features of the image.
They use an energy function as a metric to determine which seams--and in what order--to remove from the picture to reduce its size.
The most surprising thing to me was that the basic energy function is just the magnitude of the gradient function. A gradient function of an image tells you how fast the colors are changing as you're moving across the image. This means the sky would be smooth and slowly varying (low frequency), and the trees would be rough and varies quickly (high frequency). Therefore, the basic gradient energy function just allows you to selectively cut out the low frequency parts of the image while the seam selection preserves the aspect ratio and image coherence.
Apparently, this metric works pretty well, even compared to other metrics like entropy, which is the standard measure of information content. This works mostly because of the assumption that high content areas of the scene will be high frequency, and background images, like sky, road, wall, are generally low frequency images. If you had a picture of me and someone else holding up a flag with a forest as a backdrop, it'll cut out the flag first, not the trees, using the gradient energy function.
This puts text summarization into clearer focus for me. There are two competing goals in text summarization: 1) reduce the amount of text 2) keep the information content high and coherent. With content-aware image resizing, it was able to achieve both goals by finding a metric that was calculable to distinguish between important and non-important. So by comparison, we should be able to do the same with text.
However, we don't know what, if any, the gradient between words means, and how that would fare as a measure of information content. We also don't have a good way of judging coherency of a piece of text--different people will come up with different summaries. In an image, we can look and just tell. This is because we judge all pieces of an image in parallel, and we have a database of images to compare to in our heads to tell if something 'looks right' or not.
One can tell how far apart a color is from another simply by measuring the distance of the hex values that represent that color. However, words that have similar letters may have completely different meanings.
Many people have used co-occurrences of words to map words to meanings, since it makes sense that words related to each other would appear in the same text. However it was found that even if two words had the same meaning, they might have different frequency of occurrences, thus throwing off the validity of the gradient map.
In Nerd Time 8, I had mentioned the algorithm on content-aware image resizing. For those of you that didn't hear about it a couple weeks ago, watch the movie. It seems pretty magical at first. It basically computes an energy function for the image to decide which part it can cut out if it needed to.
I don't know if the rest of you had the same thought, but content-aware image resizing is essentially image summarization. You're throwing out less important information in the picture in favor of preserving informational features of the image.
They use an energy function as a metric to determine which seams--and in what order--to remove from the picture to reduce its size.
The most surprising thing to me was that the basic energy function is just the magnitude of the gradient function. A gradient function of an image tells you how fast the colors are changing as you're moving across the image. This means the sky would be smooth and slowly varying (low frequency), and the trees would be rough and varies quickly (high frequency). Therefore, the basic gradient energy function just allows you to selectively cut out the low frequency parts of the image while the seam selection preserves the aspect ratio and image coherence.
Apparently, this metric works pretty well, even compared to other metrics like entropy, which is the standard measure of information content. This works mostly because of the assumption that high content areas of the scene will be high frequency, and background images, like sky, road, wall, are generally low frequency images. If you had a picture of me and someone else holding up a flag with a forest as a backdrop, it'll cut out the flag first, not the trees, using the gradient energy function.
This puts text summarization into clearer focus for me. There are two competing goals in text summarization: 1) reduce the amount of text 2) keep the information content high and coherent. With content-aware image resizing, it was able to achieve both goals by finding a metric that was calculable to distinguish between important and non-important. So by comparison, we should be able to do the same with text.
However, we don't know what, if any, the gradient between words means, and how that would fare as a measure of information content. We also don't have a good way of judging coherency of a piece of text--different people will come up with different summaries. In an image, we can look and just tell. This is because we judge all pieces of an image in parallel, and we have a database of images to compare to in our heads to tell if something 'looks right' or not.
One can tell how far apart a color is from another simply by measuring the distance of the hex values that represent that color. However, words that have similar letters may have completely different meanings.
The difference between the almost right word & the right word is really a large matter--it's the difference between the lightning bug and the lightning. - Mark TwainI suspect that one would need to use a gradient map for words, or be able to use the etymology of words to measure how far apart the meanings of words are from each other. How to generate this map has been difficult, as far as I know.
Many people have used co-occurrences of words to map words to meanings, since it makes sense that words related to each other would appear in the same text. However it was found that even if two words had the same meaning, they might have different frequency of occurrences, thus throwing off the validity of the gradient map.
Wednesday, September 19, 2007
Diving into Rails source and explaining alias_method_chain in pictures
I've been involved with the local Ruby group, and I did a talk on diving into the Rails source code. I have to admit, I probably should have dove into it earlier, but there was a lot to learn just by using Rails, for someone that was doing engineering work before this. Originally, I was going to just relate my experience of when I had to dive into the code, but then I happened on Rails' Unusual Architecture, and it gave a good topic for introducing metaprogramming used in Rails.
At first, it was pretty confusing, since I came from a C++ background and was use to the idea of design patterns. Well, there are patterns, it's just that in dynamic languages, much of the traditional Gang of Four design patterns goes away.
I'll outline the meat of the talk. Rails doesn't use the Decorator Pattern. Instead it basically renames methods in calls to inject additional functionality that wraps around the original functionality. How does it do this? It uses something called alias_method_chain, detailed here and here.
And inside alias_method_chain, is a native method called alias_method. Since everything in Ruby is an object, and changeable, that means methods of modules and objects are also changeable. It basically makes a copy of the old method and calls it something new.
I made pictures to show the progression. Let's say we have a method called "save" that we want to enhance with "validation".
We start with ActiveRecord and the Validation module. So in our validation module, we make a method called "save_with_validation"(purple heart) that calls a method called "save_without_validation" inside it. "save_without_validation" doesn't exist yet, because we haven't called alias_method_chain().
Then our first step is to include the Validation module inside of ActiveRecord.
Then when we call alias_method_chain(:save, :validation), using alias_method, it'll make a copy of the original save method, and call it "save_without_validation".
Then, it'll rename the method inside of validation from "save_with_validation" to "save". This way, any code calling save() on ActiveRecord will execute the new save, which does validation first, and then turns around to call the original save (green sun). The client won't know any different, but in fact, code was injected between the original save (green sun), and the caller inside of the new "save" (purple heart). And the original save doesn't know any different either, since nothing's changed from its point of view.
In code, it'll look something like this:
Which if we 'executed' the above as we did in pictures, it ends up being equivalent to:
I'm not sure how I feel about it as of yet, but at first glance, it's a nice pattern once you know what's going on. At first, I saw all these methods being called in the Rails source which aren't actually defined anywhere. It ends up it's because of metaprogramming stuff like this going on. I think other people have said this is bad idea because it's hard to inject functionality in the middle of chains that already exist. If you pick up one, you pick up everything before it. One might argue the same is true of Decorator patterns.
In any case, you'll see this repeated over and over again in the Rails source, so hopefully, this'll give you some idea of what's going on if you ever decide to go Rails splunking.
At first, it was pretty confusing, since I came from a C++ background and was use to the idea of design patterns. Well, there are patterns, it's just that in dynamic languages, much of the traditional Gang of Four design patterns goes away.
I'll outline the meat of the talk. Rails doesn't use the Decorator Pattern. Instead it basically renames methods in calls to inject additional functionality that wraps around the original functionality. How does it do this? It uses something called alias_method_chain, detailed here and here.
And inside alias_method_chain, is a native method called alias_method. Since everything in Ruby is an object, and changeable, that means methods of modules and objects are also changeable. It basically makes a copy of the old method and calls it something new.
I made pictures to show the progression. Let's say we have a method called "save" that we want to enhance with "validation".
We start with ActiveRecord and the Validation module. So in our validation module, we make a method called "save_with_validation"(purple heart) that calls a method called "save_without_validation" inside it. "save_without_validation" doesn't exist yet, because we haven't called alias_method_chain().
Then our first step is to include the Validation module inside of ActiveRecord.
Then when we call alias_method_chain(:save, :validation), using alias_method, it'll make a copy of the original save method, and call it "save_without_validation".
Then, it'll rename the method inside of validation from "save_with_validation" to "save". This way, any code calling save() on ActiveRecord will execute the new save, which does validation first, and then turns around to call the original save (green sun). The client won't know any different, but in fact, code was injected between the original save (green sun), and the caller inside of the new "save" (purple heart). And the original save doesn't know any different either, since nothing's changed from its point of view.
In code, it'll look something like this:
class ActiveRecord
def save
# do saving stuff
end
end
module Validation
def save_with_validation
# do validation
save_without_validation
end
end
class ActiveRecord
include Validation
alias_method_chain :save, :validation
endWhich if we 'executed' the above as we did in pictures, it ends up being equivalent to:
class ActiveRecord
def save_without_validation
# do saving stuff
end
def save
# do validation
save_without_validation
end
endI'm not sure how I feel about it as of yet, but at first glance, it's a nice pattern once you know what's going on. At first, I saw all these methods being called in the Rails source which aren't actually defined anywhere. It ends up it's because of metaprogramming stuff like this going on. I think other people have said this is bad idea because it's hard to inject functionality in the middle of chains that already exist. If you pick up one, you pick up everything before it. One might argue the same is true of Decorator patterns.
In any case, you'll see this repeated over and over again in the Rails source, so hopefully, this'll give you some idea of what's going on if you ever decide to go Rails splunking.
Sunday, September 16, 2007
Syntactic sugar for dealing with empty containers
In any web application, we're often just reading a collection of rows from the database and displaying it in the browser. Often times, we'll have code that looks like this:
<% unless @friends.empty? -%>
<% @friends.each do |friend| -%>
<li><%= h friend.username %></li>
<% end -%>
<% else -%>
No friends yet
<% end -%>
<% if @friends.each do |friend| -%>
<li><%= h friend.username %></li>
<% end.empty? -%>
No friends yet
<% end -%>
<% @friends.each do |friend| -%>
<li><%= h friend.username %></li>
<% end.empty do -%>
No friends yet
<% end -%>
class Array
def empty(message = "")
if self.empty?
return block_given? ? (yield message) : message
end
end
end
Thursday, September 13, 2007
Unable to freeze rails due to problem in rake task
I think the current version of stable Rails is 1.2.3. For those of you using this version, rather than Edge Rails, there's a little gotcha in the rake tasks.
Since I'm on a shared host, it's good practice to freeze your version of rails into the vendor's directory. You do this by using a rake task, per "rake rails:freeze:gems" But before you do that, if you're using SVN, you'll want to use "svn delete" to remove the vendors/rails directory. None of the rake tasks use SVN delete. They all use "rm -rf", which in my experience makes SVN freak out if the .svn directory is gone.
However, even with that done, freezing a new version of gems was failing.
It was looking for rails version 1.4.0, and not being able to install it. And even worse, when you try to run rake again, it said it couldn't find it!
Well, the latter was simple. A failed freeze leaves a blank vendor/rails directory, and if you look in 'config/boot.rb', it says:
So make sure you remove vendor/rails.
The latter took a little bit of work digging around the rake tasks, and though it wasn't hard, I wasted about an hour. It ends up that the culprit is that the default rake task uses Gem.cache.search('rails'), which returns all gems with the name 'rails' in it.
I have a couple gems installed with the word 'rails' in it.
So it took the latest one, which was 1.4.0, and tried to install rails 1.4.0, which doesn't exist!
To hot fix it, the railities/lib/tasks/framework.rake file, under the freeze namespace and gems task, change "Gem.cache.search" to "Gem.cache.find_name"
That way, it only finds 'rails', and not all the other games with 'rails' in the name of the gem. This problem is solved in edge Rails, so no need to submit a patch. Tip!
Since I'm on a shared host, it's good practice to freeze your version of rails into the vendor's directory. You do this by using a rake task, per "rake rails:freeze:gems" But before you do that, if you're using SVN, you'll want to use "svn delete" to remove the vendors/rails directory. None of the rake tasks use SVN delete. They all use "rm -rf", which in my experience makes SVN freak out if the .svn directory is gone.
However, even with that done, freezing a new version of gems was failing.
It was looking for rails version 1.4.0, and not being able to install it. And even worse, when you try to run rake again, it said it couldn't find it!
Well, the latter was simple. A failed freeze leaves a blank vendor/rails directory, and if you look in 'config/boot.rb', it says:
if File.directory?("#{RAILS_ROOT}/vendor/rails")
require "#{RAILS_ROOT}/vendor/rails/railties/lib/initializer"
else
require 'rubygems'
...blah blah blah..So make sure you remove vendor/rails.
The latter took a little bit of work digging around the rake tasks, and though it wasn't hard, I wasted about an hour. It ends up that the culprit is that the default rake task uses Gem.cache.search('rails'), which returns all gems with the name 'rails' in it.
I have a couple gems installed with the word 'rails' in it.
rails (1.2.3, 1.2.0, 1.1.6)
rails_analyzer_tools (1.4.0)
railsbench (0.9.2)
So it took the latest one, which was 1.4.0, and tried to install rails 1.4.0, which doesn't exist!
To hot fix it, the railities/lib/tasks/framework.rake file, under the freeze namespace and gems task, change "Gem.cache.search" to "Gem.cache.find_name"
That way, it only finds 'rails', and not all the other games with 'rails' in the name of the gem. This problem is solved in edge Rails, so no need to submit a patch. Tip!
Tuesday, September 11, 2007
Nerd time, issue 8
I used to work at a research laboratory, where the fanfare and fads of the web aren't of great concern. But the co workers liked to know what was going on outside the ivory towers, so after I quit, I sent them an informal mailing list. I post it on here just for fun too.
---
So after a little hiatus deploying mobtropolis, nerd time is back. As
usual, easy reading is up top. This time it's on databases. Dbs and
backends usually inspire yawns, because frankly, they're not
sexy--there's no pretty screens to look at. However, dbs are often a
bottle neck, and scaling beyond the usual db configs has been a source
of pain for large scale software. Here, I point out some relatively
obscure db stuff on the horizon--after some easy reading and news.
And oh, if you don't want to get these anymore, just lemme know.
A group is its own worst enemy
Nothing to do with dbs. Just a classic piece of text on social
software. Easy reading, but good lessons for me when building
http://www.mobtropolis.com
http://www.shirky.com/writings/group_enemy.html
Firefox 3 with XUL runtime
I did comment on this, and FF3 should be less prone to crashes, unlike
FF2, and the significance of this is much like Adobe's Integrated
Runtime(AIR), web devs will be able to create native desktop
applications using the usual web tools--HTML, javascript,
actionscript, XML, etc.
http://arstechnica.com/journals/linux.ars/2007/08/21/using-firefox-3-as-a-xul-runtime-environment
http://webjazz.blogspot.com/2007/08/using-firefox-3-as-xul-runtime.html
Adobe also open sourced their Photoshop engines. Offhand, I'm not
sure what one would do with it, unless there were some type of
innovative image manipulation--of which you'll see on the next link
http://opensource.adobe.com/group__asl__overview.html
Content-aware image resizing.
This is kinda neat. It uses energy functions to resize images while
keeping important content, and killing out background parts of the
images. If you don't click on any of the links but one, I'd click on
this one.
Update:
Well, what's interesting is that the basic energy function they used is simply a two-dimensional gradient. It's under the assumption that high frequency image content is usually what contains information/foreground/interesting parts of the image. This is probably usually true, and probably works for a large number of images. However, I think if you had an image of a flag with a forest as the background, it'll cut out the flag first.
http://www.youtube.com/watch?v=c-SSu3tJ3ns
http://www.faculty.idc.ac.il/arik/imret.pdf
Byte-serving is an aspect of the HTTP protocol that I didn't know
about. Apparently, you can request specific parts of a file over
http. Web-based bittorrent?
http://www.coneural.org/florian/papers/04_byteserving.php
hBase - Google bigtable open source clone. Bigtable is a in-house
developed distributed database. I watched a video lecture of it one
time, and it seems pretty neat.
http://glinden.blogspot.com/2007/07/hbase-google-bigtable-clone.html
A free database of the world's spec-related knowledge in one place.
Oddly enough, it is populated with things. I'm not sure what
motivates people to enter things in, but probably the same motivation
as people contributing to wikipedia. The neat thing about this is
that you can query it with an API.
http://www.freebase.com/signin/
CouchDb is an database that doesn't use relational tables. Mostly for
documents. It's still in alpha.
http://couchdb.org/CouchDB/CouchDBWeb.nsf/Home?OpenForm
Ambition is an experimental ruby gem that makes SQL queries as Ruby's
Enumerable functions. Web devs seem pretty allergic to SQL in general
and has tried to build layers between the dev to have one less
language to learn. Probably also the result of wanting a 3 tiered
architecture too.
http://errtheblog.com/post/10722
Mnesia is Erlang's distributed DB. I'm under the impression that it
doesn't use SQL. One queries directly by using Erlang tuples. I'll
have to learn more about this one.
http://www1.erlang.org/documentation/doc-5.0.1/lib/mnesia-3.9.2/doc/html/part_frame.html
---
So after a little hiatus deploying mobtropolis, nerd time is back. As
usual, easy reading is up top. This time it's on databases. Dbs and
backends usually inspire yawns, because frankly, they're not
sexy--there's no pretty screens to look at. However, dbs are often a
bottle neck, and scaling beyond the usual db configs has been a source
of pain for large scale software. Here, I point out some relatively
obscure db stuff on the horizon--after some easy reading and news.
And oh, if you don't want to get these anymore, just lemme know.
A group is its own worst enemy
Nothing to do with dbs. Just a classic piece of text on social
software. Easy reading, but good lessons for me when building
http://www.mobtropolis.com
http://www.shirky.com/writings/group_enemy.html
Firefox 3 with XUL runtime
I did comment on this, and FF3 should be less prone to crashes, unlike
FF2, and the significance of this is much like Adobe's Integrated
Runtime(AIR), web devs will be able to create native desktop
applications using the usual web tools--HTML, javascript,
actionscript, XML, etc.
http://arstechnica.com/journals/linux.ars/2007/08/21/using-firefox-3-as-a-xul-runtime-environment
http://webjazz.blogspot.com/2007/08/using-firefox-3-as-xul-runtime.html
Adobe also open sourced their Photoshop engines. Offhand, I'm not
sure what one would do with it, unless there were some type of
innovative image manipulation--of which you'll see on the next link
http://opensource.adobe.com/group__asl__overview.html
Content-aware image resizing.
This is kinda neat. It uses energy functions to resize images while
keeping important content, and killing out background parts of the
images. If you don't click on any of the links but one, I'd click on
this one.
Update:
Well, what's interesting is that the basic energy function they used is simply a two-dimensional gradient. It's under the assumption that high frequency image content is usually what contains information/foreground/interesting parts of the image. This is probably usually true, and probably works for a large number of images. However, I think if you had an image of a flag with a forest as the background, it'll cut out the flag first.
http://www.youtube.com/watch?v
http://www.faculty.idc.ac.il
Byte-serving is an aspect of the HTTP protocol that I didn't know
about. Apparently, you can request specific parts of a file over
http. Web-based bittorrent?
http://www.coneural.org
hBase - Google bigtable open source clone. Bigtable is a in-house
developed distributed database. I watched a video lecture of it one
time, and it seems pretty neat.
http://glinden.blogspot.com
A free database of the world's spec-related knowledge in one place.
Oddly enough, it is populated with things. I'm not sure what
motivates people to enter things in, but probably the same motivation
as people contributing to wikipedia. The neat thing about this is
that you can query it with an API.
http://www.freebase.com/signin
CouchDb is an database that doesn't use relational tables. Mostly for
documents. It's still in alpha.
http://couchdb.org/CouchDB
Ambition is an experimental ruby gem that makes SQL queries as Ruby's
Enumerable functions. Web devs seem pretty allergic to SQL in general
and has tried to build layers between the dev to have one less
language to learn. Probably also the result of wanting a 3 tiered
architecture too.
http://errtheblog.com/post
Mnesia is Erlang's distributed DB. I'm under the impression that it
doesn't use SQL. One queries directly by using Erlang tuples. I'll
have to learn more about this one.
http://www1.erlang.org/documentation/doc-5.0.1/lib/mnesia-3.9.2/doc/html/part_frame.html
Thursday, September 06, 2007
Is preloading child tables always a good idea?
Optimization isn't something you should do too early on, but I think a little house cleaning every so often to make sure your pages aren't ridiculously slow is healthy. With any optimization task, you'd want to benchmark the results and see if there's an actual gain. The very basic tool for benchmarking is the ordinary script/performance/benchmark. The easiest to find analysis tools is the rails_analyzer gem. The last time I used rails analyzer, it wasn't that easy to use. The command line arguments seemed arcane. But its bench tool, which can benchmark controllers as opposed to just object models, is fairly easy to use.
Using the bookmarking example from before, let's say you have something like:
In the listing of books, one would display whether it's actually bookmarked by a user or not. Normally, without the :include, the listing would make repeated queries to the DB every time it displayed a book list element, since it will use bookmarked_by?(user_id) to determine if a user bookmarked the book. So instead of just 1 query, it would make n + 1 queries.
Preloading child tables isn't necessarily wise all the time. It really depends on what you intend to do with the data after you fetch it. As the Agile rails book warns, preloading all that data will take time. If you look at your log files, you'll see that it's a significant amount.
If you're only going to load a limited number of these book list elements on a single page at a time, it actually might make sense to forgo preloading of child tables, and just use a find() instead of a select.
And if you're going to display counts of arrays, but all means, use counter caching. It's easy to do (as long as you follow instructions!), for most situations.
Intuitively, if you want to display over a certain n number of book list elements, it makes more sense to use :include and select it. However, I wanted to point out that when you make decisions like this, you'll always want to measure the load times, because you earn what you measure.
Also, use the right number of runs. Too short number of a number of times you run a function, the more variation you'll have in your benchmarks. Let's say that you get two numbers for two different methods.
So it's obvious that method2 is better right? Well, not necessarily. While benchmarks only show averages, you'll need to pay attention to standard deviations. The bigger the standard deviation, the more runs you'll need to figure out the average load time, and the number of decimal points you can trust. That way, you can figure out whether the difference in load times is statistically significant or not.
That way, you can ascertain whether the optimization you made were worth the trouble or not. tip!
Using the bookmarking example from before, let's say you have something like:
class SceneController < ApplicationController
def list
@books = Book.find_books
end
end
class Book < ActiveRecord::Base
def self.find_books
find(:all, :include => [:bookmarks],
:conditions => ["books.created_on > ?", 6.month.ago])
end
def bookmarked_by?(user)
self.bookmarks.select { |bm| bm.owner_id == user.id }.empty? ? false : true
end
endIn the listing of books, one would display whether it's actually bookmarked by a user or not. Normally, without the :include, the listing would make repeated queries to the DB every time it displayed a book list element, since it will use bookmarked_by?(user_id) to determine if a user bookmarked the book. So instead of just 1 query, it would make n + 1 queries.
Preloading child tables isn't necessarily wise all the time. It really depends on what you intend to do with the data after you fetch it. As the Agile rails book warns, preloading all that data will take time. If you look at your log files, you'll see that it's a significant amount.
If you're only going to load a limited number of these book list elements on a single page at a time, it actually might make sense to forgo preloading of child tables, and just use a find() instead of a select.
class SceneController < ApplicationController
def list
@books = Book.find_books
end
end
class Book < ActiveRecord::Base
def self.find_books
find(:all, :conditions => ["created_on > ?", 6.month.ago],
:limit => 20, :order => "created_on desc")
end
def bookmarked_by?(user)
Bookmark.find(:first,
:conditions => ["book_id = ? and owner_id = ?", id, user.id]) ? true : false
end
endAnd if you're going to display counts of arrays, but all means, use counter caching. It's easy to do (as long as you follow instructions!), for most situations.
Intuitively, if you want to display over a certain n number of book list elements, it makes more sense to use :include and select it. However, I wanted to point out that when you make decisions like this, you'll always want to measure the load times, because you earn what you measure.
Also, use the right number of runs. Too short number of a number of times you run a function, the more variation you'll have in your benchmarks. Let's say that you get two numbers for two different methods.
$ bench -u http://localhost:3000/method1 -r 50 -c 5
50....45....40....35....30....25....20....15....10....5....
Total time: 240.383527755737
Average time: 4.80767055511475
$ bench -u http://localhost:3000/method2 -r 50 -c 5
50....45....40....35....30....25....20....15....10....5....
Total time: 156.147093772888
Average time: 3.12294187545776
So it's obvious that method2 is better right? Well, not necessarily. While benchmarks only show averages, you'll need to pay attention to standard deviations. The bigger the standard deviation, the more runs you'll need to figure out the average load time, and the number of decimal points you can trust. That way, you can figure out whether the difference in load times is statistically significant or not.
That way, you can ascertain whether the optimization you made were worth the trouble or not. tip!
Tuesday, September 04, 2007
Mobtropolis Public Release
I've been working on Mobtropolis for about 10-12 weeks now. It's was finally released last week Tuesday. It's something that makes people expand their world by helping them discover and share local adventures around them. The easiest way to think about it is as a dynamic large-scale photo scavenger hunt or a photo-dare site that helps you expand your world--hopefully for the better.Behavior is hard to change, so it's framed slightly in terms of a game. The basic mechanics should be familiar to those that frequent social news sites. Anyone can submit scenes or vote them up. The higher something's voted, the higher its visibility to others. Anyone can do a scene and take a photo as proof. They can then send it in via their camera phone, or upload it from their digital camera when they get back to desktop. Their friends who voted for a scene will then get an email with the photo attached.
It's been oddly thrilling to get photos of people doing scenes that you submitted.
There's still a lot of work to be done on it. Eventually, I hope to marry the virtual and the real in a tighter loop and better integration with mobile devices. However I'm putting it out in according to startup mantras of "Release early, then iterate like crazy". So check it out, and if you'd be so kind, give me some feedback, good or bad, so I can make it better.
http://www.mobtropolis.com
Enjoy!
Sunday, September 02, 2007
Use barriers to your advantage
I use to read iwillteachyoutoberich, as he had some useful tidbits, and I, like many twenty-somethings, usually worry about personal finances...But anyway, the one piece that I really like that I gleamed from him was his take on barriers.
He had another post where he turned it around and said that you can make barriers work to your advantage as well, not just in avoiding kooks, but in increasing your productivity.
Since I do web dev, the browser's up all the time, and it's really easy just to hit ctrl-t www.facebook.com. And then before you know it, a whole half hour's been wasted. It's even worse with proggit or hacker news. A couple days ago, there was a tip on 4 lines to increase your productivity (can't find it now) on reddit, and it was just lines in a /etc/hosts file. It reminded me of barriers, so I decided to try it out.
I set up my /etc/hosts file:
And lo and behold, it actually worked. Just the extra step of having to type in a command and a password is enough to deter me from not working. I do still hit ctrl-t once in a while, but then I'm reminded that it's fruitless, and I might as well get back to work. My friend Ian closes everything but a max windowed emacs as his productivity trick.
Anyone got any others they fool themselves with to get crackin'?
I think the source of 95%+ of barriers to success is…ourselves. It’s not our lack of resources (money, education, etc). It’s not our competition. It’s usually just what’s in our own heads. Barriers are more than just excuses–they’re the things that make us not get anything done. And not only do we allow them to exist around us, we encourage them. There are active barriers and passive barriers, but the result is still the same: We don’t achieve what we want to.
He had another post where he turned it around and said that you can make barriers work to your advantage as well, not just in avoiding kooks, but in increasing your productivity.
Since I do web dev, the browser's up all the time, and it's really easy just to hit ctrl-t www.facebook.com. And then before you know it, a whole half hour's been wasted. It's even worse with proggit or hacker news. A couple days ago, there was a tip on 4 lines to increase your productivity (can't find it now) on reddit, and it was just lines in a /etc/hosts file. It reminded me of barriers, so I decided to try it out.
I set up my /etc/hosts file:
127.0.0.1 www.facebook.com
127.0.0.1 news.ycombinator.com
127.0.0.1 news.octoparts.com
127.0.0.1 programming.reddit.com
# if I'm really having problems concentrating:
127.0.0.1 www.gmail.com
127.0.0.1 mail.yahoo.com
And lo and behold, it actually worked. Just the extra step of having to type in a command and a password is enough to deter me from not working. I do still hit ctrl-t once in a while, but then I'm reminded that it's fruitless, and I might as well get back to work. My friend Ian closes everything but a max windowed emacs as his productivity trick.
Anyone got any others they fool themselves with to get crackin'?
Saturday, September 01, 2007
Ajax.Ajax.PeriodicalUpdater has a decay option
Prototype JavaScript framework: Ajax.Ajax.PeriodicalUpdater
There's plenty of treasure in API docs, I've usually found--like when you need two submit buttons for an AJAX form. While tutorials are helpful for just getting started, I'm a firm believer in just browsing through API docs and references once in a while, like a lazy grounds keeper checking for garden gnomes. I also like reading dictionaries. I don't do that too often, just when I'm looking up words. I never got any papers done until internet dictionaries came around.
The past two days, I've been playing more with Javascript, and that involved looking more closely at the Prototype library that comes with Rails. So far, my experience with prototype has been pretty good. It's less high level than, say mookit, but I think it was meant to fill holes in the current javascript language. Even little things like Try.these() are nice, due to javascript discrepancies between browsers.
As a result of browsing through the Prototype API, I found that the adaptive polling I had talked about before was actually already in the Prototype library. It was just never mentioned in any of the Rails docs or tutorials about periodically_call_remote().
Though I don't know if it was around when I blogged it last December, that should be lesson to me to stop talking, and just try writing a patch, as Prototype is open source. I probably would have learned a lot.
There's plenty of treasure in API docs, I've usually found--like when you need two submit buttons for an AJAX form. While tutorials are helpful for just getting started, I'm a firm believer in just browsing through API docs and references once in a while, like a lazy grounds keeper checking for garden gnomes. I also like reading dictionaries. I don't do that too often, just when I'm looking up words. I never got any papers done until internet dictionaries came around.
The past two days, I've been playing more with Javascript, and that involved looking more closely at the Prototype library that comes with Rails. So far, my experience with prototype has been pretty good. It's less high level than, say mookit, but I think it was meant to fill holes in the current javascript language. Even little things like Try.these() are nice, due to javascript discrepancies between browsers.
As a result of browsing through the Prototype API, I found that the adaptive polling I had talked about before was actually already in the Prototype library. It was just never mentioned in any of the Rails docs or tutorials about periodically_call_remote().
Though I don't know if it was around when I blogged it last December, that should be lesson to me to stop talking, and just try writing a patch, as Prototype is open source. I probably would have learned a lot.
Friday, August 31, 2007
Javascript scoping and this
Javascript has been a surprising language, mostly because the difference between my experience with it now, and when I touched it in high school. Going along with the theme of seeing everything as the proverbial functional nail, there's been a bunch of things to get use to, such as how scoping work works in javascript.
Javascript is a lexically scoped language (scope determined when function is defined, not when it's executed). This much, I think I get.
However, unlike Java and C++, "instance methods" of Objects doesn't include attributes in its scope. One has to explicitly refer to it. So let's say I have something like the code below. Just something simple, where one wants to process each element in a list. Of course, one can do this with a for loop and it wouldn't be an issue, but since playing with Ruby, I've found that iterators were more clear when writing code. It's probably bad design, but for the sake of argument, let's say that tile's draw() method takes the Renderer object as an argument.
This will not work, because apparently, "this" inside the anonymous function of each() doesn't refer to renderer, but (I think) to the anonymous function. So draw() will usually complain something about how the renderer doesn't have the right properties.
This is a bit odd, since this would work in Ruby. The only way I've found around this is just to throw "this" into a variable, then refer to that variable.
According to javascript's scoping rules, this works. But to me, it looks rather ugly. Anyone know of an easier way to get around it?
Javascript is a lexically scoped language (scope determined when function is defined, not when it's executed). This much, I think I get.
However, unlike Java and C++, "instance methods" of Objects doesn't include attributes in its scope. One has to explicitly refer to it. So let's say I have something like the code below. Just something simple, where one wants to process each element in a list. Of course, one can do this with a for loop and it wouldn't be an issue, but since playing with Ruby, I've found that iterators were more clear when writing code. It's probably bad design, but for the sake of argument, let's say that tile's draw() method takes the Renderer object as an argument.
function Renderer() {
this.tiles = new Array();
}
Renderer.prototype = {
draw: function() {
this.tiles.each(function(tile) {
tile.draw(this);
});
}
};
This will not work, because apparently, "this" inside the anonymous function of each() doesn't refer to renderer, but (I think) to the anonymous function. So draw() will usually complain something about how the renderer doesn't have the right properties.
This is a bit odd, since this would work in Ruby. The only way I've found around this is just to throw "this" into a variable, then refer to that variable.
function Renderer() {
this.tiles = new Array();
}
Renderer.prototype = {
draw: function() {
var my_renderer = this;
this.tiles.each(function(tile) {
tile.draw(my_renderer);
});
}
};
According to javascript's scoping rules, this works. But to me, it looks rather ugly. Anyone know of an easier way to get around it?
Monday, August 27, 2007
Using Firefox 3 as a XUL runtime environment
Using Firefox 3 as a XUL runtime environment
Some other post about it
This tidbit is actually rather exciting. Firefox uses a rendering engine that reads XML to determin how its user interface is laid out. And now that they're going to release it as a runtime environment, it should be possible to create desktop clients like one develops for the web. So instead of an API like Swing, or the like, you have a declarative language for rendering layout for desktop applications.
In addition, it should also be possible to transmit user interfaces from application to application and platform to platform more easily with such a declarative language for layout. I imagine being able to drag and drop interfaces from your desktop to your mobile device as a metaphor for "taking things with you."
I'm sure other people that have been paying attention in this field saw this coming for a while. Adobe's AIR (Adobe Integrate Runtime) is aiming specifically in this field. We'll see how this plays out, but likely, developers will choose one or the other for a specific strength, and each will have their own niche. You can try out Firefox 3's current alpha. It's been much more stable in the last couple of weeks.
Some other post about it
This tidbit is actually rather exciting. Firefox uses a rendering engine that reads XML to determin how its user interface is laid out. And now that they're going to release it as a runtime environment, it should be possible to create desktop clients like one develops for the web. So instead of an API like Swing, or the like, you have a declarative language for rendering layout for desktop applications.
In addition, it should also be possible to transmit user interfaces from application to application and platform to platform more easily with such a declarative language for layout. I imagine being able to drag and drop interfaces from your desktop to your mobile device as a metaphor for "taking things with you."
I'm sure other people that have been paying attention in this field saw this coming for a while. Adobe's AIR (Adobe Integrate Runtime) is aiming specifically in this field. We'll see how this plays out, but likely, developers will choose one or the other for a specific strength, and each will have their own niche. You can try out Firefox 3's current alpha. It's been much more stable in the last couple of weeks.
Sunday, August 26, 2007
Using anonymous functions inside functions
I was reading an article on Javascript and its use of first class functions. I'd quote it if I could find it again, but basically, it extolled the virtues of being able to create anonymous functions, blocks, and closures easily in Javascript. One of the things that it claimed was that creating local functions only used within a method makes the code much more readable. I was skeptical, but as I was refactoring this function, I thought I'd give it a shot even though I was coding in Ruby. Ruby also allows for the creation of first class functions.
It's just a helper function that generates a bookmark button--however, what gets generated depends on whether a user is logged in and whether a user had already bookmarked the page.
Well, I thought it was pretty straightforward to read, though half of it was duplicated code. It wasn't very DRY. So instead of fudging around with another way to structure the control flow, I tried my hand at using blocks. I ended up with this:
As you can see, I put the link_to_remote() code into a block, since the only difference between the two is the label. In addition, it is possible for the block to use variables that are in this method, but outside the block, even if the execution is outside the scope of this function, if the block is ever passed outside of bookmark_button. I know in javascript, this causes memory leaks in IE. I haven't heard that it's a problem in Ruby yet.
But in this case, I'm just creating a function inside a method to make things more readable and cleaner. I think for the most part, it worked pretty well. As long as it's not overdone (notice I didn't create a proc for the labels), it should make for more readable code, and you don't pollute the class namespace with methods that are only used in this one method. I should get use to this. tip!
It's just a helper function that generates a bookmark button--however, what gets generated depends on whether a user is logged in and whether a user had already bookmarked the page.
def bookmark_button(page, user, indicator_id)
if page.bookmarked_by? session[:user]
"Bookmarked!"
elsif page.done_by? session[:user]
link_to_remote image_tag('heart_add.png', :height => 16, :width => 16) + " Bookmark again!",
:url => { :controller => :bookmark_list, :action => :add, :page_id => page.id },
:loading => "Element.toggle('#{indicator_id}')",
:complete => "Element.toggle('#{indicator_id}')"
else
link_to_remote image_tag('heart_add.png', :height => 16, :width => 16) + " Bookmark!",
:url => { :controller => :bookmark_list, :action => :add, :page_id => page.id },
:loading => "Element.toggle('#{indicator_id}')",
:complete => "Element.toggle('#{indicator_id}')"
end
endWell, I thought it was pretty straightforward to read, though half of it was duplicated code. It wasn't very DRY. So instead of fudging around with another way to structure the control flow, I tried my hand at using blocks. I ended up with this:
def bookmark_button(page, user, indicator_id)
bookmark_link = Proc.new { |label|
link_to_remote "#{image_tag('heart_add.png', :height => 16, :width => 16)} #{label}",
:url => { :controller => :bookmark_list,
:action => :add,
:page_id => page.id },
:loading => "Element.toggle('#{indicator_id}')",
:complete => "Element.toggle('#{indicator_id}')"
}
if page.bookmarked_by? session[:user]
"Doing this"
elsif page.done_by? session[:user]
bookmark_link.call("Bookmark again!")
else
bookmark_link.call("Bookmark!")
end
endAs you can see, I put the link_to_remote() code into a block, since the only difference between the two is the label. In addition, it is possible for the block to use variables that are in this method, but outside the block, even if the execution is outside the scope of this function, if the block is ever passed outside of bookmark_button. I know in javascript, this causes memory leaks in IE. I haven't heard that it's a problem in Ruby yet.
But in this case, I'm just creating a function inside a method to make things more readable and cleaner. I think for the most part, it worked pretty well. As long as it's not overdone (notice I didn't create a proc for the labels), it should make for more readable code, and you don't pollute the class namespace with methods that are only used in this one method. I should get use to this. tip!
I'm quitting Eclipse, like people quit crack
I'm finally swearing off Eclipse with my Rails development. When I first started using Eclipse with Java projects, it was admittingly pleasant, once you get use to how to set it all up. Colors everywhere, as well as trees to keep organized, and automatically run ANT.
Then I started using it for Rails, because of its relatively low learning curve with RadRails. And with a Subversion plugin called Subclipse, I was hooked.
Well, the honeymoon has ended, I've finally swore them off. I'm on a computer that only has 512MBs of RAM, which really should be more than enough. One could argue, "Just get more RAM!" However, Eclipse eats up more than 140MB of memory. Add that to Firefox 2's voracious appetite for memory (about 120MB until I changed the settings to get it down to about 80 to 100MB), and my computer often slowed to a crawl. having to wait more than a second for anything interrupts my train of thought. I get bored waiting, and often sidetracked--I get knocked out of my zone.
It's been a long time coming, but I've finally switched to command line SVN. It's just that any tutorials I've found were too long winded. This recent one at the top of google's rankings, isn't bad at all. In addition, the bug in Rinari, an emacs mode for Rails, got fixed. So the two reasons to stay with Eclipse is no longer around.
Even though it takes forever to figure out how to change any setting in emacs (like freaking font size!), I figured it's better for me, as I've gotten a bit comfortable in Rails. I need to keep expanding and learning--so if I have to do anything in emacs in the future, I'll have to learn elisp.
The contrast is amazing. Emacs only takes up 4.3 Megs. Switching between different Desktops doesn't take forever now, watching the screen re-draw in slow motion.
I hear rumors that Firefox 3 has better memory management. Thank goodness. It's no longer the lite browser that just popped up. I liked it better when it was more responsive. Granted, they've crammed more stuff into it since they've started, but it shouldn't keep eating up memory to the point of crashing! I'd be looking forward to that. In the meantime, I'll enjoy all my memory breathing room.
Then I started using it for Rails, because of its relatively low learning curve with RadRails. And with a Subversion plugin called Subclipse, I was hooked.
Well, the honeymoon has ended, I've finally swore them off. I'm on a computer that only has 512MBs of RAM, which really should be more than enough. One could argue, "Just get more RAM!" However, Eclipse eats up more than 140MB of memory. Add that to Firefox 2's voracious appetite for memory (about 120MB until I changed the settings to get it down to about 80 to 100MB), and my computer often slowed to a crawl. having to wait more than a second for anything interrupts my train of thought. I get bored waiting, and often sidetracked--I get knocked out of my zone.
It's been a long time coming, but I've finally switched to command line SVN. It's just that any tutorials I've found were too long winded. This recent one at the top of google's rankings, isn't bad at all. In addition, the bug in Rinari, an emacs mode for Rails, got fixed. So the two reasons to stay with Eclipse is no longer around.
Even though it takes forever to figure out how to change any setting in emacs (like freaking font size!), I figured it's better for me, as I've gotten a bit comfortable in Rails. I need to keep expanding and learning--so if I have to do anything in emacs in the future, I'll have to learn elisp.
The contrast is amazing. Emacs only takes up 4.3 Megs. Switching between different Desktops doesn't take forever now, watching the screen re-draw in slow motion.
I hear rumors that Firefox 3 has better memory management. Thank goodness. It's no longer the lite browser that just popped up. I liked it better when it was more responsive. Granted, they've crammed more stuff into it since they've started, but it shouldn't keep eating up memory to the point of crashing! I'd be looking forward to that. In the meantime, I'll enjoy all my memory breathing room.
Thursday, August 23, 2007
Review of Facebook app building experience and where things might go
I've been missing in action for a while, because during the first week of August, I finally got around to figuring out how to build a facebook app. I'm involved with a non-profit organization that wished to create an alumni database. I had written a prototype for them the year before in a week, but I never put it up. I didn't get the help or traction that I had hoped for, so I delayed working on it until this past summer, when they had another summer conference.
It's a good thing too, because facebook didn't release their full API until around May. The app is a good fit for what it needed to do, since it allows alumni to find each other. I hacked it out in 4 days, which was both good and bad. I had to bypass a lot of the discipline and good habits I cultivated on my own projects in order to whip it up.
The hardest thing about facebook apps is simply figuring out how the whole thing hangs together. When you don't know the thing is put together, filling out the long form to setup your facebook app, becomes difficult as nothing makes sense.
There are two major parts to a facebook app. One is a profile box, which is the draggable box you see in people's profiles. And the second is the canvas page, which is a kind of "home page" for your app, if you will. You application is hosted somewhere else outside of facebook. The API just lets you create an interface in facebook. It's my recommendation that you read this page on architecture in the facebook wiki before getting started on any of the "Getting Started" pages.
As you can see from the ASCII diagrams in the architecture page, your application's canvas page is called by proxy through facebook servers. That means every time someone visits your canvas page through facebook, facebook will pull your application's canvas page and display it. Conversely, the profile box is data that is pushed to the facebook servers. This makes sense in two faces of the same coin. Facebook has a LOT of users, and chances are, you can't scale as facebook has. Having lots of people bang on your server out the door would wreck havoc on you. Secondly, facebook doesn't want its profile load times to depend on 3rd party servers. If it has all the information pushed to it, facebook can quickly serve profile pages without having to rely on the latency of its 3rd party applications.
I won't go through how to make a facebook app. Plenty of places already do that well, especially this one for Ruby. However, to give my review, I will say that I give it 3 out of 5 stars. "A" for effort, but there's more improvement to be had down the line.
While the API is a step in the right direction, coding for it has felt clunky at best. This is mostly due to documentation being a bit all over the place, but especially the lack of a test environment. There is no way to set a production, test, or development environment for the same application. The only way is to simply create another application as a "test", and point it to your development machine. Then in your code, it has to switch between the development API key and the production API key. In Rails, you'd use the RAILS_ENV global variable to determine which environment you were in. As a result, there's quite a bit of setup to do before you can start hacking away.
Facebook also provides FBML, a reduced HTML, specific for facebook elements. Some of them are quite handy, such as a friend finder, and it gives you an immediate look and feel that blend in with the rest of facebook. However, debugging it isn't much fun at all. Since you can't set environments, facebook assumes all apps are production, and therefore will not show errors and stack traces. Therefore, one has to tread softly, or cut and paste back and forth to their tools.
That's right. Tools. Facebook does provide tools to test out your FBML and API calls. But it would be much nicer if it were integrated in a development environment for the application.
All, in all, it's not been too bad of an experience, outside of wrestling with the setup and FBML, but there's a lot of improvement to be had. While many have said that Facebook is a walled garden like AOL was in the 1990's, I currently don't see much of a decentralized social network solution that addresses privacy that is popular amongst developers.
The only thing that is a remotely good candidate is the combination of OpenID with Microformats like XFN. But I think it's going to take a demonstration from another successful startup to get more developers on this bandwagon. OpenID is also not yet ubiquitous as a concept in the minds of developers, let alone the common web user. Once this happens, it will be much much easier for social applications to personalize a user's experience right out of the box.
And this isn't personalization based on where your stats, but personalization based on what your other friends have done in the very same application. The web app won't know anything about where you live, or where your friends live, but if it's a book store, it will know everything about what books you like, and what books your friends like, and be able to draw conclusions to better serve your experience based on that.
I hope that is an indication of the things to come. Apps will know more about a user's habits as related to their function, and yet not know who this user is to protect privacy. This will be especially useful with mobile devices, since we carry them with us. Any mobile application that knows if you come into contact with your friends in real time will have a tremendous utility value in producing information useful in a social context. We still have to wait, however. The mobile industry is still not as open as we'd like for open development on mobile devices as a platform.
As an aside, I imagine that the reams of data (anonymous, of course) would be a great boon to social scientists. We might see a revolution in that field, especially if it's combined with decentralized computing concepts.
It's a good thing too, because facebook didn't release their full API until around May. The app is a good fit for what it needed to do, since it allows alumni to find each other. I hacked it out in 4 days, which was both good and bad. I had to bypass a lot of the discipline and good habits I cultivated on my own projects in order to whip it up.
The hardest thing about facebook apps is simply figuring out how the whole thing hangs together. When you don't know the thing is put together, filling out the long form to setup your facebook app, becomes difficult as nothing makes sense.
There are two major parts to a facebook app. One is a profile box, which is the draggable box you see in people's profiles. And the second is the canvas page, which is a kind of "home page" for your app, if you will. You application is hosted somewhere else outside of facebook. The API just lets you create an interface in facebook. It's my recommendation that you read this page on architecture in the facebook wiki before getting started on any of the "Getting Started" pages.
As you can see from the ASCII diagrams in the architecture page, your application's canvas page is called by proxy through facebook servers. That means every time someone visits your canvas page through facebook, facebook will pull your application's canvas page and display it. Conversely, the profile box is data that is pushed to the facebook servers. This makes sense in two faces of the same coin. Facebook has a LOT of users, and chances are, you can't scale as facebook has. Having lots of people bang on your server out the door would wreck havoc on you. Secondly, facebook doesn't want its profile load times to depend on 3rd party servers. If it has all the information pushed to it, facebook can quickly serve profile pages without having to rely on the latency of its 3rd party applications.
I won't go through how to make a facebook app. Plenty of places already do that well, especially this one for Ruby. However, to give my review, I will say that I give it 3 out of 5 stars. "A" for effort, but there's more improvement to be had down the line.
While the API is a step in the right direction, coding for it has felt clunky at best. This is mostly due to documentation being a bit all over the place, but especially the lack of a test environment. There is no way to set a production, test, or development environment for the same application. The only way is to simply create another application as a "test", and point it to your development machine. Then in your code, it has to switch between the development API key and the production API key. In Rails, you'd use the RAILS_ENV global variable to determine which environment you were in. As a result, there's quite a bit of setup to do before you can start hacking away.
Facebook also provides FBML, a reduced HTML, specific for facebook elements. Some of them are quite handy, such as a friend finder, and it gives you an immediate look and feel that blend in with the rest of facebook. However, debugging it isn't much fun at all. Since you can't set environments, facebook assumes all apps are production, and therefore will not show errors and stack traces. Therefore, one has to tread softly, or cut and paste back and forth to their tools.
That's right. Tools. Facebook does provide tools to test out your FBML and API calls. But it would be much nicer if it were integrated in a development environment for the application.
All, in all, it's not been too bad of an experience, outside of wrestling with the setup and FBML, but there's a lot of improvement to be had. While many have said that Facebook is a walled garden like AOL was in the 1990's, I currently don't see much of a decentralized social network solution that addresses privacy that is popular amongst developers.
The only thing that is a remotely good candidate is the combination of OpenID with Microformats like XFN. But I think it's going to take a demonstration from another successful startup to get more developers on this bandwagon. OpenID is also not yet ubiquitous as a concept in the minds of developers, let alone the common web user. Once this happens, it will be much much easier for social applications to personalize a user's experience right out of the box.
And this isn't personalization based on where your stats, but personalization based on what your other friends have done in the very same application. The web app won't know anything about where you live, or where your friends live, but if it's a book store, it will know everything about what books you like, and what books your friends like, and be able to draw conclusions to better serve your experience based on that.
I hope that is an indication of the things to come. Apps will know more about a user's habits as related to their function, and yet not know who this user is to protect privacy. This will be especially useful with mobile devices, since we carry them with us. Any mobile application that knows if you come into contact with your friends in real time will have a tremendous utility value in producing information useful in a social context. We still have to wait, however. The mobile industry is still not as open as we'd like for open development on mobile devices as a platform.
As an aside, I imagine that the reams of data (anonymous, of course) would be a great boon to social scientists. We might see a revolution in that field, especially if it's combined with decentralized computing concepts.
Wednesday, August 22, 2007
How to find your way along the tracks
I started helping out with the local Chicago Ruby scene. It was a small meetup group, but I was asked to do a small presentation on how to get started in Rails. Radiant CMS apparently gets things up and running quickly, so there is a blog for the group.
I wasn't able to make it to this one, so I recorded an mp3 of the talk as well as slides. (as a side note, I remember Zenter allows for slides and recording...but it wasn't up since their acquisition by Google.) Apparently, it went well. Thanks to Ray for getting on the ball with it. Many hands make light work.
At the next meeting, I'm to take the group on a splunking adventure through the rails source. It'll be a bit of work remembering what I was doing at the time, but it should be ok. I just need to make sure that I'm on the ball with everything that I'm doing.
I wasn't able to make it to this one, so I recorded an mp3 of the talk as well as slides. (as a side note, I remember Zenter allows for slides and recording...but it wasn't up since their acquisition by Google.) Apparently, it went well. Thanks to Ray for getting on the ball with it. Many hands make light work.
At the next meeting, I'm to take the group on a splunking adventure through the rails source. It'll be a bit of work remembering what I was doing at the time, but it should be ok. I just need to make sure that I'm on the ball with everything that I'm doing.
Thursday, August 16, 2007
The new YC.news
This ended up to be a long comment on YC.news, now turned Hacker news. I figured it was worth reposting here on my blog:
When I was working at an engineering job at a research lab, I was told what had to be accomplished. Of course, the degree of freedom, amount of creativity, and problem solving you can do varies from project to project, but in the end, I went to bed happy knowing in the back of my mind that someone, somewhere asked for whatever you're working on--that's why you're getting paid. When you're a startup founder, however, you aren't even sure of that, mostly because of the nature of startups and the markets they decide to persue.
A startup can be successful in an already crowded and proven market--especially if the market sucks (online dating comes to mind). But often times, where startups shine is where others fear to tread, and that's in potential markets, unproven markets, and useless markets (until you prove that it isn't).
But how do you figure that out? And given that you see a potential, how do you find a creative solution to build a business from it?
I don't think anyone can tell the future when it comes to these things. But you can certainly learn to get a good intuition for it. One of the ways to do that is to constantly read broadly about interesting things that are going on in the fringes of any number of industries. By interesting, I mean, things that you didn't know that stretchs your understanding of the world and perhaps your imagination a little bit more. When you start to get your finger on the pulse of possibility, you're able to see blooming solutions where others only see wilting dead-ends.
In addition, when you dig deep enough into any topic, it gets quite interesting. You'll start to find that all subjects are intertwined in one way or another. The way all subjects are compartmentalized in school is just so students don't get confused. But really, all subjects are inter-related. Sociology's studies on coordination and biology's study on social insects actually relates to optimization. Weather phenomenon actually relates to crytography. This inter-relatedness works to your advantage in finding creative solutions to make a business out of your new market, because creative things are usually a combination of old things put together in new ways.
Contrary to popular belief, creativity isn't often completely out of nowhere, just as masterpieces don't just materialize in front of masters. For every masterpiece you see hanging in the galleries, there are hundreds of sketches and throwaway paintings that you don't see the master artist practicing on. By the same token, a creative solution for a startup isn't just a stroke of inspiration from nowhere. It's a culmination of a slow absorbing of interesting tidbits that you've gathered and processed in the back of your mind until you've put the relevant pieces together.
So as far as I can tell, Paul Graham views hackers that are startup founders not as just really really good programmers. He believes what makes these people good startup founders is their innate curiosity in the world around them, and the willingness and drive to keep on learning about it to produce and create solutions--which lead to profit if attacked in a business way. That not only drives their strengths as programmers, but as thinkers and builders that change the world and will make money doing so. Not trying to put words in his mouth, but that's my best summary thus far.
Therefore, if you buy into that, then I think the direction into the new YC news as Hacker news is a good change. It will allow people to keep seeing and learning interesting things so they have a better gut for potential and emerging markets, as well as helping them along in their creativity for novel solutions. That is, at least my take on it.
Of course, we'll see how it actually all plays out, but I for one, am looking forward to the change.
Thursday, August 02, 2007
Parallel array processing with zip()
When I'm working in Ruby, it's a blast, but a lot of the concepts are borrowed from functional style programming, and I'm not sure what things are called. Often times, I end up writing the same boiler plate code over and over again, refactoring it into a function that I add to the standard library on my own, while knowing there's probably a built-in function to do that. It was like that with inject(). Today, I discovered zip().
I run into this particular problem when I want to line up arrays with one another, and perform an operation on the same corresponding element of each array--much like a parallel map function. For example, vector addition and multiplication is like this. When you add arrays, you take the the nth element from each vector and add them together.
Usually, I write something like this:
Which is pretty ugly, as I shamefully admit. However, due to the post I last mentioned about Erlang, as well as this post on Ruby talk, I (finally) made the connection that there's such a function in ruby, and it's (also) called zip().
The solution then would look something like this:
MUCH better. That means something like dot_product is a lot more elegant now.
To be honest, I've read the reference API on Ruby's enumerable, array, and hash libraries before. However, at the time that I read zip(), I was thinking, "I'm not sure when I'll ever need to do that.", and just put it out of my mind. :P
tip!
I run into this particular problem when I want to line up arrays with one another, and perform an operation on the same corresponding element of each array--much like a parallel map function. For example, vector addition and multiplication is like this. When you add arrays, you take the the nth element from each vector and add them together.
Usually, I write something like this:
a,b = [1,2,3], [2,3,4]
class Array
def parallel_map(b)
result = []
each_with_index do |e, i|
result << yield e, b[i]
end
end
end
a.parallel_map(b) { |ea, eb| ea + eb }Which is pretty ugly, as I shamefully admit. However, due to the post I last mentioned about Erlang, as well as this post on Ruby talk, I (finally) made the connection that there's such a function in ruby, and it's (also) called zip().
The solution then would look something like this:
a.zip(b).map { |i, j| i + j }MUCH better. That means something like dot_product is a lot more elegant now.
def dot_prod(a, b)
a.zip(b).map { |i, j| i + j }.inject { |t, e| t += e }
endTo be honest, I've read the reference API on Ruby's enumerable, array, and hash libraries before. However, at the time that I read zip(), I was thinking, "I'm not sure when I'll ever need to do that.", and just put it out of my mind. :P
tip!
Sunday, July 29, 2007
Erlang and Neural Networks article on TrapExit.org
Thanks for the reader that submitted my article to Trapexit.org, the guy that runs it asked me to post the full article on there. It ended up being about 7th on programming.reddit.com for about a day or two. So far, I haven't had any complaints on it, so either people haven't read it, or it's been pretty accurate. :)
I didn't think writing that was going to be such a big job when I started, but I suppose I took the position that anyone reading it had minimal mathematical and Erlang background. Therefore, there was a lot of explain. I can appreciate why good textbooks are hard to come by now. :P
But I did learn a lot about Erlang and functional style programming. In addition, Neural networks aren't really mystifyingly magical, like I think many people think of them. I use to think you can just solve anything with them. And while they solve a particular class of problems quite well, they're essentially just a high dimensional gradient descent.
I'll probably work on the neural network code later on--as I haven't written a trainer for it. And I'll probably try to do a particle swarm optimization article in Erlang some other time. In the meantime, I have other things to experiment with and work on. You'll hear about it here first!
I didn't think writing that was going to be such a big job when I started, but I suppose I took the position that anyone reading it had minimal mathematical and Erlang background. Therefore, there was a lot of explain. I can appreciate why good textbooks are hard to come by now. :P
But I did learn a lot about Erlang and functional style programming. In addition, Neural networks aren't really mystifyingly magical, like I think many people think of them. I use to think you can just solve anything with them. And while they solve a particular class of problems quite well, they're essentially just a high dimensional gradient descent.
I'll probably work on the neural network code later on--as I haven't written a trainer for it. And I'll probably try to do a particle swarm optimization article in Erlang some other time. In the meantime, I have other things to experiment with and work on. You'll hear about it here first!
Saturday, July 28, 2007
Self-referential many-to-many join with extra data and non-standard foreign keys
Here's another gotcha that I ran into that's kind of an edge case in Rails. So I'm documenting it, so that the rest of you can go about your business of building great apps, instead of scratching your heads. So what was I doing? I wanted a self-referential join table with rich associations for an "acts_as_friendable" plugin I was writing.
I made table in database as follows:
In the model, put:
Then in the table that you want to act as friendable, you put:
And that's it. you can now access friends() and friendships() of a person.
However, it doesn't work when you try to push the association through, like:
It will fail complain about how "user_id" doesn't exist in any of the tables. I was scratching my head for a good 2 hours before I figured that it wasn't actually me this time. I think the << method doesn't correctly use the foreign keys correctly when they're non-standard like this. According to #6466 ([PATCH] fix for has_many :through push and delete with legacy/non-standard foreign_keys), this doesn't work, until a patch is written for it. I'm not as familiar with the Rails code base as I should be. This is probably a good chance to get started looking at it...unless someone else fixes it first...
As a workaround, I simply used the Friendship ActiveRecord object directly to create the association. I'll have to override the method in the plugin so that it allows correct use of << for my specific case. Tip!
I made table in database as follows:
create_table "friendships", :force => true do |t|
t.column "owner_id", :integer, :null => false
t.column "friend_id", :integer, :null => false
endIn the model, put:
class Friendship < ActiveRecord::Base
belongs_to :owner, :class_name => "User", :foreign_key => :owner_id
belongs_to :friend, :class_name => "User", :foreign_key => :friend_id
endThen in the table that you want to act as friendable, you put:
class User < ActiveRecord::Base
acts_as_friendable
endAnd that's it. you can now access friends() and friendships() of a person.
However, it doesn't work when you try to push the association through, like:
user = User.find(1)
friend = User.find(2)
user.friends << friend
It will fail complain about how "user_id" doesn't exist in any of the tables. I was scratching my head for a good 2 hours before I figured that it wasn't actually me this time. I think the << method doesn't correctly use the foreign keys correctly when they're non-standard like this. According to #6466 ([PATCH] fix for has_many :through push and delete with legacy/non-standard foreign_keys), this doesn't work, until a patch is written for it. I'm not as familiar with the Rails code base as I should be. This is probably a good chance to get started looking at it...unless someone else fixes it first...
As a workaround, I simply used the Friendship ActiveRecord object directly to create the association. I'll have to override the method in the plugin so that it allows correct use of << for my specific case. Tip!
Subscribe to:
Posts (Atom)