The 3 body problem in Erlang

The three body problem is a simulation of the paths objects with mass in space would take, if all three had gravity effects on each other. With two bodies, the problem can be solved analytically, as done by Kepler. But with three bodies, the paths are chaotic. If you just hit play on the last link, and watch for a minute, you'll see what I mean. And that's the easy version of the problem, since the two suns are fixed. If they were three bodies of comparable masses, then it'd be even harder.

From Ezra: http://ezrakilty.net/research/2006/02/3body_problem_in_erlang.html

The first conceptual problem I hit was the question of synchronization. In order for the sim to be a decent finite approximation to the continuous world of physics, we need to break it into discrete time steps, each of which depends on the previous time step (at least, that's the only way I know of to get a fair approximation). This means that each particle can't just crunch away at it's own speed, working as fast as it can to calculate its position at various times. Easily the particles could grow out of sync with one another, making an inaccurate physical model.

I hadn't thought about this, but I think Ezra is right. In terms of simulation of the 3 body problem, if the correct calculation in the future depends on current calculations, and the current calculations depend on each other, you need to make sure that the calculations are 'in step'.

This calls into question my thought before that asynchronous simulations would work, since whenever the messages arrive, that's when they arrive and process them. In a decentralized simulation of termites gathering wood chips, I imagine an asynchronous simulation would suffice. It doesn't really matter what exact paths the termites take, but rather, the end result of that chaos. But in a gravity simulation, asynchronous simulation doesn't seem to work, because what you're interested in is the actual paths.

If the calculations of all other threads must be synchronous or in lockstep, it would seem like it would give an upper bound to how fast the simulation can go, even in a multi-threaded environment. Since the calculations will be wrong, the further into the future you calculate with slightly incorrect values, what kind of useful computations can you do if you don't have all the initial conditions in your formula?

The only thing I can think of is if you had different sets of three threads--one for each mass--processing the simulation at different simulation times, you can reduce the processing load for the trailing set of threads. So say you had a leading set of threads that operated on simulation time of t + n always. That leading set can narrow the scope of possible answers. Since it knows it's operating on a chaotic system, it knows that what the error is given a certain lead time of n. Therefore, it should be able to limit the upper and lower bound of the possible right answers. Then, the trailing set of threads that operate on simulation time of t, only has to adjust the error, which hopefully is less computationally intensive.

Google Gears Lets Developers Take Apps Offline

Google Gears Lets Developers Take Apps Offline

This is certainly newsworthy. Google announced Gears, which is something that you install on your desktop to be able to operate online applications offline. I remember about 3 to 5 years ago when Google said, no, we're not interested in desktop, because it's not what we're good at. We're doing search.

If anything I think they learned from Netscape's mistake in the past. Marc Andersen, the founder of Netscape, announced that, as a startup, they were taking on Microsoft, and was going to beat it to the ground. Of course, when you use strong words like that, you're going to get Bill Gate's attention, and it's always dangerous to wake a sleeping dragon, when you're not bigger yourself.

Despite the ever growing ubiquity of wireless connections and connectivity all around, I think there's still a place for offline applications. This sort of thing to me, isn't really about being able to do your work on planes, though it's certainly useful for that. To me, this is about caching results that the user might possibly want to see/do next, so that the user experience is fast and responsive without possible network latency. While AJAX is fast, and tolerable for most things, I imagine that there will be some applications that can make good use of this type of offline caching mechanism, so that what was impossible before is now possible.

Of course, caching is irrelevant when the bandwidth is high, but you will either find yourself 1) in places where bandwidth is lower or 2) the bandwidth requirement for your dataset is higher than what you currently have. Mapping applications come to mind as benefiting a lot from caching mechanisms. And if bandwidth jumps up, that makes caching in mapping applications obsolete, there will be other datasets that will be too large to stream in the future. I can only imagine classifiers or their training data sets being one example, as well as a record of the user's digital life.

Update: I didn't mention this, but I think it makes even more sense for mobile devices, per this opengardens post on it.

Internet retailer connects with TextPayMe - Puget Sound Business Journal (Seattle):

Internet retailer connects with TextPayMe - Puget Sound Business Journal (Seattle):

This is a bit of interesting news to me. Originally, I was wondering where textpayme was going to get its traction. Amazon apparently is the answer. While mobile payments is an idea that's not new, and is currently implemented in Japan and other Asian countries, it still doesn't have much traction in the states.

I'm a bit amazed at what's possible through the limited common cell phone interface. Mobile phones are gaining in power, and lots of people anticipate that something's around the corner, but no one knows on which platform, and what's going to be built on top of it. Others are more dismissive, saying that mobile phones are underpowered devices that have limited interface, and therefore not exciting to develop on, compared to the web and desktop.

I think they forget that many said the same about web applications when they first started in the 90's. Web applications back then were clunky at best, and didn't boast a responsiveness until two years ago, when google maps came out.

In addition, mobile devices aren't just small desktops. They have characteristics that are unique to the platform. They are always on, considered personal devices, always on a person, always connected, has sensors on it, and knows its location. While mobile video and music currently offered by the carriers is nice, I don't think it quite hits the spot yet. The mobile application that plays to the platform's strengths will be the one that hits it.

Google maps street view released~

2140 Taylor St, San Francisco, CA 94133 - Google Maps

Google just released street views in SF. You can see the street corners of the city as if you were actually there. That's kinda amazing. At this point, they have enough man power to go and scour a city I suppose...or they bought this information from another company (or the whole company itself) that was doing this. I like how the street arrows tell you which way is north when you're panning around.

I think the potential for this is in augmented reality for mobile devices. I would not be surprised if Google either released a mobile phone or a mobile phone application that allowed you to do the panning in real-time, all the while telling you which way is north, as well as where the nearest gas-station/food/store is.

What I think they'll miss is the potential for social information such as, where your friends are all gathering. I'd chalk it up to Loopt or Facebook to see the potential for that.

Upgrading backgroundrb

When upgrading to backgroundrb 0.2.x, make sure you delete the old ./script/backgroundrb directory. Also make sure that backgroundrb.yml is renamed/deleted. After installing, run "rake backgroundrb:setup" to generate the appropriate setup files.

If you get something like
"LoadError: no such file to load -- slave"

Make sure you have the gems, slave 1.1.0+ and daemon 1.0.2+ installed.

Then, according to the mailing list, if you get:
"ERROR: there is already one or more instance(s) of the program running"

Make sure you delete the old log/backgroundrb.pid . Thing should work after this.

Exploring: reCAPTCHA: A new way to fight spam

Exploring: reCAPTCHA: A new way to fight spam

This particular piece of news has been floating around lately. It's a CAPTCHA service that also uses the CAPTCHA information entered by users to teach computers how to digitize books.

It's so freakin' obvious, I slapped myself on the forehead. I even advocated and watched Luis Von Ahn's videos on human computation, and didn't think about it. Anyway, it seems a little bit odd, though, using a technique that computers can't solve to teach computers how to read--hence solve CAPTCHAs. Not knowing enough details--I wonder if the success of reCAPTCHA will call for the demise of the CAPTCHA.

The usual concerns of cheating were rampant on reddit comments. "What if people just put in random stuff? Then you'll have a computer that spew out crap when digitizing books." If his lecture on the ESP game was any indication, he has a number of ways to fight it (not to mention he specializes in online cheating also). In the ESP game, he counteracts cheating by giving the player a couple ones he knows the answers to and sees how much they're off. Also, he keeps track of the statistics for each image as well as throwing away results randomly. It's a little hard to see how he'll track individual users--other than through their IP--but otherwise, one can feasibly use the same methods for reCAPTCHA.

Naive Bayesian Classifier Hashed and Slashed

I know, I've been bad at posting over the course of the week, where readers would be more likely to be reading this stuff at work. But I have been busy myself too...mostly thinking about non-technical stuff.

Well, some weeks ago, I spent some time looking at Bayseian Probabilities. I learned it way back in high school, though I never gave it much thought. Then I stumbled on it again in college in my ECE courses, and nearly failed the class. And then I took it again in grad school, and though I did well enough in the class, I still felt weak in probability.

This time, when implementing a bayesian classfifier, I learned in and outs of a seemingly simple naive bayesian classifier, and I learned how to spell 'bayesian'. That 'e' after the 'y' gets me every time.

Anyway, I was looking at Paul Graham's famous a plan for spam, and I couldn't figure out where he got that last formula. Kinda mad at myself for not seeing it earlier, cuz, it really is pretty simple. Anyway, it took me a while, but I worked it out. Turns out I learned more when I implemented it...or rather, I would have learned it had I concentrated the first two times.

We know Bayes Theorem is as follows:

(1) P(a,b) = P(a|b) * P(b) = P(b|a) * P(a)

With some algebra of the above we can also derive

(2) P(a|b) = P(b|a) * P(a) / P(b)

But also note that if we take (1), and put a given 'd' behind it, it'd hold true if the probabily on the other side also had a given 'd' behind it. If you draw out the Venn Diagrams, you'll see this is true.

(3) P(a,b|d) = P(a|b,d) * P(b|d) = P(b|a,d) * P(a|d)

We also have the Total Probability Rule, which says that the total probability is made up of its parts. If you apply bayes rule, in (1), you'll see that it's true.

(4) P(b) = P(b|a) * P(a) + P(b|a') * P(a')

So this means that Baye's rule in (2) can be rewritten with (4) as:

(5) P(a|b) = P(b|a) * P(a) / (P(b|a) * P(a) + P(b|a') * P(a'))

We also need the Probability Chain Rule. It says that the joint probability of a, b, and c can be rewritten as the following due to equation (1), applied over and over again:

(6) P(a,b,c) = P(a,b|c) * P(c) = P(a|b) * P(b|c) * P(c)

And lastly, the Independence Rule, which makes the bayesian classifier naive:

(7) P(a,b) = P(a|b) * P(b) => P(a) * P(b) iff "a" indp. from "b"

Now, we can solve for what's the probability of spam given these joint probability of words, where each word is considered an orthogonal and independent dimension?

P(s|f0, f1) = P(f0, f1|s) * P(s) / 
(1)                P(f0, f1)
            = P(f0, f1|s) * P(s) / 
(4)           (P(f0, f1|s) * P(s) + P(f0, f1|s') * P(s'))
            = P(f0|f1,s) * P(f1|s) * P(s) / 
(6)           (P(f0|f1,s) * P(f1|s) * P(s) + P(f0|f1,s') * P(f1|s') * P(s')
            = P(f0|s) * P(f1|s) * P(s) / 
(6)           (P(f0|s) * P(f1|s) * P(s) + P(f0|s') * P(f1|s') * P(s')
           ~= P(f0|s)*..*P(fn|s) * P(s) / 
(7)           (P(f0|s)*..*P(fn|s) * P(s) + P(f0|s')*..*P(fn|s') * P(s'))
           ~= P(f0|s)*..*P(fn|s) / 
              (P(f0|s)*..*P(fn|s) + P(f0|s')*..*P(fn|s'))

The last step needs a little explaining. all the P(s) and P(s') drop out of the equation when we're doing a classifier, since for any piece of evidence, f0...fn, the P(s), the probability of spam occurring, is alway the same across any classification. Since P(s) is constant, and P(s') is (1 - P(s)), it is also constant. Therefore, when we are comparing the values to determine if it belong in the spam or ham category, we can get rid of constants.

The actual hard part about bayesian classifiers is how to estimate the underlying probability distribution. If you've never seen a piece of evidence in training, you're going to say the probability of it occurring is zero, which isn't correct if the evidence shows up during classification. There's various techniques for dealing with this, mostly under the term 'smoothing'. I won't describe the various techniques here, but that should be enough to get you started.

Profanity in code

fuck - Google Code Search

Was just procrastinating and found that it's pretty funny to search for profanities in google code search and see what you come up with.

Hackszine.com: Detecting and reducing power consumption in Linux

Hackszine.com: Detecting and reducing power consumption in Linux

Power consumption was usually something of secondary concern for desktop computer engineers for a while. But not so today. When you're cramming all those transistors in such a small space, operating at high speeds, power definitely becomes an issue. Now that mobile and embedded devices are experiencing their slow infiltration of our daily lives, power should be on the table for improvement. Google has been doing some work for this, and they now build their own power supplies, and even build their data centers where power is cheaper (old news, so I won't link it).

Battery life is one area of computing that really is way behind. I'm still hoping for some feat of chemical engineering to save us...but hopefully, in this time of scarce energy for mobile devices will drive some creative hardware engineering.

I have hope for Facebook being the new Google

Facebook just released facebook marketplace, where its members can sell things, like housing, jobs, or textbooks. Strategically, this makes a lot of sense, since it's something that's actually useful to its members, especially if it ties your social network information into what you want to buy and sell. From the looks of it though, it doesn't do that. But I'm sure someone at Facebook is thinking about it.

Facebook is social networking done right--at least better than any competitors that I've seen. On the surface they might all seem the same; there's a personal profile page, there's a list of friends, and you can send messages back and forth with each other. However, I think there's some critical differences.

MySpace has a larger user base, but it is largely seen by its owners as a platform for media advertising. It's an unsupported assertion, but given its mishmash feature set and large ads, it's hard to think otherwise.

Friendster was the leader for quite some time, but has since lost the attention of the under 25 demographic (anecdotal evidence). Their mistake was adding things that were technically neat, but ultimately made the site too slow to use. It's a lot better now, and people are still using it. But based on the features they've put out it seems like they are interested in helping people publishing media to a user's personal network--using blogs, videos, etc. However, no news trickles of them attracting otaku developers, and I'm sure firing the now founder of Renkoo didn't help win over the hearts and minds of otaku developers.

On the other hand, Facebook is seen by its owners as a platform for technology driven innovation to help keep up social interactions between individuals. I'm not sure when the transition happened, but it was more evident to me after news feeds were released. Now, most people were vehemently opposed to it, but I saw it as two things.

First, it was a feedback mechanism to open up sharing. The more you share about yourself to your friends, the more you appear on their radar, and the more interaction you'll interact/message them. This seems to be inline with the goal of keeping people talking with each other.

Secondly, it was the basis of publishing personal news without even having to push a button. We all gather news about the world, but beyond CNN, there's also another type of news we're interested in--information about what our trusted friends are doing. Blogs lets you publish just by pushing a button. Facebook Mini-feeds lets you publish just by doing what you normally do on Facebook. It's not inconceivable that in the future, you can also publish from your mobile that you have free time to chill out, and people can just join you to hang out because they saw that you were available in their mini-feed on their mobiles.

Facebook is pulling ahead in terms of their feature offerings because they seem to be able to attract developers that are the otaku of programmers that are willing to innovate something that's actually useful to their users. Which other social network puts programming puzzles in their mini-feeds? Which other social network has an API? The alacrity in which they deploy features is stunning as well. They implemented twitter pretty easily by listing their status updates. It is in this way that I see them being a 'new Google'--they are setting themselves up as a hacker's paradise and attracting otaku programmers that way.

When Zuckerberg held out against getting brought out, he was either being greedy or he had future plans on what he would be able to do with a social network. Most of the press criticized him for being the former, but it's looking like it's the latter. As long as Facebook is useful for their users, there's value in the social network data that can be used by future applications. If they can establish themselves as the standard platform from which all social information about an individual is gathered through their API, this world would be a changed place, just as Google changed the world with its technology.

Collateral damage caused by incidental limitations

Arto Bendiken | The Road to Enlightenment Is Littered with Irritating, Superfluous Parentheses:

"Python truly sold me on the benefits of dynamically-typed languages and rapid prototyping. I began to see that many of the sacred GoF design patterns were not, in actuality, grand universal truths of software engineering, but simply collateral damage caused by incidental limitations in the abstractive power and object model of certain manifestly-typed programming languages."

This is pretty much the way I feel about it too. I had spent a good year of someone else's money learning UML and design patterns, and it ends up that the only pattern that is remotely useful with dynamic languages is the observer pattern. All others have fallen away because the problem they solved were no longer problems in dynamically typed languages.

That said, I think that the majority of us come from imperative backgrounds of C++ and Java, and it's probably no way to judge static-typeness. Modern static-typed languages such as Haskell and OCaml probably has more tricks up their sleeves.

Erlectricity: Hi Ruby, I'm Erlang.

Educate. Liberate. - Erlectricity: Hi Ruby, I'm Erlang.People are thinking along the same lines I am, though I have nowhere near the guts at the moment to write a bridger...unless this one sucks.

I think lots of people are excited about the language, though I think that those that are just finding Ruby probably won't look at or touch it for a while longer. "Why do I have to learn yet ANOTHER programming language?" They ask.

In any case, due to Erlang's structure around the Actor Model, it makes algorithms like particle swarm optimization and ant optimization seem pretty exciting. More to come later. In the meantime, enjoy the bridge.

Ruby Snippet: Caching object attributes

Here's another little snippet. I'm not sure if I'm extracting out boiler plate code prematurely, but it seems right.

A good general rule of thumb for refactoring is that one should always call an object for its value, rather than storing object values in temporary variables when you're using the object. Generally, you can get away with setting the value of an object to a temporary variable, and then using that temporary variables for subsequent calculations

# c = Collection.new((0..500).to_a)
collection_sum = c.sum
collection_sum * 3 + 2 # some calculation with the sum
if collection_sum == 4 # some comparison with the sum.
  # do something else
end

You're essentially caching the value outside of the object, 'c'. This is good if 'sum' is an expensive call. However, if the code segment gets large, this method tends to get confusing, especially if you stored it in temporary variables at different times, etc. (Also a good rule of thumb is that your scope should never be bigger than about 10-20 lines) It's better to refer to the value directly from the object.

# c = Collection.new((0..500).to_a)
c.sum * 3 + 2 # some calculation with the sum
if c.sum == 4 # some comparison with the sum.
  # do something else
end

This is not applicable to functional programming languages, since all variables are consts within a scope. But for object-orientated imperative languages, this generally makes for less messy code.

However, performance is a problem, if you have to calculate the sum each time you needed it. An easy solution would be to cache the value. An example would be summing the values across some collection. Normally, that wouldn't be a problem, except if you had hundreds thousand entries. If nothing was added to the collection, the sum would stay the same. If something was added, mark the collection to need updating, and then the next time sum() is called, update the value.

You'd need both a variable to keep track of whether the attribute needed updating, and another variable to hold the result. If you only had one attribute, that'd be ok. If you have more than one, it starts to get a little bit messy. This is where meta programming comes in. I wrote a piece of code that did the bookkeeping of caching attributes for you. You'd use it like:

class Collection
  include AttributeCache

  def initialize
    @array = []
    cache :sum, :initial => 0
  end

  def add(x)
    outdate_sum
    @array << x
  end

  def sum
    cached_sum { @array.inject {|t, e| t += e} }
  end
  
end

The code is simple. In the initialization method, there is a call to cache an attribute, sum. And then in the other methods, you'd mark where the sum would be outdated, and in the actual call to sum, you'd specify what to do to update the sum.

module AttributeCache

  def metaclass; class << self; self; end; end
  
  def cache(attr_name, options)
    instance_variable_set "@#{attr_name}", options[:initial]
    instance_variable_set "@#{:sum}_outdated", true

    metaclass.instance_eval do 
      define_method("outdate_#{attr_name}") do
        instance_variable_set "@#{attr_name}_outdated", true
      end
    end
    
    metaclass.class_eval %Q{
      def cached_#{attr_name}(&block)
        if @#{attr_name}_outdated
          @#{attr_name}_outdated = false
          @#{attr_name} = block.call
        end
        return @#{attr_name}
      end
    }

  end
  
end

The biggest hurdle was to figure out how to define a method that accepted a block with meta programming. The best I came up with was to use class_eval. If you have better suggestions, let me know. tip!

Code quickie: instance_variable_set name in ruby

Normally, you can just set an instance variable to initialize it...but if you're doing some meta programming in Ruby, then you might end up using, eval() or instance_eval(). Or you use instance_variable_set(). But notice! You need the "@" symbol in the first argument:

class SomeClass
  def initialize(var_name)
    instance_variable_set "@#{var_name}", 0
  end
end

Well, it doesn't quite make sense to me, since instance variables implies variables of the instance...So you wouldn't think the call to instance_variable_set requires the "@" symbol in the first argument. Tip!

Defaults and Guards

I was watching the javascript lecture, and it mentioned guards and defaults in javascript, and I realized that it was the same in Ruby, and was something that I've been looking for. I had already known about defaults. If you were coming from a C or Java background, you probably would write it in ruby as:

if !a.nil?
  b = a
else
  b = 0
end

If you were more versed in other more dynamically typed languages, you'd rather write it in ruby as:

b = a || 0

It seems a bit obtuse, but I think this structure is invoked so often in my experience, that the shorthand is actually quite welcome. And it's not too bad once you know what it is.

In Erlang (and I think Haskell), there are guards for a function. Guards are basically a condition that must exist before the function is run. If you were coming from a C or Java background, you'd probabily write it in ruby as:

if !a.nil?
  b = a.some_method()
end

But really, it can be written as:

b = a && a.some_method

or, like

b = a.some_method unless a.nil?

I think the latter is more readable, but there would be no default if 'a' didn't exist. With the bitwise operators, you can chain it:

b = (a.respond_to?(:gsub) && a.gsub(/h/, '')) || "default!"

Now, that's starting to be hard to read. It has a guard for a, so that it doesn't raise an error when it tries to run gsub() if a is nil, and if it is nil, it will return the default. With this kind of thing, you'd want to be judicious and careful when using it. It's definitely easy to go crazy with this sort of thing. Remember, the goal is ease of readability, cuz you only ever write a line once, but you read it lots of times.

Erlang and Neural Networks Part III

I had meant to do more on Erlang more quickly, but I got sidetracked by meta-programming. Here's Part III of Erlang and Neural Networks!

Last time, I did Erlang and Neural Networks Part II. And we saw that neural network is basically made up of interconnected perceptrons (or neurons), and they are basically modeled as a linear combination of inputs and weights with a non-linear function that modifies the output.

Drawing a line in the sand

Classifiers often do very well strictly on probabilities. But often times, we don't know what the underlying probabilities are for the data, and not only that, we don't have lots of training data to build accurate probability densities. One way around that is to draw a line in the data space that acts as the decision boundary between two classes. That way, you only have to find the parameters (i.e. weights) of the line, which is often fewer in number than the entire probability space.

This is exactly what a perceptron does. It creates a decision boundary in data space. If the data space is a plane (2D, or having two inputs), then it draws a line. For higher data space dimensions (4D or more), it draws a hyperplane.

So Why Not Go Linear?

The problem with just using a perceptron is that it can only classify data that is linearly separable--meaning data you can separate with a line. The XOR problem is a simple illustration of how you can't draw a line that separates between on and off in an XOR. Minsky and Papert wrote a famous paper that kinda killed off research in this field for about a decade because they pointed this out.

So to get around this linearity, smart people eventually figured out that they can chain perceptrons together in layers, and that gives them the ability to express ANY non-linear function, given an adequate number of hidden layers.

Shake my hand and link up to form Voltron

Let's try linking our perceptrons together. We're going to add two more messages to our perceptrons:

perceptron(Weights, Inputs, Output_PIDs) ->
 receive
   % The other messages from part II

   {connect_to_output, Receiver_PID} ->
     Combined_output = [Receiver_PID | Output_PIDs],
     io:format("~w output connected to ~w: ~w~n", [self(), Receiver_PID, Combined_output]),
     perceptron(Weights, Inputs, Combined_output);
   {connect_to_input, Sender_PID} ->
     Combined_input = [{Sender_PID, 0.5} | Inputs],
     io:format("~w inputs connected to ~w: ~w~n", [self(), Sender_PID, Combined_input]),
     perceptron([0.5 | Weights], Combined_input, Output_PIDs)
 end.

connect(Sender_PID,  Receiver_PID) ->
 Sender_PID ! {connect_to_output, Receiver_PID},
 Receiver_PID ! {connect_to_input, Sender_PID}.

We would never call connect_to_output() or connect_to_input() directory [1]. We'd just use connect(). It basically just adds the perceptron's process ID to each other, so they know who to send messages to when they have an output.

We can now connect up our perceptrons, but with the way it is, currently, we'd have to send a separate message to each perceptron connected to an input to the network. This is tedious. We are programmers and we are lazy. Let's make a perceptron also double as an source node. As source node simply passes its input to to its outputs.

perceptron(Weights, Inputs, Output_PIDs) ->
 receive
   % previous messages above and in part II

   {pass, Input_value} ->
     lists:foreach(fun(Output_PID) ->
                     io:format("Stimulating ~w with ~w~n", [Output_PID, Input_value]),
                     Output_PID ! {stimulate, {self(), Input_value}}
                   end,
                   Output_PIDs);
 end.

Now we can start creating perceptrons.

64> N1_pid = spawn(ann, perceptron, [[],[],[]]).
<0.325.0>
65> N2_pid = spawn(ann, perceptron, [[],[],[]]).
<0.327.0>
66> N3_pid = spawn(ann, perceptron, [[],[],[]]).
<0.329.0>

Note that we get back three process IDs of the three perceptrons we created. Then we start connecting them.

67> ann:connect(N1_pid, N2_pid).
<0.325.0> output connected to <0.327.0>: [<0.327.0>]
<0.327.0> inputs connected to <0.325.0>: [{<0.325.0>,0.500000}]
{connect_to_input,<0.325.0>}
68> ann:connect(N1_pid, N3_pid).
<0.325.0> output connected to <0.329.0>: [<0.329.0>,<0.327.0>]
<0.329.0> inputs connected to <0.325.0>: [{<0.325.0>,0.500000}]
{connect_to_input,<0.325.0>}

We used N1 as an input node connected to perceptrons 2 and 3. So if N1 is passed a value, N2 and N3 should be stimulated with that value.

69> N1_pid ! {pass, 0.5}.
Stimulating <0.329.0> with 0.500000
{pass,0.500000}Stimulating <0.327.0> with 0.500000

<0.329.0> outputs: 0.562177

<0.327.0> outputs: 0.562177

Hurray! So now, the network's got tentacles, that we can connect all over the place, writhing, and wiggling with all its glee. However, this is currently a DUMB network. It can't classify anything because we haven't told it how to learn anything yet. How does it learn to classify things? It does so by adjusting the weights of the inputs of each perceptron in the network. And this, is the crux of neural networks in all its glory. But you'll have to wait til next time!

(1) Note that the last message connect_to_input() isn't followed by a semicolon. That means every message before it in perceptron needs to end with one. So if you've been following along, the stimulate() message from part II needs a semicolon at the end of it now.

Erlang and Neural Networks Part I
Erlang and Neural Networks Part II
Erlang and Neural Networks Part III

Innovation is force fed; someone get the lube!

In an earlier post, I had talked about what users know and what you know, when it comes to listening to your users. That said, when it comes to building new products, either in another line, or something to replace your old product, you should go back to not listening to your users--at least on the first draft. The act of creation is effectively the effort of one (or the few). At least when it comes to first drafts, too many cooks do spoil the broth. That might be a bit Ayn Randian, but the only thing I've ever heard of where design by committee was successful was the Space Shuttle and the Lunar Lander. (If there's more examples, please enlighten me.)

When you're building a product you're essentially forcing your world view onto others. You're basically saying, "I find this to be a pain. And this is not the world as it should be. As a builder, I can correct it after mouthing off for a while." And this is usually why people don't warm up to innovative ideas readily--someone is shoving their world view in your face. And unless you're someone that has been looking for a solution to the same problem when it's introduced to you, you won't be receptive to it. Even innovative people suffer from this affliction of shortsightedness.

“Don’t worry about people stealing an idea. If it’s original, you will have to ram it down their throats.” – Howard Aiken

Because innovative products can be so jarring, they should soften the blow a bit--or as others like to call it lowering the barriers. This is where influences from design, gaming, and etiquette can help.

Beyond the current trend of sleek lines, horn-rimmed glasses and black turtle necks of designers, design isn't just about putting a gradient background on your web app, or painting things in pastel colors. Hackers making a product should understand that design is the study of how to best solve communication and usability problems with limiting constraints. What information would the user need to know right this second, and how should you convey it to make it as easy to understand as possible? And from the answers to those questions will emerge a form that is also pleasing to the eye.

Gaming is an avenue more familiar to hackers than design is. However, games are often seen as mere trifles of play reserved for kids--though this is changing. If you've played enough video games and thought about WHY they're fun, will help also, because to bring out the essence of fun in what's normally perceived as tedium will give your product an edge. In the lecture about the ESP game by Luis von Ahn, he laments the fact that there's millions of cycles of human computation wasted. There was 9 billion hours played of solitaire last year (est.). Considering that the Empire State Building took 7 million hours and Panama Canal took 10 million hours, that's a lot of wasted hours. We should be able to put those cycles to good use by making people play games to solve problems that computers can't yet solve. So a symbiosis of humans and computers can be considered a large distributed computer to solve hard problems, such as object recognition in images. You might have played it.

In other web apps, the idea of a collection is a powerful mechanism of play. Social networking sites play on the idea of collecting friends, much in the same way that in Pokemon, you "gotta catch them all!". In others, the idea of a scoreboard is a powerful motivator, as seen on Digg and Reddit.

And last of all, the idea of etiquette seems far removed from being applicable to innovative products. However, no matter how much technology people surround themselves with, we are still social beings and will have social tendencies. Because of that, we expect certain behaviors and interactions between ourselves and our machines. We get mad and frustrated at computers and devices because they're usually not very polite. They stop responding when they're busy doing something, but don't tell you what they're doing. They don't remember what you told them last time and asks us over and over again. And when they don't know how to ask for help when something goes wrong, since the error messages are unintelligible to most users. These are all hallmarks of an annoying person, and were it a real person, I'd have kick them to the curb.

The iPod, and in general, Apple products, are known for their politeness. When I first got an iPod, it was the 5th generation. I was surprised that it stopped the music, if the ear buds got unplugged, and that it turned itself off, after it's been paused for a while. Basically, it knew what was going on, and reacted to it in a fashion that makes sense to its owner. That sounds like the promise of Agent based software hyped so long ago. Maybe it should make a slow come-back.

The sad thing is, computer apps and devices have been annoying us for so long, that we have kinda gotten use to it. I think as research on classifiers become more readily available to programmers as being embedded in the language, and the rising influence of designers in applications, we should see a trend towards more polite products. If you can make a product that is polite, it'll go a long way in gathering fans.

In the end, you want people to use what you build if it has value. And users want to GET THINGS DONE, so they can move on with their lives. All products should solve problems, there's no doubt that it's essential. All other points are moot if your product is useless. But given that it does solve a problem, if it is also beautiful, fun, and polite, it will go a long way in lowering barriers so that we can all have pearls Before Breakfast.