Erlang: a lesson to learn...again!

Erlang is a great language.

[2019-UPDATE] Erlang 22 is OUT, so I wanna to come to the party!

On April 2015, Erlang father's Joe Armstrong give us a very interesting lesson I want to tell about.

There was a long thread titled "Erlang and Akka, The Sequel" on the erlang mailing list, reasoning about the need of some standard pattern on Promises and Future. A lot of JavaScript libraries deal about that (also jQuery has its implementation). I want to report the Joe Amstrong reply because it give us a very clear understanding on the reason Erlang is different and you should at least try it once.

"From Joe email (code bold by me) Promises, futures and so on are extremely easy to carry out in Erlang. This is how I explain things when I teach Erlang:

We'll start with an RPC - we can write this in many ways. One way might be:

  rpc(Pid, Query) ->
   Ref = make_ref(),
   Pid ! {self(), Ref, Query},
   receive
     {Ref, Response} ->
         Response
   end.

The server that receives this message does something like:

 receive
    ...
    {From, Ref, Query} ->
        Response = ...
           From ! {Ref, Response}
    ...
 end,

This basic pattern is repeated all over the place is with many small variations.

Remember this pattern (repeat 100 times) - this should dance out of your fingertips and not require conscious thought

For example, all the gen_server does is wrap this pattern with a few convenience functions.

Now keeping the server code unchanged we can modify the RPC

Start with the original (and stare at the added comment):

 

rpc(Pid, Query) ->
 Ref = make_ref(),
 Pid ! {self(), Ref, Query},
 %% watch this space **************
 receive
 {Ref, Response} ->
   Response
 end.

Now I'll rename rpc as rpc1 and split it into two functions at the comment:

rpc1(Pid, Query) ->
 Ref = make_ref(),
 Pid ! {self(), Ref, Query},
 Ref.

wait(Ref) ->
 receive
   {Ref, Response} ->
      Response
 end.

So obviously

 

rpc(Pid, Query) ->
 Ref = rpc1(Pid, Query),
 wait(Ref).

 

How about some renaming? I'll call rpc1 "promise" and wait "yield"

So

promise(Pid, Query) ->
 Ref = make_ref(),
 Pid ! {self(), Ref, Query},
 Ref.

yield(Ref) ->
 receive
   {Ref, Response} ->
       Response
 end.

<aside>we've invented futures :-) </aside> Now we can do something in the gap between the promise and the yield:

compute_something(...) ->
 P1 = promise(...)
 Val1 = ... some local computation ...
 Val2 = yield(P1),
 ...

So now Val1 and Val2 are computed in parallel. (We've now invented one of the basic mechanisms for parallel programming this might appear as parbegin ... parend in some programming language :-)

The reason why Erlang does not have futures/promises/ .. or whatever else you might like to call them is that they are trivially implemented in a few lines of code using just spawn/send/receive. In languages that are basically sequential this is not possible - that's why it's a big deal (TM) to have libraries or language features to support this.

And now for the tricky part .... Broken Promises - Remember that scene in Casablanca when Iisa confronts Rick, this is all about broken promises. Fulfilled promises are easy to deal with, but we must ask what happens if the server crashes and never sends back a message? Now life gets difficult, and as Rick found out the consequences of a broken promise lead to all sorts of problems...

In Erlang, dealing with broken promises is possible (though not easy) using links, monitors and by trapping exits.

Giovanni humble note: because Erlang is a functional side effect-free language, dealing with broken promise is a lot easier.

On Ruby/Python/Java/PHP if you get an exception you fail and report the error to the user:  you have an hard time figuring how to redo your work, and you end up rolling back your work. You need to be sure your object are in the right state, your variable are reset and so on: a nightmare-

With Erlang you can relaunch the processes, and the entire business logic can be playback again. There are some limitations (like dealing with data already commited, system to resync and so on) but is is a lot easier in a functional language.

 

The spawn/send/receive group of primitives are used to program the non error cases where things don't go wrong. trap_exits/links/monitors deal with error cases where things go wrong.

The gen_servers, supervision trees and so on just wrap these primitives in combinations that we have found useful for  solving "common" problems.

In cases where the library behaviors don't do exactly what you want it's often easier to "roll you own" rather than shoehorning your problem into the wrong solution.

The reason there are not a load of design patterns is that we don't need them.

We do need to teach the basics though. spawn/send/receive are as basic to Erlang as for/if/case are to sequential programming .

This is where we have a problem - in sequential languages nobody bothers to teach what for/if/case/switch/while etc do - it is "implicit knowledge that all programmers have" (I'm excluding total beginners here) - Experienced Erlang programers know the power of spawn/send/receive so rarely bother to explain how to build things with these primitives.

My advice would be to stare hard at spawn/send/receive and memorise the RPC pattern and the basic client/server setup.

Then understand links. Write things without using the libraries - then learn the libraries. Note I said you can write RPC in "many" ways - what did I mean by this? To illustrate, here's are some variations:

rpc(Pid, Query) ->
 Pid ! {self(), Query},
 receive
 {Pid, Response} ->
 Response
 end.

or

rpc(Pid, Query) ->
 Pid ! {self(), Query},
 receive
 {Pid, Response} ->
 Response
 after 10000 ->
 exit(timout)
 end.

or

rpc(Pid, Query) ->
 Pid ! {self(), Query},
 receive
 {Pid, Response} ->
 {ok, Response}
 after 10000 ->
 {error, timeout}
 end.

You can see how the basic pattern remains - we can add timeouts etc. but then we have to decide what to do with the variants.

Only the most common patterns are found in the gen_server so it's a good idea to understand the basic pattern and modify it to your specific needs.

/Joe