Synchronous invocation in JavaScript, part 3: returning values

We’ve seen how to simulate a blocking sleep() in JavaScript and what happens under the hood when we do.

Another scenario where a blocking call makes a big difference in terms of clarity is requesting data from the network and suspending execution until the answer arrives:


    var resume = proc(function() {
        dump("Retrieving URL...\n");
        var data = yield(getUrl('http://hyperstruct.net'));
        dump(data + '\n');
    });

getUrl() produces a pseudo-blocking function which, when invoked, sends an AJAX request and handles the response. “Handling the response” in this case means passing the received data back to the process driver.

We don’t really care about the implementation of getUrl() now, so it’ll use a fake ajaxRequest() internally; see bottom of post for an implementation that actually does something.


    function getUrl(url) {
        return function(driver) {
            sendAjaxRequest(url, function(response) {
                driver(response);
            });
        }
    }

The implementation of driver() we saw in the previous parts won’t allow passing values back to the process. That’s easy to fix, though: just turn process.next() into process.send(value) where value is whatever was passed to driver().


    function proc(processDef) {
        var process = processDef();

        function driver(value) {
            var partialComputation = process.send(value);

            partialComputation(advance);
        }

        return driver;
    }

Before, the pseudo-blocking call would call driver(), which would in turn call process.next(), causing execution of the process to be resumed from the line after yield().

Now, the pseudo-blocking call calls driver(value) instead. Inside the driver, that leads to the process.send(value) which passes control back to the process, where value becomes the return value of yield() and gets assigned to data. Execution finally resumes from the line after yield().

Here’s the working version of getUrl():


    function getUrl(url) {
        return function(driver) {
            var req = new XMLHttpRequest();
            req.open('GET', url, true);
            req.onreadystatechange = function(event) {
                if(req.readyState == 4)
                    driver(req.responseText);
            };
            req.send(null);
        }
    }

Articles in this series:

  1. Part 1: Problem and basic solution
  2. Part 2: Execution dissected
  3. Part 3: Returning values
  4. Part 4: Error handling
Share

Synchronous invocation in JavaScript, part 2: execution dissected

Let’s review the process definition and process constructor from part 1:


     1  function proc(processDef) {
     2      var process = processDef();
     3
     4      function driver() {
     5          var pseudoBlockingCall = process.next();
     6
     7          pseudoBlockingCall(driver);
     8      }
     9
    10      return driver;
    11  }

    12  var driver = proc(function() {
    13      dump("Going to bed...\n");
    14      var pseudoBlockingCall = function(driver) {
    15          window.setTimeout(driver, 3000);
    16      });
    17      yield(pseudoBlockingCall);
    18      dump("Sigh, I only slept three seconds.\n")
    19  });

    20  driver();

The conversation among the three actors (process definition, process driver and process costructor) goes like this. Notice the many control transfers, and remember that executing the process definition to get the process (at line 2) returns a generator—that’s why process can be entered and exited many times at intermediate stages.

(line) main constructor driver process
12 driver = proc(…
02 var process = fn()
10 return driver
20 driver()
05 process.next()
(process.next() in the driver is called for the first
time. Control goes to the process, and the process advances until the
first yield().)
13 dump(“Going to bed…\n”);
14 pseudoBlockingCall = …
08 yield(pseudoBlockingCall)
(yield() causes control to pass back to the
driver, and yield()’s argument (the pseudo-blocking
function) becomes the return value of
process.next().)
17 pseudoBlockingCall = process.next()
07 pseudoBlockingCall(driver)
15 window.setTimeout(driver, 3000);
05 process.next();
(The pseudo-blocking function is executed and told to invoke
the driver again when it’s finished.)
18 dump(“Sigh, I only slept three seconds.\n”);
(The driver has been invoked again, and the process is told to
advance to the next yield() (or to the end).)

Not bad for less than twenty lines of code…

In the next part, we’ll see how to make conversation more lively by allowing pseudo-blocking functions to compute and return values.

Articles in this series:

  1. Part 1: Problem and basic solution
  2. Part 2: Execution dissected
  3. Part 3: Returning values
  4. Part 4: Error handling
Share

Synchronous invocation in JavaScript, part 1: problem and basic solution

(Update: a library based on this series is in use in SamePlace and is available here.)

Here’s something most languages can do which JavaScript can’t (at least when hosted in the browser):


    print("Going to bed...");
    sleep(3000);
    print("Sigh, I only slept three seconds.")

In JavaScript that would be:


    dump("Going to bed...\n");
    setTimeout(function() {
        dump("Sigh, I only slept three seconds.\n");
    }, 3000);

sleep() blocks until it’s done. That wouldn’t work very well in JavaScript because, while sleep() is sleeping, buttons, menus, and so on would stop responding. Same goes for sending AJAX requests and waiting for responses, or for querying the user and waiting for input.

So, JavaScript has to resort to callbacks. It isn’t that bad when you just want to wait three seconds, but imagine sending an AJAX request, waiting for the result, asking the user something based on that result, sending another AJAX request… callback depending on callback depending on callback. Or asynchrospaghetti, for friends.

Anyway, I’ll show a way to get in JavaScript something that looks like the first example. It’ll take a lot of effort, which sucks because the outcome is so trivial, but it’ll also lay foundations to build cool stuff, e.g. message passing.

You’ll need a JavaScript implementation that supports generators, which means JavaScript 1.7, which means no Internet Explorer. But then again if you’re reading this blog you’re not too heart-broken about that.

A basic form of pseudo-blocking call

Last things first. This is what we’ll end up with:


    var driver = proc(function() {
        dump("Going to bed...\n");
        yield(sleep(3000));
        dump("Sigh, I only slept three seconds.\n")
    });

    driver();

Aside from yield(), the body of the function looks refreshingly similar to the first example. We’ll explode sleep() immediately so that we don’t have to keep track of extra indirection. As for proc(), we’ll talk about it in the next part, but if you’re eager to see something running, scroll to the bottom to find its implementation.


    var driver = proc(function() {
        dump("Going to bed...\n");
        var pseudoBlockingCall = function(driver) {
            window.setTimeout(driver, 3000);
        });
        yield(pseudoBlockingCall);
        dump("Sigh, I only slept three seconds.\n")
    });

    driver();

Let’s take this apart. There are three actors here:

  • a process constructor (the proc() call)
  • a process driver, defined in proc() and called from various places)
  • the process definition itself (the argument to proc())

The process constructor proc() receives the process definition as an argument, internally sets up a process driver for it, then returns the process driver. Invoking the process driver starts the process.

The process definition passed to proc() looks like a function but it isn’t—it’s a generator. Think of a generator as a function that can be called, it suspends, returns a value, it’s called again, it resumes, it suspends again, returns another value, and so on. Those intermediate returns happen when a yield() instruction is met. We’re going to exploit this to simulate blocking calls.

In the next part, we’ll unroll execution line by line. For now, here’s the implementation of proc():

  function proc(processDef) {     // Execute the argument, resulting in a generator object.  The     // process isn't executed yet.          var process = processDef();      // The process driver.  It's deceptively simple!  This will be     // returned from proc() and passed as argument to the pseudo-blocking     // calls, and will be invoked many times.      function driver() {         var pseudoBlockingCall = process.next();          pseudoBlockingCall(driver);     }          return driver; } 

Articles in this series:

  1. Part 1: Problem and basic solution
  2. Part 2: Execution dissected
  3. Part 3: Returning values
  4. Part 4: Error handling
Share

SamePlace, the contest, and Italy

This blog used to be mostly about SamePlace. If you’ve been reading it long enough to remember that, you’re probably one of the early adopters, so please virtually celebrate with me this little bit of news. :-)

SamePlace won a grand prize together with Shareaholic and Minimap (the announcement lists them in alphabetic order with a numeric bullet and they look like first/second/third prize—they’re actually co-winners, just like the twelve runners up.)

And it seems it’s been a productive time for Italy: my friend Davide is there too with Table2Clipboard. Not bad for this old little country. :-)

Share

Dynamic variables hack in Erlang

One thing that jumps to the eye in functional code sources is longer-than-usual argument lists.


process(Tree, Env, Plugins, Config, ...).

It’s easy to see that as a consequence of functions using arguments as sole input channel. (Global -define()‘s don’t qualify as input channels—they can’t vary at runtime, arguments can.)

Usually it’s no big hassle.

Sometimes though you have a front-end function to a complex computation which is arranged in many layers of function calls, and you need to pass something to the front-end that only the lowest layer needs. You’d rather to not care about it in intermediate layers, but you still have to because intermediate layers forward it to the lowest one.

Obligatory contrived example (if you can read Ruby, Dave Thomas has a better one):


%% User invocation
library:do_it(A, B, C, D).

%% Library
do_it(A, B, C, D) ->
    part1(A) ++ part2(B, C, D).

part2(B, C, D) ->
    extra_computation(B) ++ part3(C, D).

part3(C, D) ->
    another_extra_computation(C) ++ part4(D).

%% ...

Above, D isn’t really needed until part4, but part1, part2 and part3 have to accept it as argument.

Functions being unable to see beyond their argument list spares me many a headache compared to when I dwell in OO-land, so I tend to deem it a feature, not a bug. In cases like the above however I’d like to relax boundaries. Especially when doing exploratory programming and part1, part2, partN have several clauses and invocation points, maintaining “forwarding” argument lists can impair your flow.

Common Lisp has dynamic variables. They’re best described through an example (leaving out the parentheses, since they scare away large chunks of the population):


> defvar *player* "Dr. Falken"
*PLAYER*

> defun player-greeting
>   concatenate 'string "Hello, " *player*
PLAYER-GREETING

> player-greeting
"hello, Dr. Flaken"

> let ((*player* "Matt")) player-greeting
"hello, Matt"

> player-greeting
"hello, Dr. Flaken"

let temporarily changes the binding of *player* for the scope player-greeting is evaluated in.

Here’s a way to do it in Erlang. (Read it up to the end, there are caveats.)

First, user’s point of view:


player_greeting() ->
    "Hello, " ++ dynvar:fetch(player).

test() ->
    Greeting = dynvar:with([{player, "Dr. Falken"}],
                           fun player_greeting/0),
    Greeting.

Invoking test/0 will produce "Hello, Dr. Falken".

The code:


-module(dynvar).
-export([with/2, fetch/1, test/0]).

with(Bindings, Action) ->
    lists:foreach(fun({Name, Value}) -> put(Name, Value) end, Bindings),
    try apply(Action, []) of
        X -> X
    after
        lists:foreach(fun({Name, _Value}) -> erase(Name) end, Bindings)
    end.

fetch(VarName) ->
    get(VarName).

Yes, the process dictionary seems to be at the epicenter of dirty Erlang hacks. :-)

A few considerations:

  • Common Lisp’s player-greeting can work without a “let” explicitly binding a value to *player*. The Erlang example can’t. This isn’t necessarily a drawback (see next point).
  • Syntax of the Erlang example is more verbose. A few bytes could be shaved off using macros and ditching dynvar:fetch/1, but that would only be a win in the eyes of the “beauty above all” crowd (surface beauty, let me add). First, indirecting via a macro would make life harder for whomever maintains the code six months down the road. Second, when you’re going against the grain of the language, you may want the comments to mention it, and you really want the code to scream it. (Do as I say, don’t do as I do.)
  • This may collide with other uses of the process dictionary. Consider adding a unique prefix to keys (e.g. dynvar-) before put()’ting them into the dictionary if that is an issue.
  • Where will you want to use this? Well, I’m using it in the internals of the seethrough template engine to avoid passing arguments through intermediate layers. Using it in library internals, preferably within a single module, not leaking signs of dynvar to the library user, is where I can advise to use this and still retain a clear conscience.
Share