Category Archives: Documentation

Metabrowsing! (Or, custom interactors for MozRepl)

First guy: “So how would an object inspector look like in MozRepl?”

(Flashback: MozRepl is an interactive shell for Firefox and other Mozilla apps. You connect to Firefox with telnet or similar, type some JavaScript code, and it gets executed on the fly.)

I patiently explained how a command line inspector would look like. More or less like:

First guy: “Well, and how do you get this in MozRepl?”

I (im)pantietly explained that there was no inspector yet.

First guy: “Why not?”

I (im)patiently…

Second guy, out of nowhere: “Yeah, why not?”

You know that you’re in trouble when even the nicest guy in the place teams up to nag you.

Then I spent some time not to write an inspector, but to make it possible (and easy) for third parties to write new interaction modes for MozRepl, so that the onus of writing the damn inspector would be on the above guys. :) (The best thing you can do with challenges isn’t winning them, it’s turning them upside down.)

However things quickly got fancier while I explored, because the same request/response pattern interactive interpreters are built around is common to a lot of applications. Object inspectors, sure. Even text adventures (“You’re in a dimly-lit browser, surrounded by menacing angled brackets and curly braces”).

But also HTTP.

One custom interactor later, there it was: Firefox understanding HTTP and browsing itself. :)

Where to go from here? Once there’s a halfway decent implementation of HTTP (hint: mine isn’t), you may be able to drive Firefox with XML-RPC, JSON-RPC, REST-style manipulation of resources, and who knows what else. (Oh, yeah, I guess server-side long-timers in the audience will know what-else very well.)

Support is available in the new stand-alone MozRepl package. To try out the http-inspect mode, just go to about:config and set the “extensions.mozrepl.defaultInteractor” pref to “http-inspect”, then point the browser to http://localhost:4242. Documentation on how to write custom interactors is on the wiki.

Credits to Sergi for coming up with the “metabrowsing” title!

Synchronous invocation in JavaScript, part 4: error handling

Last time we saw how to transfer results of computations from pseudo-blocking calls back into the main process. What happens if when something goes wrong, though?

In the (real) synchronous world, error conditions used to travel on the same road as return values:


    char *foo;
    foo = doSomething();
    if (foo == NULL) {
        error("can't get foo");
        exit(1);
    } else {
        doSomethingElseWith(foo);
    }

Then someone realized that error handling and the real task would better not be intertwined, or the general sense of the latter might become less and less clear. After all, errors were, er, the exception, not the rule, and might best be handled in a parallel world that veery now and then resurfaced, but not in the middle of nice-looking, expressive code.


    try {
        var foo = doSomething();
        doSomethingElseWith(foo);
    } catch(e if e == NoFooException) {
        error("can't get foo");
    }

Normally, asynchronous code in JavaScript puts us back in the first scenario, although we don’t get the result/error through a return value but as a function argument:


    var req = new XMLHttpRequest();
    req.open('GET', url, true);
    req.onreadystatechange = function(event) {
        if(req.readyState == 4) {
            if(req.status != 200)
                error('Something went wrong!');
            else
                doSomethingWith(req);
        }
    };
    req.send(null);

Turning it into pseudo-synchronous makes things only a bit better. The error comes through a return value and we deal with errors exactly as in the first scenario:


    function getUrl(url) {
        return function(driver) {
            var req = new XMLHttpRequest();
            req.open('GET', url, true);
            req.onreadystatechange = function(event) {
                if(req.readyState == 4)
                    driver(req);
            };
            req.send(null);
        }
    }

    var driver = proc(function() {
        dump("Retrieving URL...\n");
        var req = yield(getUrl(''));
        if(req.status != 200)
            dump('error!');
        else
            dump(req.responseText + '\n');
    });

    driver();

What we want, though, is to make use of the exception “channel”, for the reasons given above: separating description of the task from description of what to do when something goes wrong.

It takes some extra load in the process definition that hopefully isn’t too hard on the eyes:


    var driver = proc(function() {
        try {
            var result = check(yield(
                errorProneFunction()
            ));
            dump('Success! ' + result);
        } catch(e) {
            dump('Something went wrong :-( ' + e);
        }
    });

    driver();

Implementation of check() and changes to the process driver are trivial:


    function check(obj) {
        if(obj instanceof Error)
            throw obj;
        else
            return obj;
    }

    function proc(processDef) {
        var process = processDef();

        function driver(value) {
            try {
                var pseudoBlockingCall = process.send(value);

                try {
                    pseudoBlockingCall(driver);
                } catch(e) {
                    driver(e)
                }
            } catch(e if e == StopIteration) {

            }

        }

        return driver;
    }

  • The inner try...catch inside the process driver catches exceptions thrown from pseudo-blocking functions, and passes them back to the process as if they were values, so they will be returned to the caller of the pseudo-blocking function.
  • check() receives yield()‘s return value. If it’s an exception, it throw()s it (thereby sending it across the usual exception channel), otherwise just returns the untouched vaule.
  • An extra try...catch in the process driver lets it ignore the StopIteration exception—we won’t go into that here.

The example below, relying on the revised implementation of proc(), shows that exceptions thrown synchronously in a pseudo-blocking call can be caught and exposed within the process:


    function errorProneFunction() {
        return function(driver) {
            throw new Error('Something went wrong!');
            driver('Some other value for correct computation');
        }
    }

    var driver = proc(function() {
        try {
            var result = check(yield(
                errorProneFunction()
            ));
            dump('Success! ' + result);
        } catch(e) {
            dump('Error. :-( ' + e);
        }
    });

    driver();

Exceptions can be thrown by the asynchronous part of pseudo-blocking call, too: just pass them as argument to driver() where you’d normally pass the result of the computation:


    function getUrl(url) {
        return function(driver) {
            var req = new XMLHttpRequest();
            req.open('GET', url, true);
            req.onreadystatechange = function(event) {
                if(req.readyState == 4) {
                    if(req.status != 200)
                        driver(new Error('Non-success HTTP status!'));
                    else
                        driver(req.responseText);
                }
            };
            req.send(null);
        }
    }

Articles in this series:

  1. Part 1: Problem and basic solution
  2. Part 2: Execution dissected
  3. Part 3: Returning values
  4. Part 4: Error handling

Synchronous invocation in JavaScript, part 3: returning values

We’ve seen how to simulate a blocking sleep() in JavaScript and what happens under the hood when we do.

Another scenario where a blocking call makes a big difference in terms of clarity is requesting data from the network and suspending execution until the answer arrives:


    var resume = proc(function() {
        dump("Retrieving URL...\n");
        var data = yield(getUrl(''));
        dump(data + '\n');
    });

getUrl() produces a pseudo-blocking function which, when invoked, sends an AJAX request and handles the response. “Handling the response” in this case means passing the received data back to the process driver.

We don’t really care about the implementation of getUrl() now, so it’ll use a fake ajaxRequest() internally; see bottom of post for an implementation that actually does something.


    function getUrl(url) {
        return function(driver) {
            sendAjaxRequest(url, function(response) {
                driver(response);
            });
        }
    }

The implementation of driver() we saw in the previous parts won’t allow passing values back to the process. That’s easy to fix, though: just turn process.next() into process.send(value) where value is whatever was passed to driver().


    function proc(processDef) {
        var process = processDef();

        function driver(value) {
            var partialComputation = process.send(value);

            partialComputation(advance);
        }

        return driver;
    }

Before, the pseudo-blocking call would call driver(), which would in turn call process.next(), causing execution of the process to be resumed from the line after yield().

Now, the pseudo-blocking call calls driver(value) instead. Inside the driver, that leads to the process.send(value) which passes control back to the process, where value becomes the return value of yield() and gets assigned to data. Execution finally resumes from the line after yield().

Here’s the working version of getUrl():


    function getUrl(url) {
        return function(driver) {
            var req = new XMLHttpRequest();
            req.open('GET', url, true);
            req.onreadystatechange = function(event) {
                if(req.readyState == 4)
                    driver(req.responseText);
            };
            req.send(null);
        }
    }

Articles in this series:

  1. Part 1: Problem and basic solution
  2. Part 2: Execution dissected
  3. Part 3: Returning values
  4. Part 4: Error handling

Synchronous invocation in JavaScript, part 2: execution dissected

Let’s review the process definition and process constructor from part 1:


     1  function proc(processDef) {
     2      var process = processDef();
     3
     4      function driver() {
     5          var pseudoBlockingCall = process.next();
     6
     7          pseudoBlockingCall(driver);
     8      }
     9
    10      return driver;
    11  }

    12  var driver = proc(function() {
    13      dump("Going to bed...\n");
    14      var pseudoBlockingCall = function(driver) {
    15          window.setTimeout(driver, 3000);
    16      });
    17      yield(pseudoBlockingCall);
    18      dump("Sigh, I only slept three seconds.\n")
    19  });

    20  driver();

The conversation among the three actors (process definition, process driver and process costructor) goes like this. Notice the many control transfers, and remember that executing the process definition to get the process (at line 2) returns a generator—that’s why process can be entered and exited many times at intermediate stages.

(line) main constructor driver process
12 driver = proc(…
02 var process = fn()
10 return driver
20 driver()
05 process.next()
(process.next() in the driver is called for the first
time. Control goes to the process, and the process advances until the
first yield().)
13 dump(“Going to bed…\n”);
14 pseudoBlockingCall = …
08 yield(pseudoBlockingCall)
(yield() causes control to pass back to the
driver, and yield()’s argument (the pseudo-blocking
function) becomes the return value of
process.next().)
17 pseudoBlockingCall = process.next()
07 pseudoBlockingCall(driver)
15 window.setTimeout(driver, 3000);
05 process.next();
(The pseudo-blocking function is executed and told to invoke
the driver again when it’s finished.)
18 dump(“Sigh, I only slept three seconds.\n”);
(The driver has been invoked again, and the process is told to
advance to the next yield() (or to the end).)

Not bad for less than twenty lines of code…

In the next part, we’ll see how to make conversation more lively by allowing pseudo-blocking functions to compute and return values.

Articles in this series:

  1. Part 1: Problem and basic solution
  2. Part 2: Execution dissected
  3. Part 3: Returning values
  4. Part 4: Error handling

Synchronous invocation in JavaScript, part 1: problem and basic solution

(Update: a library based on this series is in use in SamePlace and is available here.)

Here’s something most languages can do which JavaScript can’t (at least when hosted in the browser):


    print("Going to bed...");
    sleep(3000);
    print("Sigh, I only slept three seconds.")

In JavaScript that would be:


    dump("Going to bed...\n");
    setTimeout(function() {
        dump("Sigh, I only slept three seconds.\n");
    }, 3000);

sleep() blocks until it’s done. That wouldn’t work very well in JavaScript because, while sleep() is sleeping, buttons, menus, and so on would stop responding. Same goes for sending AJAX requests and waiting for responses, or for querying the user and waiting for input.

So, JavaScript has to resort to callbacks. It isn’t that bad when you just want to wait three seconds, but imagine sending an AJAX request, waiting for the result, asking the user something based on that result, sending another AJAX request… callback depending on callback depending on callback. Or asynchrospaghetti, for friends.

Anyway, I’ll show a way to get in JavaScript something that looks like the first example. It’ll take a lot of effort, which sucks because the outcome is so trivial, but it’ll also lay foundations to build cool stuff, e.g. message passing.

You’ll need a JavaScript implementation that supports generators, which means JavaScript 1.7, which means no Internet Explorer. But then again if you’re reading this blog you’re not too heart-broken about that.

A basic form of pseudo-blocking call

Last things first. This is what we’ll end up with:


    var driver = proc(function() {
        dump("Going to bed...\n");
        yield(sleep(3000));
        dump("Sigh, I only slept three seconds.\n")
    });

    driver();

Aside from yield(), the body of the function looks refreshingly similar to the first example. We’ll explode sleep() immediately so that we don’t have to keep track of extra indirection. As for proc(), we’ll talk about it in the next part, but if you’re eager to see something running, scroll to the bottom to find its implementation.


    var driver = proc(function() {
        dump("Going to bed...\n");
        var pseudoBlockingCall = function(driver) {
            window.setTimeout(driver, 3000);
        });
        yield(pseudoBlockingCall);
        dump("Sigh, I only slept three seconds.\n")
    });

    driver();

Let’s take this apart. There are three actors here:

  • a process constructor (the proc() call)
  • a process driver, defined in proc() and called from various places)
  • the process definition itself (the argument to proc())

The process constructor proc() receives the process definition as an argument, internally sets up a process driver for it, then returns the process driver. Invoking the process driver starts the process.

The process definition passed to proc() looks like a function but it isn’t—it’s a generator. Think of a generator as a function that can be called, it suspends, returns a value, it’s called again, it resumes, it suspends again, returns another value, and so on. Those intermediate returns happen when a yield() instruction is met. We’re going to exploit this to simulate blocking calls.

In the next part, we’ll unroll execution line by line. For now, here’s the implementation of proc():

  function proc(processDef) {     // Execute the argument, resulting in a generator object.  The     // process isn't executed yet.          var process = processDef();      // The process driver.  It's deceptively simple!  This will be     // returned from proc() and passed as argument to the pseudo-blocking     // calls, and will be invoked many times.      function driver() {         var pseudoBlockingCall = process.next();          pseudoBlockingCall(driver);     }          return driver; } 

Articles in this series:

  1. Part 1: Problem and basic solution
  2. Part 2: Execution dissected
  3. Part 3: Returning values
  4. Part 4: Error handling