Category Archives: Erlang

Dynamic variables hack in Erlang

One thing that jumps to the eye in functional code sources is longer-than-usual argument lists.


process(Tree, Env, Plugins, Config, ...).

It’s easy to see that as a consequence of functions using arguments as sole input channel. (Global -define()‘s don’t qualify as input channels—they can’t vary at runtime, arguments can.)

Usually it’s no big hassle.

Sometimes though you have a front-end function to a complex computation which is arranged in many layers of function calls, and you need to pass something to the front-end that only the lowest layer needs. You’d rather to not care about it in intermediate layers, but you still have to because intermediate layers forward it to the lowest one.

Obligatory contrived example (if you can read Ruby, Dave Thomas has a better one):


%% User invocation
library:do_it(A, B, C, D).

%% Library
do_it(A, B, C, D) ->
    part1(A) ++ part2(B, C, D).

part2(B, C, D) ->
    extra_computation(B) ++ part3(C, D).

part3(C, D) ->
    another_extra_computation(C) ++ part4(D).

%% ...

Above, D isn’t really needed until part4, but part1, part2 and part3 have to accept it as argument.

Functions being unable to see beyond their argument list spares me many a headache compared to when I dwell in OO-land, so I tend to deem it a feature, not a bug. In cases like the above however I’d like to relax boundaries. Especially when doing exploratory programming and part1, part2, partN have several clauses and invocation points, maintaining “forwarding” argument lists can impair your flow.

Common Lisp has dynamic variables. They’re best described through an example (leaving out the parentheses, since they scare away large chunks of the population):


> defvar *player* "Dr. Falken"
*PLAYER*

> defun player-greeting
>   concatenate 'string "Hello, " *player*
PLAYER-GREETING

> player-greeting
"hello, Dr. Flaken"

> let ((*player* "Matt")) player-greeting
"hello, Matt"

> player-greeting
"hello, Dr. Flaken"

let temporarily changes the binding of *player* for the scope player-greeting is evaluated in.

Here’s a way to do it in Erlang. (Read it up to the end, there are caveats.)

First, user’s point of view:


player_greeting() ->
    "Hello, " ++ dynvar:fetch(player).

test() ->
    Greeting = dynvar:with([{player, "Dr. Falken"}],
                           fun player_greeting/0),
    Greeting.

Invoking test/0 will produce "Hello, Dr. Falken".

The code:


-module(dynvar).
-export([with/2, fetch/1, test/0]).

with(Bindings, Action) ->
    lists:foreach(fun({Name, Value}) -> put(Name, Value) end, Bindings),
    try apply(Action, []) of
        X -> X
    after
        lists:foreach(fun({Name, _Value}) -> erase(Name) end, Bindings)
    end.

fetch(VarName) ->
    get(VarName).

Yes, the process dictionary seems to be at the epicenter of dirty Erlang hacks. :-)

A few considerations:

  • Common Lisp’s player-greeting can work without a “let” explicitly binding a value to *player*. The Erlang example can’t. This isn’t necessarily a drawback (see next point).
  • Syntax of the Erlang example is more verbose. A few bytes could be shaved off using macros and ditching dynvar:fetch/1, but that would only be a win in the eyes of the “beauty above all” crowd (surface beauty, let me add). First, indirecting via a macro would make life harder for whomever maintains the code six months down the road. Second, when you’re going against the grain of the language, you may want the comments to mention it, and you really want the code to scream it. (Do as I say, don’t do as I do.)
  • This may collide with other uses of the process dictionary. Consider adding a unique prefix to keys (e.g. dynvar-) before put()’ting them into the dictionary if that is an issue.
  • Where will you want to use this? Well, I’m using it in the internals of the seethrough template engine to avoid passing arguments through intermediate layers. Using it in library internals, preferably within a single module, not leaking signs of dynvar to the library user, is where I can advise to use this and still retain a clear conscience.

Literal XML in Erlang with parse_transform/2

One of the things I dislike about Erlang is that it severely impairs bragging opportunities. Yesterday I wrote a module that allows writing literal XML in the source and have it parsed into Erlang structures at compile time—sort of like E4X minus the manipulation goodies at runtime (at least for now).

You write:


Doc = '<greeting>Hello!</greeting>',
io:format("~p~n", [Doc]).

And it prints…


{xmlElement,greeting,
            greeting,
            [],
            {xmlNamespace,[],[]},
            [],
            1,
            [],
            [{xmlText,[{greeting,1}],1,[],"Hello!",text}],
            [],
            "/tmp",
            undeclared}

In most languages I’m familiar with, this would have granted the author instant Yacc-demigod status. With Erlang… it was less than 40 LOC. Hardly something you’d wear at a party.

Anyway, this code owes everything to Philip’s writings. It also uses parse_transform/2, and “programmers are strongly advised not to engage in parse transformations and no support is offered for problems encountered”. So unless you, like me, are still at the kid-in-a-candy-shop stage of Erlang experience, think twice before using this in production, ok?

The code is here.

Installing the StartCom SSL certificate in ejabberd

The XMPP Software Foundation established an intermediate certification authority with StartCom. If you run a public federated XMPP server, in order to provide secure communication, you no longer need to buy an SSL certificate (or resort to a self-signed certificate): simply register an account at http://www.xmpp.net and follow the certificate request process.

At least up to ejabberd 1.1.2 1.1.4, however, there is an extra step which involves patching a file and recompiling. (Update: the patching step is no longer required in ejabberd 2.0.0.)

Here is the complete procedure I followed.

After the certificate request process, you should have these files:

Decode ssl.key. openssl will ask you for a password, provide the one you gave during the certificate request process:


$ openssl rsa -in ssl.key -out ssl.key

Concatenate your server’s certificate plus key and the intermediate certificate into a single file:


cat ssl.crt ssl.key sub.class1.xmpp.ca.crt >ejabberd.pem


Place the resulting file where the ejabberd server is able to access it.

On Debian:


chown ejabberd.ejabberd ejabberd.pem
chmod 400 ejabberd.pem
mv ejabberd.pem /etc/ejabberd

Configure ejabberd.cfg:


% Ordinary client-2-server service
 [{5222, ejabberd_c2s,     [{access, c2s},
                            {max_stanza_size, 65536},
                            starttls, {certfile, "/etc/ejabberd/ejabberd.pem"},
                            {shaper, c2s_shaper}]},

% SSL-enabled client-2-server service
  {5223, ejabberd_c2s,     [{access, c2s},
                            {max_stanza_size, 65536},
                            tls, {certfile, "/etc/ejabberd/ejabberd.pem"},
                            {shaper, c2s_shaper}]},

% [...]

% Use STARTTLS+Dialback for S2S connections
{s2s_use_starttls, true}.
{s2s_certfile, "/etc/ejabberd/ejabberd.pem"}.

Restart the server.

At this point, the certificate is installed but ejabberd is not presenting it correctly. If you run the following:


openssl s_client -connect your.server.org:5223 -CAfile /path/to/ca.crt

You will get an incomplete certificate chain:


[...]
verify error:num=21:unable to verify the first certificate
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:/C=US/ST=Your State/L=Your Location/O=Your Name/OU=Domain validated only/CN=your@server.org/emailAddress=hostmaster@server.org
   i:/C=US/ST=Colorado/O=Jabber Software Foundation/OU=Secure Certificate Signing/CN=StartCom Class 1 Intermediate CA - Jabber Software Foundation/emailAddress=certmaster@jabber.org
---
Server certificate
-----BEGIN CERTIFICATE-----
[...]

To fix it, you have to apply a small patch. Start by downloading ejabberd source.

On Debian:


apt-get source ejabberd

Now you could grab the patch from the ejabberd bug tracker and use the “patch” tool to apply it, however as of ejabberd 1.1.2 line numbers have shifted and it won’t apply cleanly. Since it’s a one-liner, just open the file src/tls/tls_drv.c and locate the following line:


res = SSL_CTX_use_certificate_file(d->ctx, buf, SSL_FILETYPE_PEM);

Replace it with the following:


res = SSL_CTX_use_certificate_chain_file(d->ctx, buf);

To compile it, either go to the src/ directory and type:


make

Or, on Debian, generate a new package:


fakeroot dpkg-buildpackage -uc -nc

Reinstall, and you’re done.

To verify that it’s working, run again:


$ openssl s_client -connect sameplace.cc:5223 -CAfile /path/to/ca.crt

This time you should get:


[...]
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:/C=US/ST=Your State/L=Your Location/O=Your Name/OU=Domain validated only/CN=your@server.org/emailAddress=hostmaster@server.org
   i:/C=US/ST=Colorado/O=Jabber Software Foundation/OU=Secure Certificate Signing/CN=StartCom Class 1 Intermediate CA - Jabber Software Foundation/emailAddress=certmaster@jabber.org
 1 s:/C=US/ST=Colorado/O=Jabber Software Foundation/OU=Secure Certificate Signing/CN=StartCom Class 1 Intermediate CA - Jabber Software Foundation/emailAddress=certmaster@jabber.org
   i:/C=IL/ST=Israel/L=Eilat/O=StartCom Ltd./OU=CA Authority Dep./CN=Free SSL Certification Authority/emailAddress=admin@startcom.org
---
Server certificate
-----BEGIN CERTIFICATE-----
[...]

Which is a complete certificate chain and openssl is able to verify it against the root certificate.

seethrough 0.1: sub-templates, and cache-ready

Uh oh. Beware when you take an evening’s hack and put it online, as someone who knows what he’s doing might come along and turn it into something useful.

That happened to seethrough, with a patch courtesy of Joel, who added two major features:

  • sub-template inclusion, via the <e:include/> tag;
  • parse/render stage split.

The latter means that the (possibly expensive) parse and preparation stage only needs to be performed once. The resulting intermediate form can be cached, and rendered many times within different environments. For example:


%% Parsing
{XMLTree, _Misc} = xmerl_scan:file(File),
Intermediate = seethrough:visit(XMLTree),

%% Rendering intermediate form within Env1
Render1 = seethrough:render(Intermediate, Env1),
xmerl:export_simple(lists:flatten([Render1]), xmerl_xml,
                    [#xmlAttribute{name = prolog, value = ""}]).

%% More rendering, but within Env2
Render2 = seethrough:render(Intermediate, Env2),
xmerl:export_simple(lists:flatten([Render2]), xmerl_xml,
                    [#xmlAttribute{name = prolog, value = ""}]).

%% Yet more rendering, within Env3
Render3 = seethrough:render(Intermediate, Env3),
xmerl:export_simple(lists:flatten([Render3]), xmerl_xml,
                    [#xmlAttribute{name = prolog, value = ""}]).


Kudos to Joel!

seethrough: a simple XML/XHTML templating system for Erlang

Speaking of winter evenings, a couple of weeks ago over Christmas I spent a few hours whipping together a simple templating system for Erlang.

Today I took another couple of hours to put it under version control and to add some ideas I found in ZPT, a similar system (hat tip to Jérôme for pointing it out).

This is how it works: you start with an XHTML (or any XML) file with some extra attributes thrown in and properly namespaced:


<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:e="http://dev.hyperstruct.net/seethrough">
  <head>
    <title e:content="title"/>
  </head>
  <body>
    <h1 e:content="title"/>

    <h2><span e:replace="subtitle"/></h2>

    <table>
      <thead>
        <tr>
          <th>Address</th>
          <th>Name</th>
        </tr>
      </thead>
      <tbody>
        <tr e:repeat="crew">
          <td e:content="address"/>
          <td e:content="name"/>
        </tr>
      </tbody>
    </table>
  </body>
</html>

Since it is a valid document, even designers can load it into their WYSIWYG editors and be happy.

Next, you pass the file name along with an environment (a list of key/value tuples) to apply_template/2:


seethrough:apply_template("test.html",
    [{title, "Space"},
     {subtitle, {mymod, get_subtitle, []}},
     {crew, {mymod, get_crew, []}}])).

Given such input and such environment:

  • <h1 e:content="title"/> will be replaced by <h1>Space</h1>.
  • <span e:replace="subtitel"/> will be replaced by the result of mymod:get_subtitle/0.
  • <tr e:repeat="crew">...</tr> will be repeated as many times as there are elements in the (list-)result of mymod:get_crew/0.

Here is the output:


<?xml version="1.0"?>
<html xmlns:e="http://dev.hyperstruct.net/seethrough"
      xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Space</title>
  </head>
  <body>
    <h1>Space</h1>

    <h2>The last frontier...</h2>

    <table>
      <thead>
        <tr>
          <th>Address</th>
          <th>Name</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td>kirk@enterprise.glx</td>
          <td>Jim</td>
        </tr><tr>
          <td>spock@enterprise.glx</td>
          <td>Spock</td>
        </tr><tr>
          <td>mccoy@enterprise.glx</td>
          <td>Doc</td>
        </tr>
      </tbody>
    </table>
  </body>
</html>

Only a tiny part of ZPT (basically, what is needed in the above example) is mirrored so far. While I will bring more of it in, since it is obviously well designed, this will not be a port, partly because Erlang is not Python and there might be other good design choices around the corner, partly because I spent some time generating XHTML in Erlang the usual way ({body, [], [{p, [], [...]}], gen_table(...)}) and while it might not be the most familiar thing in the world, dealing with XML generation as function composition is fascinating and it would be nice to bring that into the picture.

The code is available at http://repo.hyperstruct.net/seethrough.

Update 13/01: Nevow XML Templates provide more excellent inspiration.

Update 17/01: The <attr> tag á la Nevow has been added (it affects attributes of the parent element). A project page is also available at /projects/seethrough, please see that for an updated example of template processing.

Update 13/02: Now with sub-templates and cache-ready.