learning NIO via clojure

Short summary of recent events:

• Heysan got acquired
• I work for the acquirer (Good Technology / previously known as Visto Corp)
• Good acquired InterCasting Corp
• Met Ben from ICC
• Moved to new appt with Ben

One of the first things I was tasked with at Good was to get IM working in the Intercasting product using heysan as the backend; intercasting had been using Openfire. This involved us building a protocol which essentially replaced Jabber + XEP-0100 + Gateways. It’s called GIMP. I’ve not done any direct socket stuff like this before, at least not in Java, I was put on to the Apache MINA Project. In short, 3-4 months later – it’s really not a good library.

Part of living with Ben has involved me getting into the clojure scene a little. The combination of MINA being crap and clojure being interesting has me re-writing GIMP in clojure, this meant I have to learn NIO. I’ve learned some interesting things about NIO in the process:

• It sucks pretty bad.
• Most examples on the internet suffer because of that, i.e. most of them are inaccurate

The latter point forms about 50% of the reason for this post. 25% is about clojure, the remainder was the update above :)

The stage I’m at is still very early on, I have a socket and I want events that tell me connections are being made. In NIO you work with selectors, they “select” the socket (this is basic Socket 101). When an event takes place it goes into a queue which you read from the selector using the method .selectedKeys() – it returns a Set<SelectionKey>. Most examples show you removing the key after processing it, this is not required  (at least so far it isn’t..); this caused a lot of distraction for me because Clojure generally has immutable collections, thus remove seemed weird to require. The event you register with your Selector determines what you must do to each SelectionKey to ensure it is removed from the selector; otherwise you end up with a the SelectionKey not ever being removed.

Consider, you register OP_ACCEPT, you must accept the connection for it to leave the queue. This wasn’t immediately clear to me, perhaps I’m stupid and missed this. If you don’t do this that connection attempt event stays in the selector and thus in the inifinte loop you have which selects on the socket you end up with a tight loop. The set you get from .selectedKeys() is populated at the time of calling so only if you’re going to iterate it multiple times do you need to remove events from it (as many of the examples online show).

I’ve two implementations of a very simple system which accepts a connection and prints out a message. The exercise of getting this done in clojure was pretty difficult for a couple of reasons:

• There are examples in Java (though they are often inaccurate they work)
• clojure removes a few constructs which make “sense” (for loops for example)
• the clojure equivalents are often purposefully lazy (incase of large data sets)

Implementations: (Java) (Clojure)

You’ll see the (doall map …) stuff in the recv function, well that forces the realization of the sequence. I believe this is possible using the for form, I’ll work that out later.

I’m thinking that this socket stuff might form the basis of me attempting to create a jabber library, I’ll probably not get that far. The only real java library I managed to be get on with was smack, and that was bad for our usage. We had a server side instance of it thus could have benefited from them creating sockets using NIO. I actually think the functional form makes a lot of sense for something like jabber, we’ll see. Anyways, expect a few posts detailing what I find about NIO and clojure.

mport java.nio.channels.*;

import java.net.*;

import java.util.*;

import java.io.*;

public class Server {

private ServerSocketChannel server;

private Selector selector;

public static void main(String[] args) throws Exception {

Server s = new Server(args[0], Integer.valueOf(args[1]));

s.recvLoop();

}

public void recvLoop() throws Exception {

for (;;) {

System.out.println(“About to select”);

System.out.println(String.format(“found %d elements”, selector.select()));

Set<SelectionKey> keys = selector.selectedKeys();

for (SelectionKey key : keys) {

//SelectionKey key = it.next();

//it.remove(); //required

keys.remove(key);

Socket s = ((ServerSocketChannel)key.channel()).accept().socket();

PrintWriter out = new PrintWriter(s.getOutputStream(), true);

out.println(“k, thx, bye”);

out.close();

}

Thread.sleep(500);

}

}

public Server(String host, int port) throws Exception {

server = ServerSocketChannel.open();

server.configureBlocking(false);

server.socket().bind(new InetSocketAddress(host, port));

selector = Selector.open();

server.register(selector, SelectionKey.OP_ACCEPT);

}

}