That's Research?

Today, Halley achieved an important milestone: for the first time ever, a network round-trip successfully made it from the browser to the application server and back to the browser again. This is good evidence that the Halley approach will work eventually.

It's not quite an echo server yet, though. The Halley::httpd server creates a hard-coded event whenever it receives anything at all from the browser, and sends it to the application server via the Halley::Client object proxying for the browser. Then the Halley::httpd server sends a hard-coded block of JSON whenever it receives anything at all from the Halley::Client object proxying for the application server. And then it stops working until the page is refreshed in the browser. Hum.

Also, at this point, the application server and all the Halley::Client objects are pinned down using POE::Kernel->alias_set. This is bad because it prevents POE from garbage collecting dead clients. Some architectural work will need to be done in the section of Halley::httpd that deals with fetching a Halley::Client to find a better way to pin the client only while there's a browser connected to it.

Next steps: integrate JSON parsing/encoding into the server side, nail down the event wire representation, fix the problem where only one interaction goes through, then figure out what to do about client pinning. With any luck, the last two are related.

The first cut of the client side seems to be done.  That was an order of magnitude less difficult than I thought, though that probably means I have 90% buggy code..

The client can register event handlers that get called with the event name and event data as parameters.  The client can also send events to the server.  That's pretty much it for the interface.  Internally, there are event queues for both directions, with event processing happening one event at a time, and a timer set to continue processing any remaining events.  This avoids locking up the JavaScript runtime for long periods of time.

Next up - back to the server side to design the client object that pairs up the two streams into one coherent interface.

I've finished writing the POE filter to handle HTTP requests.  It's nowhere close to RFC 2616 compliant but it should be good enough to handle the bootstrap page load and the XHR connections.  After accepting a HTTP request, the filter emits a hashref containing the contents of the request.  Other than Content-Length and Transfer-Encoding (which it rejects) it doesn't try to understand any of the headers.

The next step is to take care of the bootstrap page loads.  That should be fairly straightforward.

I've decided to use Perl POE for the server side. It's a nice framework that lets me touch some fairly low level details while still providing a high level interface to application developers, all without doing too much work. Also, I've been meaning to learn it and give it a try, and this isn't a bad opportunity for that.

My first attempt at designing the server side of this framework ("middleware") looks like this:

Each of the sessions in the diagram is a POE::Session. The HTTP sessions will be created by a HTTPD session (not in diagram) and belong to the framework. Client sessions will be associated with HTTP sessions. These Client sessions will interact with an Application session using POE events -- they'll post events to the Application session and the Application session will post events to them.

Sounds pretty convenient, assuming I can make it work.

Well, it's project time again.

This time I am motivated by a long-standing desire to write a browser-based thick client multiplayer RPG, theme undecided. One of the major stumbling blocks I encountered while trying to design one in the past was how to work around the problem where clients (players) could not be aware of each other in real time, even when they were standing in the same room, due to limitations in the nature of the web. Sure, there's AJAX for sending commands quickly while keeping the browser-based UI responsive, but there was no mechanism, other than frequent polling by the client, to allow the server to keep the client's data up to date. Since I had no desire to drown my server in update checks, most of which would be fruitless by design to keep down the latency, that was where the idea stopped.

About a year ago, I became aware of Comet, a name for an idea that's been around for countless years. I believe this page is where the term was first coined. In contrast to the ignorant and abusive comments posted on that page, in my case, giving the idea a name was an important step in realising that the idea actually exists. There was a model for setting up low latency bidirectional communication between the web browser and the web server.

Existing implementations such as cometd using the Bayeux protocol seem to do a good job of the publish/subscribe data distribution model, where services make data streams available and clients can register to receive whichever streams of data they care to receive. This data distribution model is not, however, appropriate for an application where the server must send a more or less unique data stream to each client. (What's really different here is that it's the clients that initiate the actions on each other, not the server. In some ways it's a peer-to-peer command structure with the server only making sure nobody's cheating.) Further, with IP multicast being more or less nonexistent on the public Internet, and not used by cometd anyway, nothing is gained for this particular application by forcing it into a publish/subscribe model by, say, subscribing clients to channels containing public updates relevant to multiple clients as well as channels containing private updates relevant only to a single client.

As existing solutions don't fit the requirements, and tend not to be written in Perl, which I've chosen as the language to use for the server component of the game, it seems that the only solution is to write something myself.

Update: You can see the final post on this project here: Halley 0.7 or read all the posts with the cs539w tag.