runroot-eco /

Filename Size Date modified Message
824 B
1.0 KB
3.9 KB
what:  The first version of runroot comet + Erlang tie ins
when:  February 2009 to November 2009.  Released January 2010.
where: everywhere
why:   It's around 2k LOC and full of good examples I don't want to waste.

  This repository should be called "damn you, Richard Jones!"  I thought it
would be a snazzy idea to make a generic, fast, and scalable Erlang comet
system based on what metabrew described in parts one, two, and three at:

  eco stands for 'Erlang comet.'  It could also mean I've been watching too
many episodes of Dollhouse.  The idea was to make a specific implementation
of a comet<->Erlang bridge then break out common elements so anybody could
attach to the comet and Erlang guts using plugins/modules/shared objects.  This
is why the code is extremely runroot specific in places (it never got
generalized since the premise of using libevent is flawed).

  It started out so simple.  Use Richard's experiences as a guideline and
eventually build something everybody could use as a browser<->comet<->Erlang
system.  It turns out some of the examples are fatally flawed at scale.
Libevent is nowhere near threadsafe using the same event base with its built
in http library.  The metabrew examples segfault spectacularly when
abused with load.

  So I added fixes.  I worked around the problems.  One thread can't talk to
the other?  Fine.  Let libevent do its thing and we can listen on a socket
whose only purpose is to send data between threads.  Sure it's wasteful, but
it's quick enough.  Uh oh.  Clients aren't timing out.  Libevent won't tell 
us when a client disconnects.  Let's add some timers.  Uh oh, the timers are
hitting race conditions.  Hey, why don't we re-write the entire thing in Erlang?

  And thus this codebase was abandoned and the entire system was rewritten in
500 lines of Erlang in 8 hours instead of this monstrosity of 2000 lines of code
I fought on and off for almost 10 months.  The Erlang version is what production uses now.

  But all is not for naught.  This code has some great examples of using the 
Erlang C interfaces, libevent, CMake, SMF, and gluing everything together.  As
long as you don't need complex libevent timers and cleanup code, everything
here would be trivial to modify and have running quickly.

  If anybody wants to attempt this again, don't use libevent.  Use libev instead
and write a simple http interface for it.  Everything should be much more
stable at that point.  Unwritten code is always perfectly stable, right?

This Code Contains
This repository has good examples of:
  - Erlang cnode code with error handling and auto-reconnect (node.c)
  - How to deal with and properly free terms from erl_decode
  - Handling net_adm:ping in a cnode (comet_server.c)
  - Freeing all memory used by creating Erlang terms.
    The eco system has no memory leaks.
  - CMake for detecting Erlang library locations and pretty auto-building
  - The circular buffer may be mildly interesting.  You can ask it for the most
    recent N entries.
  - Wrapping common things in macros
  - Using lazy debugging printf macros
  - OpenSolaris/Solaris 10 SMF scripts

This repository has poor examples of:
  - Data structures
    - Everything is a statically allocated array with entries stored sorted
      so we can look up using a binary search of the array.
      Entries are kept sorted by inserting to the first open slot in the
      array then re-sorting the array.  For each insert.
  - Command line argument parsing (really, don't duplicate it anywhere)
  - Test code for the cbuf.  I was testing for circularish-ness, and not
    absolute correctness.

How to Build
  Inside the 'src' directory, you can use regular 'make' or 'cmake'.

  Regular make:

  CMake out-of-tree build (best option):
    mkdir build
    cmake ..