1. Kenneth Love
  2. DjangoCon 2011 Notes


DjangoCon 2011 Notes / docs / pdx_python_meetup_sept_11.rst

PDX Python Meetup

Delicious data with mmstats


  • You have an app.
  • It has state.
  • What is my app doing?
  • How do you inspect for that state?
  • Simple in-memory stats get hard to expose in multi-process environments.

Solution #1: Logging


  • Universally supported
  • Easily enhanced
  • Persistent


  • Libraries suck
  • Operational burden (rotating, shipping, routing, etc)
  • Records events more than inspects state
  • Difficult to predict where needed
  • Performance impact if too verbose

Solution #2: Graphite


  • Fast, sexy, "enterprise"
  • Python


  • Do you want a graph or a graph?
  • Have fun installing it.
  • Still not great for introspection.

Solution #3: socketconsole


  • Pure Python.

  • Very useful for deadlocks, blocking code, threaded app.

  • Simple to integrate:

    import socketconsole


  • CPython only.
  • Doesn't work with gevent or eventlet monkeypatching.
  • Doesn't work with greenthreads.
  • Limited functionality.
  • All the fun of Python threads.

Solution #4: REPL Backdoors


  • Pure Python!
  • Changing code at runtime is for winners.
  • Inspect all the things!


  • With great power comes great responsibility.
  • Requires threads or event loop.
  • Still can't reach all state.

Solution #5: GDB


  • Well, you wanted introspection.
  • Has some Python helpers: pygdb, gdb-heap


  • Seriously? Definitily only a a last resort.

Solution #6: JMX


  • Universally supported.
  • Powerful, extensible, 2-way.
  • Helpful tools.


  • Where "universal" means "runs on the JVM".
  • Not as easy to monitor as you'd think.


Python needs this.

mmstats Goals

  • Simple API to expose state.
  • Separate publishing from reading, aggregating, etc.
  • Language, platform, framework agnostic.
  • Minimal & predictable performance impact.
  • Optional persistence (e.g. post-mortems)
  • 1-way (for now?)

What is mmstats?

  • mmap alows sharing memory between processes.

  • Language independent data structure:

    • Series of fields (structs)
    • Fields have label, type, and values.
  • Exposed in Python app as a model class.

Performance Implementation

Single writer, multi-reader

  • No locks.
  • No syscalls (write, read, send, recv, etc).
  • All in userspace for readers & writers.
  • Reading has no impact on writers.
  • Fixed field sizes.

Consistency without Locks

Q: How are reads consistent without locks?
A: Double buffering.