Blocksworld, repopulated!



In the experimental setup a fluid with Euglena is observed via a microscope connected to a camera. The image is analyzed with a proximity based identity tracking algorithm (using OpenCV) to identify individual Euglena and determine their position, size and motion. The data is send via a network protocol (OSC) to a game engine (Unity3D) to alter elements of the game situation. A feedback system from the game to the microorganisms, by controlling LEDs to affect the environment of the biological system, is planed.

The Game

Menu: Before you start the game you need to configure the OSC connection data to connect the game with the tracking software (see OSC configuration). Than press start.

Top view: Before you enter Blocksworld you see the world from above. The four white cubes/lights on each side are conceptually the four LED`s under the microscope setup. You can activate them by clicking on them and an OSC message is send (e.g. /LED/1 1). After changing the light setup, you can enter the world by pressing the button.

First person view: When you enter the world your objective is to collect keys (rainbow coloured spheres) dropped by the Euglena NPCs in order to activate the level gate and escape the maze. The maze consist of blocks, that are continuously rearranged by the Euglenas.

Keyboard Shortcuts

  • TAB: use tab key to switch between Top View and First Person View

  • Esc: use Esc key to restart game (generate new level)

Application (.exe / .app)


OSC configuration

Listener (to recieve osc data)

  • listening port Choose the correct listening port as defined in the Processing patch. Default is 8000.


/meand f: float[x,y]

/euglena fffffff: float[id,x,y,width,height,dx,dy]

Transmitter (to send osc data)

  • transmit ip Choose ip address to define where to send data. Default is (localhost)
  • transmit port Choose port to define over which port the data should be send. Default is 8001.

Send Messages

/LED/1 f: float (intensity 0.0 - 1.0)

/LED/2 f: float (intensity 0.0 - 1.0)

/LED/3 f: float (intensity 0.0 - 1.0)

/LED/4 f: float (intensity 0.0 - 1.0)

Source code



  • File > Open Scene > StartGame.unity
  • Press Play Button (or Ctrl + P) to start game

OSC Notes

You can change the default IP & Port number of the OSC server/client, to not always change it manually in the build application.

  • Open the Unity project
  • Load Scene: StartGame (Double Click on: Project/Assets/StartGame
  • Select OSC in Hierarchy
  • Change the values in Inspector/OSC Control (Script)
  • press File/Save Scene
  • check File/Build Settings
  • build new application


Processing > Unity

/meand f: float[x,y]

/euglena fffffff: float[id,x,y,width,height,dx,dy]

Unity > Processing (or something else)

/LED/1 f: float (intensity 0.0 - 1.0)

/LED/2 f: float (intensity 0.0 - 1.0)

/LED/3 f: float (intensity 0.0 - 1.0)

/LED/4 f: float (intensity 0.0 - 1.0)


The MIT License (MIT) Copyright (c) 2015 Jonas Hansen,