Bitbucket is a code hosting site with unlimited public and private repositories. We're also free for small teams!


pyPOMDP is a POMDP implementation for Python 2.x.


The following example show how the contoller and the simulator can be used.

First, the POMDP model is read from file (either Cassandra's file format or a JSON file).

The controller returns a control action on sending an observation (senseact()). The control action is selected according to it's current belief state and policy. The policy is also read from a file (so far no online planning).

The simulator samples the current system state internally from the process model with respect to the applied action. Based on that, it records the reward and emmits a sampled observation to the controller.

See the following examples for implementation details:

import time
import pypomdp

def main():
    # model
    model = None

    # read from JSON
    model_filename = './examples/models/shuttle.95.JSON'
    with open(model_filename, 'r') as fh:
        model = pypomdp.model.POMDP.read_json(fh)

    # simulator
    simulator = pypomdp.simulator.Simulator(pomdp=model)

    # controller
    policy_filename = './examples/models/shuttle.95.alpha'
    policy = pypomdp.policy.AlphaVectorValueFunction.\
            initFromCassandraFile(policy_filename, model)
    controller = pypomdp.control.ProdController(model=model, policy=policy)

    # run system
    action = controller.last_action
        reward, observ = simulator.simulate(action)
        action = controller.senseact(observ)

if __name__ == "__main__":

Development Notes:

Indexing for transition probabilities, observation probabilities and rewards:

  • transition probabilities: [action][source][sink]
  • observation probabilities: [action][state][observ]
  • rewards: [action][source][sink][observ]

Recent activity

Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.