# POMDP example by Milos Hauskrecht # Maze20B with a zero cost absorbing goal state # # !!! Note: the structure of the maze is completely different than # the structure used for Maze20 problem # # ************ basic description **************** # Task: minimization # Horizon: infinite # Discount: 0.95 # Goal: state 0 (zero cost sink) # # Number of states: 20 # Number of actions: 6 # Number of observations: 8 # # States: numbered from 0 to 19 # # Actions: numbered from 0 to 5 # 0 - move north # 1 - move south # 2 - move east # 3 - move west # 4 - make observation in north-south # 5 - make observation in east-west # # Observations: numbered from 0 to 7 # 0 no-observation (unknown) # 1 no wall # 2 north wall # 3 south wall # 4 both north and south walls # 5 east wall # 6 west wall # 7 both east and west walls # # ************************************************** # the following is the description of the problem using Tony # Cassandra's exchange format for transition and observation # models. However cost model is provided only in a simple one step # cost matrix form. values: cost discount: 0.95 # horizon: infinite states: s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s12 s13 s14 s15 s16 s17 s18 s19 actions: a0 a1 a2 a3 a4 a5 observations: o0 o1 o2 o3 o4 o5 o6 o7 ################################################################ # TRANSITION PROBABILITIES ################################################################ T: a0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 T: a1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0.15 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0.15 0.15 T: a2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.7 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.85 T: a3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.15 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.15 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.15 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0.7 0.15 T: a4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 T: a5 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 ############################################################### # OBSERVATION PROBABILITIES ############################################################### O: a0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a1 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a3 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a4 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.89 0.05 0.05 0.01 0 0 0 0 0.89 0.05 0.05 0.01 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.05 0.1 0.1 0.75 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.89 0.05 0.05 0.01 0 0 0 0 0.89 0.05 0.05 0.01 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.05 0.1 0.1 0.75 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 O: a5 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 0 0.05 0 0 0 0.1 0.1 0.75 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 0 0.14 0 0 0 0.01 0.8 0.05 0 0.89 0 0 0 0.05 0.05 0.01 0 0.89 0 0 0 0.05 0.05 0.01 0 0.89 0 0 0 0.05 0.05 0.01 0 0.14 0 0 0 0.8 0.01 0.05 0 0.05 0 0 0 0.1 0.1 0.75 0 0.14 0 0 0 0.01 0.8 0.05 0 0.89 0 0 0 0.05 0.05 0.01 0 0.89 0 0 0 0.05 0.05 0.01 0 0.14 0 0 0 0.8 0.01 0.05 0 0.05 0 0 0 0.1 0.1 0.75 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 ######################################################### # REWARDS ######################################################## # EXPECTED ONE STEP COST/REWARD MODEL (rows: actions, columns: states) 0 21.5 23.0 21.5 21.5 28.5 27.0 20.0 20.0 21.5 30.0 21.5 27.0 27.0 21.5 23.0 28.5 28.5 28.5 28.5 0 28.5 30.0 28.5 28.5 21.5 20.0 27.0 20.0 21.5 23.0 28.5 27.0 20.0 21.5 23.0 21.5 28.5 21.5 21.5 0 28.5 28.5 21.5 28.5 21.5 21.5 21.5 20.0 27.0 28.5 21.5 23.0 21.5 27.0 27.0 21.5 30.0 21.5 28.5 0 21.5 28.5 28.5 21.5 28.5 21.5 21.5 20.0 20.0 28.5 28.5 23.0 21.5 20.0 27.0 28.5 23.0 28.5 21.5 0 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 0 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20