# Example from a paper by Milos Hauskrecht submitted to AAAI'97 # # POMDP - model: Maze20 (from the AAAI-97 paper) # # ;; ************ basic description **************** # Number of states: 20 # Number of actions: 6 # Number of observations: 8 # # Task: maximization # Horizon: infinite # Discount: 0.9 # Goal: big reward when move action at state 7 # # States: numbered from 0 to 19 # # Actions: numbered from 0 to 5 # 0 - move north # 1 - move south # 2 - move east # 3 - move west # 4 - make observation (north-south) # 5 - make observation (east-west) # # Observations: numbered from 0 to 7 # 0 no-observation (unknown) # 1 no wall # 2 north wall # 3 south wall # 4 both north and south walls # 5 east wall # 6 west wall # 7 both east and west walls # # ;; ************************************************** # ;; for your format values: reward discount: 0.9 states: s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s12 s13 s14 s15 s16 s17 s18 s19 actions: a0 a1 a2 a3 a4 a5 observations: o0 o1 o2 o3 o4 o5 o6 o7 start: 0.3 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.4 ###################################################################### # TRANSITION PROBABILITIES ###################################################################### T: a0 0.15 0.15 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0.15 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0.3 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.4 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.15 0 0 0 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 T: a1 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.4 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0 0.15 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0.15 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0 0 0 0.15 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.85 T: a2 0.15 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.3 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.4 0 0 0 0.15 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0.7 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.3 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.15 0.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.0 T: a3 0.85 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.15 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.15 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0.3 0 0 0 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.4 0 0 0 0.15 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.15 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.7 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0 0.85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0 0 0.7 0.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.7 0.3 T: a4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 T: a5 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 ###################################################################### # OBSERVATION PROBABILITIES ###################################################################### O: a0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a1 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a3 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 O: a4 0 0.14 0.01 0.8 0.05 0 0 0 0 0.05 0.1 0.1 0.75 0 0 0 0 0.05 0.1 0.1 0.75 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.05 0.1 0.1 0.75 0 0 0 0 0.89 0.05 0.05 0.01 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.89 0.05 0.05 0.01 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.89 0.05 0.05 0.01 0 0 0 0 0.14 0.01 0.8 0.05 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.05 0.1 0.1 0.75 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.14 0.8 0.01 0.05 0 0 0 0 0.05 0.1 0.1 0.75 0 0 0 O: a5 0 0.14 0 0 0 0.01 0.8 0.05 0 0.89 0 0 0 0.05 0.05 0.01 0 0.89 0 0 0 0.05 0.05 0.01 0 0.89 0 0 0 0.05 0.05 0.01 0 0.14 0 0 0 0.8 0.01 0.05 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 0 0.05 0 0 0 0.1 0.1 0.75 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 0 0.05 0 0 0 0.1 0.1 0.75 0 0.14 0 0 0 0.01 0.8 0.05 0 0.14 0 0 0 0.8 0.01 0.05 0 0.14 0 0 0 0.01 0.8 0.05 0 0.89 0 0 0 0.05 0.05 0.01 0 0.89 0 0 0 0.05 0.05 0.01 0 0.89 0 0 0 0.05 0.05 0.01 0 0.14 0 0 0 0.8 0.01 0.05 ###################################################################### # REWARDS ###################################################################### #;; EXPECTED ONE STEP COST/REWARD MODEL (rows: actions, columns: states) #### Action a0 R : a0 : s0 : * : * 3.4 R : a0 : s1 : * : * 1.2 R : a0 : s2 : * : * 1.2 R : a0 : s3 : * : * 4.0 R : a0 : s4 : * : * 0.6 R : a0 : s5 : * : * 3.4 R : a0 : s6 : * : * 3.4 R : a0 : s7 : * : * 150.0 R : a0 : s8 : * : * 0.6 R : a0 : s9 : * : * 3.4 R : a0 : s10 : * : * 3.4 R : a0 : s11 : * : * 0.6 R : a0 : s12 : * : * 2.8 R : a0 : s13 : * : * 3.4 R : a0 : s14 : * : * 0.6 R : a0 : s15 : * : * 0.6 R : a0 : s16 : * : * 1.2 R : a0 : s17 : * : * 1.2 R : a0 : s18 : * : * 1.2 R : a0 : s19 : * : * 0.6 #### Action a1 R : a1 : s0 : * : * 0.6 R : a1 : s1 : * : * 1.2 R : a1 : s2 : * : * 1.2 R : a1 : s3 : * : * 1.2 R : a1 : s4 : * : * 0.6 R : a1 : s5 : * : * 3.4 R : a1 : s6 : * : * 0.6 R : a1 : s7 : * : * 150.0 R : a1 : s8 : * : * 3.4 R : a1 : s9 : * : * 0.6 R : a1 : s10 : * : * 3.4 R : a1 : s11 : * : * 3.4 R : a1 : s12 : * : * 2.8 R : a1 : s13 : * : * 0.6 R : a1 : s14 : * : * 3.4 R : a1 : s15 : * : * 3.4 R : a1 : s16 : * : * 1.2 R : a1 : s17 : * : * 4.0 R : a1 : s18 : * : * 4.0 R : a1 : s19 : * : * 0.6 #### Action a2 R : a2 : s0 : * : * 3.4 R : a2 : s1 : * : * 2.8 R : a2 : s2 : * : * 2.8 R : a2 : s3 : * : * 3.4 R : a2 : s4 : * : * 0.0 R : a2 : s5 : * : * 4.0 R : a2 : s6 : * : * 0.6 R : a2 : s7 : * : * 150.0 R : a2 : s8 : * : * 3.4 R : a2 : s9 : * : * 0.6 R : a2 : s10 : * : * 4.0 R : a2 : s11 : * : * 0.6 R : a2 : s12 : * : * 1.2 R : a2 : s13 : * : * 3.4 R : a2 : s14 : * : * 0.6 R : a2 : s15 : * : * 3.4 R : a2 : s16 : * : * 2.8 R : a2 : s17 : * : * 3.4 R : a2 : s18 : * : * 3.4 R : a2 : s19 : * : * 0.0 #### Action a3 R : a3 : s0 : * : * 0.6 R : a3 : s1 : * : * 2.8 R : a3 : s2 : * : * 2.8 R : a3 : s3 : * : * 3.4 R : a3 : s4 : * : * 2.8 R : a3 : s5 : * : * 1.2 R : a3 : s6 : * : * 3.4 R : a3 : s7 : * : * 150.0 R : a3 : s8 : * : * 0.6 R : a3 : s9 : * : * 3.4 R : a3 : s10 : * : * 1.2 R : a3 : s11 : * : * 3.4 R : a3 : s12 : * : * 1.2 R : a3 : s13 : * : * 0.6 R : a3 : s14 : * : * 3.4 R : a3 : s15 : * : * 0.6 R : a3 : s16 : * : * 2.8 R : a3 : s17 : * : * 3.4 R : a3 : s18 : * : * 3.4 R : a3 : s19 : * : * 2.8 #### Action a4 R : a4 : * : * : * 2 #### Action a5 R : a5 : * : * : * 2