Q-Learner question from chapter 5

[jesmitty]

I was trying to run Q_learner_MountainCar.py in Chapter 5. I had an issue running it. I think the issue is in:

def discretize(self, obs):
    return tuple(((obs - self.obs_low) / self.bin_width).astype(int))

I get a:
Input In [6], in Q_Learner.discretize(self, obs)
37 def discretize(self, obs):
38 #print("obs from discretize is ",obs, self.obs_low, self.bin_width)
39 #print tuple(((obs - self.obs_low) / self.bin_width).astype(int))
---> 40 return tuple(((obs - self.obs_low) / self.bin_width).astype(int))

TypeError: unsupported operand type(s) for -: 'dict' and 'float'

The script was written for learning environments like 'MountainCar-v0' with vector (Box(2)) as the observation space but Q_learner_MountainCar.py is using the MountainCar-V0 example. Was anyone able to run Q_learner_MountainCar.py and if so, what changes did you have to make to the code (if any).

[jesmitty]

Note that Chapter 5 code does use 'MountainCar-vo'. It is that env that throws the error.

[praveen-palanisamy]

Hi @jesmitty ,
Thank you for posting on the discussion thread with the details. Upon debugging to reproduce the issue you reported, it turns out, you are using a different version of the OpenAI Gym library than the version used in the Book's recipe samples which is pinned down in the python environment spec here.

Recommended Solution
To resolve the issue you reported, you may want to install the recommended gym library version in your python environment. For example, by running this command: pip install gym==0.10.5.

More details

The Gym library has changed the interface contract of the reset() method in this commit to return an auxiliary information dict in addition to the observation when reset() is called. This breaking change affects gym versions >=0.26.0 based on the commits included in this tagged release. The latest compatible gym version for this Book is gym==0.25.2.

Alternate Solution
For some reason, if you would like to keep using your existing python environment with an unsupported gym version, to run the Q_learner_MountainCar.py in Chapter 5, please change this following line:

Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch5/Q_learner_MountainCar.py

Line 61 in 174ccf9

obs = env.reset()

with:
obs, _ = env.reset()

[jesmitty]

Thank you

[praveen-palanisamy]

Feel free to post if you have any follow-up questions or if it's resolved, please mark as answer to complete this discussion thread. Thank you!