DQN tutorial page broken

[taliesinb]

I don't know if this is the right repo, but the tutorial page is broken.

The ipynb notebook on the page (evaluated using Python 3.7.1, pytorch 1.0.0, gym 0.10.9, macOS) fails with the following stacktrace on the final training loop:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-9-9f03f522635e> in <module>
      9         # Select and perform an action
     10         action = select_action(state)
---> 11         _, reward, done, _ = env.step(action[0, 0])
     12         reward = Tensor([reward])
     13 

~/anaconda3/lib/python3.7/site-packages/gym/envs/classic_control/cartpole.py in step(self, action)
     90 
     91     def step(self, action):
---> 92         assert self.action_space.contains(action), "%r (%s) invalid"%(action, type(action))
     93         state = self.state
     94         x, x_dot, theta, theta_dot = state

AssertionError: tensor(0) (<class 'torch.Tensor'>) invalid

The colab link also fails on the final training loop, but for a different reason:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-9-0db7c0f10295> in <module>()
      3     # Initialize the environment and state
      4     env.reset()
----> 5     last_screen = get_screen()
      6     current_screen = get_screen()
      7     state = current_screen - last_screen

<ipython-input-4-db314a5502f8> in get_screen()
     14 
     15 def get_screen():
---> 16     screen = env.render(mode='rgb_array').transpose(
     17         (2, 0, 1))  # transpose into torch order (CHW)
     18     # Strip off the top and bottom of the screen

/usr/local/lib/python3.6/dist-packages/gym/envs/classic_control/cartpole.py in render(self, mode)
    148 
    149         if self.viewer is None:
--> 150             from gym.envs.classic_control import rendering
    151             self.viewer = rendering.Viewer(screen_width, screen_height)
    152             l,r,t,b = -cartwidth/2, cartwidth/2, cartheight/2, -cartheight/2

/usr/local/lib/python3.6/dist-packages/gym/envs/classic_control/rendering.py in <module>()
     21 
     22 try:
---> 23     from pyglet.gl import *
     24 except ImportError as e:
     25     reraise(prefix="Error occured while running `from pyglet.gl import *`",suffix="HINT: make sure you have OpenGL install. On Ubuntu, you can run 'apt-get install python-opengl'. If you're running on a server, you may need a virtual frame buffer; something like this should work: 'xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>'")

/usr/local/lib/python3.6/dist-packages/pyglet/gl/__init__.py in <module>()
    225     else:
    226         from .carbon import CarbonConfig as Config
--> 227 del base
    228 
    229 # XXX remove

The actual reinforcement_q_learning.py file runs fine, though.

I think the web page is out of date... because this issue seems to be related and it refers to a change that happened in pytorch 0.4.

[kurshakuz]

Hi, I think you could simply call an .item() on action[0, 0]. Worked for me.

# Select and perform an action
     action = select_action(state)
     _, reward, done, _ = env.step(action[0, 0].item())
     reward = Tensor([reward])

Yeah It worked for me to @kurshakuz