Basic Q learning training script #33
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mIXs222
reviewed
Feb 11, 2024
Contributor
mIXs222
left a comment
mIXs222
left a comment
There was a problem hiding this comment.
Looks good overall. Please post if have a promising result on actual notebooks, then we can merge this in.
When we train on actual notebooks, it may be worth looking at parallel learning in both episodes + learners to save learning time.
experiments/pod.Dockerfile
Outdated
Comment on lines
39
to
40
| # Copying over the simple notebook for basic training tests | ||
| COPY ./notebooks/simple.ipynb /pod/notebooks/simple.ipynb |
Contributor
There was a problem hiding this comment.
No need for this. Docker compose mount the pod directory so notebooks/simple.ipynb will be there
pod/model.py
Outdated
| self.history = [] | ||
|
|
||
| def plot_rewards(self): | ||
| # Can't plt.show when running on docker apparently, so printing them out to plot on other machine |
Contributor
There was a problem hiding this comment.
If you really want to visualize, you can 1) plot and save fig, or 2) dump to csv/json and plot later
pod/train.py
Outdated
Comment on lines
1
to
13
| from pod.bench import Notebooks, NotebookExecutor, BenchArgs | ||
| from pod.pickling import StaticPodPickling | ||
| from pod.storage import DictPodStorage | ||
| from model import QLearningPoddingModel | ||
| from pod.stats import ExpStat | ||
| from pod.feature import __FEATURE__ | ||
| from typing import List | ||
| import time | ||
| from pod.common import PodId | ||
| from loguru import logger | ||
| import gc | ||
| import random | ||
| import numpy as np |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reward function needs more thought.