Skip to content

Adding heterogeneous observation social dilemma environment #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
marimeireles wants to merge 38 commits into
base: main
Choose a base branch
from
Draft

Adding heterogeneous observation social dilemma environment #4

marimeireles wants to merge 38 commits into from

Conversation

@marimeireles
Copy link
Collaborator

@marimeireles marimeireles commented Mar 15, 2024

No description provided.

@marimeireles
add base env for multi obs and socialdill with multi obs
4e65617
@marimeireles
change names to MultipleObsSocialDilemma from socialdilemma
5a21dac
@marimeireles
Work feb21
0860304
@marimeireles
add hetero
5416ebb
@marimeireles
Modify multi obs social dilemma
51b9adf
@marimeireles
dont remember wt i changed hetero and mul obs env
cc44813
@marimeireles
fix observabiity reward and transition for heterogeneuous environments
6dea33c
@marimeireles
Fix the observation gen tension function and fix from transparency to…
fa3668d 
… opacity to keep consistency among the code
@marimeireles
Last version of hetero env before making it compat with history
85f10e5
@marimeireles
Change Observation type to be explicitely string
3ab50a5
@marimeireles
Change hetero env so its env = . .
26f086f
@marimeireles
Change set in the correct place
b827a82
@marimeireles
Remove this nb bc it's confusing me
e45d1b6
@marimeireles
Add the states to the right class finally
e96586d
@marimeireles
Update py files for heteroobs and multiobsdilemma
5e05c86
@marimeireles
Stringfication of history into py file
006d119
@marimeireles
Call Tss function with an explicity object
b415071
@marimeireles
Organize and add some tests to the multi obs env
a558dca
@marimeireles
Observations dont have to sum up[ to 1 anymore because we might have …
f960634 
…observations = 0.5 for partial observable agents
@marimeireles
Fix the label generation for Osets
c297264
@marimeireles
Testing a bunch of stuff remove null chars so i can compile?
ef85f5b
@marimeireles
Remove null chars from ebase
627a9c4
@marimeireles
Change the if conditions so it accepts any kinds of numbers not only …
81deb30 
…ints
@marimeireles
Make call for parent class explicit on APOBase
aaf561c
@marimeireles
Make call for parent class explicit on APOBase
3c70da3
@marimeireles
Notebooks with corrected observations and different kinds + a few exa…
dffeeb6 
…mples set up
@marimeireles
Finish 12multipleobssoc and rm explorations
ce6b25b
@marimeireles
Add the ability to have different types of obs within the same game
6d47d3d
@marimeireles
Remove content added by me on hist env embedding nb
8ed5f37
@marimeireles
Adding screenshot from tests ran in h222 env
fa8f6dc
@review-notebook-app
Copy link

review-notebook-app bot commented Mar 15, 2024

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

marimeireles
marimeireles commented Mar 15, 2024
View reviewed changes
pyCRLD/Environments/HistoryEmbedding.py
return histSjA_RewardTensor(self.baseenv, self.h)

def ObservationTensor(self):
Copy link
Collaborator Author

@marimeireles marimeireles Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed this function to be able to generate different observation tensors for each agent.

pyCRLD/Environments/HeterogeneousObservationsEnv.py
@@ -0,0 +1,170 @@
# AUTOGENERATED! DO NOT EDIT! File to edit: ../../nbs/Environments/02_HeterogeneousObservationsEnv.ipynb.
Copy link
Collaborator Author

@marimeireles marimeireles Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the changes are within this file. It's largely adapting the ebase file to deal with multiple observations.

pyCRLD/Environments/MultipleObsSocialDilemma.py
@@ -0,0 +1,127 @@
# AUTOGENERATED! DO NOT EDIT! File to edit: ../../nbs/Environments/12_MultipleObsSocialDilemma.ipynb.
Copy link
Collaborator Author

@marimeireles marimeireles Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file simply implements the social dilemma layer into the heterogeneous observation env. file.
I've initially tried to incorporate the "contract" idea because I saw this in the Uncertain Environment file, however, I don't really understand the dynamics of contract and I don't think it's fully functional, I need to work on it.
I thought it wasn't relevant for our project as IPD only has the one state .. Please let me know if I misunderstood this.

@marimeireles marimeireles marked this pull request as draft March 15, 2024 15:09
@marimeireles
Copy link
Collaborator Author

marimeireles commented Mar 15, 2024 •
edited
Loading

I'm also a bit confused on whether it's possible to have observation tensors summing for numbers > 1 or < 1. I guess the only reason why we cannot is because of the generate_stochastic_observations... But we could change that.
I'm not sure if it makes sense, but I thought it could be possible for an agent to have for example, 0% chance of observing something. Or having tensors looking like [0, 0.8, 0.6, 0.4] or like [0.7,0.,0.,0.]. If this is not possible are there other reasons why it is not possible other than using generate_stochastic_observations in the step function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@marimeireles