|
816 | 816 | "source": [ |
817 | 817 | "Games loaded from EFG in OpenSpiel do not take advantage of the full functionality of the package, for example, it is not possible to carry out training with RL algorithms on these games, as in the example above with Tiny Hanabi. The OpenSpiel documentation explains [how to submit new games to the library](https://openspiel.readthedocs.io/en/latest/developer_guide.html#adding-a-game) if you wish to add your own games.\n", |
818 | 818 | "\n", |
819 | | - "We can however use the state representation and play through the game step by step:" |
| 819 | + "We can however use the state representation to simulate a playthrough of the game:" |
820 | 820 | ] |
821 | 821 | }, |
822 | 822 | { |
823 | 823 | "cell_type": "code", |
824 | 824 | "execution_count": null, |
825 | | - "id": "c01c4d6f", |
| 825 | + "id": "97913fe5", |
826 | 826 | "metadata": {}, |
827 | 827 | "outputs": [], |
828 | 828 | "source": [ |
829 | | - "ops_one_card_poker.num_distinct_actions()" |
830 | | - ] |
831 | | - }, |
832 | | - { |
833 | | - "cell_type": "markdown", |
834 | | - "id": "9986860c", |
835 | | - "metadata": {}, |
836 | | - "source": [ |
837 | | - "The one-card poker game has 4 distinct actions, 2 are for the first player (Alice in the example game): \"Bet\" and \"Fold\", and 2 for the second player (Bob): \"Call\" and \"Fold\".\n", |
| 829 | + "players = {0: \"Alice\", 1: \"Bob\", -1: \"Chance\"}\n", |
838 | 830 | "\n", |
839 | | - "Initialising the game state, we can see the current player at the start is the chance player, who deals the cards:" |
840 | | - ] |
841 | | - }, |
842 | | - { |
843 | | - "cell_type": "code", |
844 | | - "execution_count": null, |
845 | | - "id": "3b9cc43b", |
846 | | - "metadata": {}, |
847 | | - "outputs": [], |
848 | | - "source": [ |
| 831 | + "# Create an initial game state, then play through to completion\n", |
849 | 832 | "state = ops_one_card_poker.new_initial_state()\n", |
850 | | - "state" |
851 | | - ] |
852 | | - }, |
853 | | - { |
854 | | - "cell_type": "code", |
855 | | - "execution_count": null, |
856 | | - "id": "e23df723", |
857 | | - "metadata": {}, |
858 | | - "outputs": [], |
859 | | - "source": [ |
860 | | - "state.legal_actions()" |
861 | | - ] |
862 | | - }, |
863 | | - { |
864 | | - "cell_type": "markdown", |
865 | | - "id": "7b0959f9", |
866 | | - "metadata": {}, |
867 | | - "source": [ |
868 | | - "Let's make the chance player's action dealing a King (action 0):" |
869 | | - ] |
870 | | - }, |
871 | | - { |
872 | | - "cell_type": "code", |
873 | | - "execution_count": null, |
874 | | - "id": "4dd5d504", |
875 | | - "metadata": {}, |
876 | | - "outputs": [], |
877 | | - "source": [ |
878 | | - "state.apply_action(0)\n", |
879 | | - "state" |
880 | | - ] |
881 | | - }, |
882 | | - { |
883 | | - "cell_type": "code", |
884 | | - "execution_count": null, |
885 | | - "id": "be557706", |
886 | | - "metadata": {}, |
887 | | - "outputs": [], |
888 | | - "source": [ |
889 | | - "state.legal_actions()" |
890 | | - ] |
891 | | - }, |
892 | | - { |
893 | | - "cell_type": "markdown", |
894 | | - "id": "b4291f07", |
895 | | - "metadata": {}, |
896 | | - "source": [ |
897 | | - "As expected, it's now the first player's (Alice's) turn.\n", |
898 | | - "Let's have Alice choose to \"Bet\" (action 0):" |
899 | | - ] |
900 | | - }, |
901 | | - { |
902 | | - "cell_type": "code", |
903 | | - "execution_count": null, |
904 | | - "id": "bd15369f", |
905 | | - "metadata": {}, |
906 | | - "outputs": [], |
907 | | - "source": [ |
908 | | - "state.apply_action(0)\n", |
909 | | - "state" |
910 | | - ] |
911 | | - }, |
912 | | - { |
913 | | - "cell_type": "markdown", |
914 | | - "id": "cd63f7d7", |
915 | | - "metadata": {}, |
916 | | - "source": [ |
917 | | - "As expected, the current player is now player 2 (Bob), let's check the legal actions available to Bob:" |
918 | | - ] |
919 | | - }, |
920 | | - { |
921 | | - "cell_type": "code", |
922 | | - "execution_count": null, |
923 | | - "id": "8d81ff6b", |
924 | | - "metadata": {}, |
925 | | - "outputs": [], |
926 | | - "source": [ |
927 | | - "state.legal_actions()" |
928 | | - ] |
929 | | - }, |
930 | | - { |
931 | | - "cell_type": "markdown", |
932 | | - "id": "fdb5194f", |
933 | | - "metadata": {}, |
934 | | - "source": [ |
935 | | - "Player 2 (Bob) now has the option to \"Call\" (action 1) or \"Fold\" (action 2).\n", |
936 | | - "Let's have Bob choose to \"Fold\":" |
937 | | - ] |
938 | | - }, |
939 | | - { |
940 | | - "cell_type": "code", |
941 | | - "execution_count": null, |
942 | | - "id": "97913fe5", |
943 | | - "metadata": {}, |
944 | | - "outputs": [], |
945 | | - "source": [ |
946 | | - "state.apply_action(2)\n", |
947 | | - "state" |
| 833 | + "while not state.is_terminal():\n", |
| 834 | + "\n", |
| 835 | + " # Store legal actions of current player in a dict\n", |
| 836 | + " legal_actions = {}\n", |
| 837 | + " for action in state.legal_actions():\n", |
| 838 | + " legal_actions[action] = state.action_to_string(state.current_player(), action)\n", |
| 839 | + "\n", |
| 840 | + " # If player is chance, choose an action according to probability\n", |
| 841 | + " if state.is_chance_node():\n", |
| 842 | + " outcomes_with_probs = state.chance_outcomes()\n", |
| 843 | + " action_list, prob_list = zip(*outcomes_with_probs, strict=True)\n", |
| 844 | + " action = np.random.choice(action_list, p=prob_list)\n", |
| 845 | + " print(\"Dealt card: \", legal_actions[action])\n", |
| 846 | + " state.apply_action(action)\n", |
| 847 | + "\n", |
| 848 | + " # Regular players pick a random legal action.\n", |
| 849 | + " else:\n", |
| 850 | + " action = np.random.choice(state.legal_actions())\n", |
| 851 | + " print(players[state.current_player()], \" action: \", legal_actions[action])\n", |
| 852 | + " state.apply_action(action)\n", |
| 853 | + " print()\n", |
| 854 | + "\n", |
| 855 | + "print(\"Alice receives: \", state.player_return(0), \", Bob receives: \", state.player_return(1))" |
948 | 856 | ] |
949 | 857 | }, |
950 | 858 | { |
951 | 859 | "cell_type": "markdown", |
952 | 860 | "id": "1bf09576", |
953 | 861 | "metadata": {}, |
954 | 862 | "source": [ |
955 | | - "Since Bob Folded, Alice takes the small win and we reach a terminal state." |
| 863 | + "Run the code cell above to simulate different possible playthroughs of the game." |
956 | 864 | ] |
957 | 865 | } |
958 | 866 | ], |
|
0 commit comments