formatting applied

README.md

@@ -245,7 +245,7 @@ You can use the the following interface to make an environment. You may optional
  * `allow_step_back`: Default `False`. `True` if allowing `step_back` function to traverse backward in the tree.
  * Game specific configurations: These fields start with `game_`. Currently, we only support `game_num_players` in Blackjack, .
 
-Once the environemnt is made, we can access some information of the game.
+Once the environment is made, we can access some information of the game.
 * **env.num_actions**: The number of actions.
 * **env.num_players**: The number of players.
 * **env.state_shape**: The shape of the state space of the observations.

docs/games.md

@@ -90,7 +90,7 @@ At each decision point of the game, the corresponding player will be able to obs
 | ------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
 | seen\_cards | Three face-down cards distributed to the landlord after bidding. Then these cards will be made public to all players. | TQA |
 | landlord | An integer of landlord's id | 0 |
-| self | An integer of current player's id | 2 |
+| cls | An integer of current player's id | 2 |
 | trace | A list of tuples which records every actions in one game. The first entry of the tuple is player's id, the second is corresponding player's action. | \[(0, '8222'), (1, 'pass'), (2, 'pass'), (0 '6KKK'), (1, 'pass'), (2, 'pass'), (0, '8'), (1, 'Q')\] |
 | played\_cards | As the game progresses, the cards which have been played by the three players and sorted from low to high. | \['6', '8', '8', 'Q', 'K', 'K', 'K', '2', '2', '2'\] |
 | others\_hand | The union of the other two player's current hand | 333444555678899TTTJJJQQAA2R |
@@ -134,7 +134,7 @@ If the landlord first get rid of all the cards in his hand, he will win and rece
 ## Mahjong
 Mahjong is a tile-based game developed in China, and has spread throughout the world since 20th century. It is commonly played
 by 4 players. The game is played with a set of 136 tiles. In turn players draw and discard tiles until 
-The goal of the game is to complete the leagal hand using the 14th drawn tile to form 4 sets and a pair. 
+The goal of the game is to complete the legal hand using the 14th drawn tile to form 4 sets and a pair. 
 We revised the game into a simple version that all of the winning set are equal, and player will win as long as she complete 
 forming 4 sets and a pair. Please refer the detail on [Wikipedia](https://en.wikipedia.org/wiki/Mahjong) or [Baike](https://baike.baidu.com/item/麻将/215).
 

docs/high-level-design.md

@@ -25,4 +25,4 @@ Card games usually have similar structures. We abstract some concepts in card ga
 To summarize, in one `Game`, a `Dealer` deals the cards for each `Player`. In each `Round` of the game, a `Judger` will make major decisions about the next round and the payoffs in the end of the game.
 
 ## Agents
-We provide examples of several representative algorithms and wrap them as `Agent` to show how a learning algorithm can be connected to the toolkit. The first example is DQN which is a representative of the Reinforcement Learning (RL) algorithms category. The second example is NFSP which is a representative of the Reinforcement Learning (RL) with self-play. We also provide CFR (chance sampling) and DeepCFR which belong to Conterfactual Regret Minimization (CFR) category. Other algorithms from these three categories can be connected in similar ways.
+We provide examples of several representative algorithms and wrap them as `Agent` to show how a learning algorithm can be connected to the toolkit. The first example is DQN which is a representative of the Reinforcement Learning (RL) algorithms category. The second example is NFSP which is a representative of the Reinforcement Learning (RL) with self-play. We also provide CFR (chance sampling) and DeepCFR which belong to Counterfactual Regret Minimization (CFR) category. Other algorithms from these three categories can be connected in similar ways.

docs/toy-examples.md

@@ -339,7 +339,7 @@ def train(args):
  # Seed numpy, torch, random
  set_seed(args.seed)
 
- # Initilize CFR Agent
+ # Initialize CFR Agent
  agent = CFRAgent(
  env,
  os.path.join(

examples/evaluate.py

@@ -1,19 +1,16 @@
-''' An example of evluating the trained models in RLCard
-'''
+"""An example of evaluating the trained models in RLCard"""
 import os
 import argparse
 
 import rlcard
-from rlcard.agents import (
- DQNAgent,
- RandomAgent,
-)
+
 from rlcard.utils import (
  get_device,
  set_seed,
  tournament,
 )
 
+
 def load_model(model_path, env=None, position=None, device=None):
  if os.path.isfile(model_path): # Torch model
  import torch
@@ -29,14 +26,14 @@ def load_model(model_path, env=None, position=None, device=None):
  else: # A model in the model zoo
  from rlcard import models
  agent = models.load(model_path).agents[position]
- 
+
  return agent
 
-def evaluate(args):
 
+def evaluate(args):
  # Check whether gpu is available
  device = get_device()
- 
+
  # Seed numpy, torch, random
  set_seed(args.seed)
 
@@ -54,6 +51,7 @@ def evaluate(args):
  for position, reward in enumerate(rewards):
  print(position, args.models[position], reward)
 
+
 if __name__ == '__main__':
  parser = argparse.ArgumentParser("Evaluation example in RLCard")
  parser.add_argument(
@@ -99,4 +97,3 @@ def evaluate(args):
 
  os.environ["CUDA_VISIBLE_DEVICES"] = args.cuda
  evaluate(args)
-

examples/human/blackjack_human.py

@@ -1,5 +1,4 @@
-''' A toy example of self playing for Blackjack
-'''
+"""A toy example of self playing for Blackjack """
 
 import rlcard
 from rlcard.agents import RandomAgent as RandomAgent
@@ -23,7 +22,7 @@
 
 print(">> Blackjack human agent")
 
-while (True):
+while True:
  print(">> Start a new game")
 
  trajectories, payoffs = env.run(is_training=False)

examples/human/gin_rummy_human.py

@@ -1,9 +1,9 @@
-'''
+"""
  Project: Gui Gin Rummy
  File name: gin_rummy_human.py
  Author: William Hale
  Date created: 3/14/2020
-'''
+"""
 
 # You need to install tkinter if it is not already installed.
 # Tkinter is Python's defacto standard GUI (Graphical User Interface) package.

examples/human/leduc_holdem_human.py

@@ -1,5 +1,4 @@
-''' A toy example of playing against pretrianed AI on Leduc Hold'em
-'''
+"""A toy example of playing against pretrianed AI on Leduc Hold'em"""
 
 import rlcard
 from rlcard import models
@@ -17,7 +16,7 @@
 
 print(">> Leduc Hold'em pre-trained model")
 
-while (True):
+while True:
  print(">> Start a new game")
 
  trajectories, payoffs = env.run(is_training=False)

examples/human/limit_holdem_human.py

@@ -1,5 +1,4 @@
-''' A toy example of playing against a random agent on Limit Hold'em
-'''
+"""A toy example of playing against a random agent on Limit Hold'em"""
 
 import rlcard
 from rlcard.agents import LimitholdemHumanAgent as HumanAgent
@@ -17,7 +16,7 @@
 
 print(">> Limit Hold'em random agent")
 
-while (True):
+while True:
  print(">> Start a new game")
 
  trajectories, payoffs = env.run(is_training=False)

examples/human/nolimit_holdem_human.py

@@ -1,5 +1,4 @@
-''' A toy example of playing against pretrianed AI on Leduc Hold'em
-'''
+"""A toy example of playing against pretrained AI on Leduc Hold'em"""
 from rlcard.agents import RandomAgent
 
 import rlcard
@@ -17,7 +16,7 @@
 env.set_agents([human_agent, human_agent2])
 
 
-while (True):
+while True:
  print(">> Start a new game")
 
  trajectories, payoffs = env.run(is_training=False)

examples/human/uno_human.py

@@ -1,5 +1,4 @@
-''' A toy example of playing against rule-based bot on UNO
-'''
+"""A toy example of playing against rule-based bot on UNO"""
 
 import rlcard
 from rlcard import models
@@ -16,7 +15,7 @@
 
 print(">> UNO rule model V1")
 
-while (True):
+while True:
  print(">> Start a new game")
 
  trajectories, payoffs = env.run(is_training=False)

examples/pettingzoo/run_dmc.py

@@ -1,6 +1,4 @@
-''' An example of training a Deep Monte-Carlo (DMC) Agent on PettingZoo environments
-wrapping RLCard
-'''
+"""An example of training a Deep Monte-Carlo (DMC) Agent on PettingZoo environments wrapping RLCard"""
 import os
 import argparse
 

examples/pettingzoo/run_rl.py

@@ -1,6 +1,4 @@
-''' An example of training a reinforcement learning agent on the PettingZoo 
-environments that wrap RLCard
-'''
+"""An example of training a reinforcement learning agent on the PettingZoo environments that wrap RLCard"""
 import os
 import argparse
 

examples/run_cfr.py

@@ -1,5 +1,4 @@
-''' An example of solve Leduc Hold'em with CFR (chance sampling)
-'''
+"""An example of solve Leduc Hold'em with CFR (chance sampling)"""
 import os
 import argparse
 
@@ -15,6 +14,7 @@
  plot_curve,
 )
 
+
 def train(args):
  # Make environments, CFR only supports Leduc Holdem
  env = rlcard.make(
@@ -34,7 +34,7 @@ def train(args):
  # Seed numpy, torch, random
  set_seed(args.seed)
 
- # Initilize CFR Agent
+ # Initialize CFR Agent
  agent = CFRAgent(
  env,
  os.path.join(
@@ -71,6 +71,7 @@ def train(args):
  # Plot the learning curve
  plot_curve(csv_path, fig_path, 'cfr')
 
+
 if __name__ == '__main__':
  parser = argparse.ArgumentParser("CFR example in RLCard")
  parser.add_argument(

examples/run_dmc.py

@@ -1,5 +1,4 @@
-''' An example of training a Deep Monte-Carlo (DMC) Agent on the environments in RLCard
-'''
+"""An example of training a Deep Monte-Carlo (DMC) Agent on the environments in RLCard"""
 import os
 import argparse
 
@@ -8,8 +7,8 @@
 import rlcard
 from rlcard.agents.dmc_agent import DMCTrainer
 
-def train(args):
 
+def train(args):
  # Make the environment
  env = rlcard.make(args.env)
 
@@ -29,6 +28,7 @@ def train(args):
  # Train DMC Agents
  trainer.start()
 
+
 if __name__ == '__main__':
  parser = argparse.ArgumentParser("DMC example in RLCard")
  parser.add_argument(
@@ -95,4 +95,3 @@ def train(args):
 
  os.environ["CUDA_VISIBLE_DEVICES"] = args.cuda
  train(args)
-

examples/run_random.py

@@ -1,5 +1,4 @@
-''' An example of playing randomly in RLCard
-'''
+"""An example of playing randomly in RLCard"""
 import argparse
 import pprint
 

examples/run_rl.py

@@ -1,5 +1,4 @@
-''' An example of training a reinforcement learning agent on the environments in RLCard
-'''
+"""An example of training a reinforcement learning agent on the environments in RLCard"""
 import os
 import argparse
 
@@ -16,11 +15,11 @@
  plot_curve,
 )
 
-def train(args):
 
+def train(args):
  # Check whether gpu is available
  device = get_device()
- 
+
  # Seed numpy, torch, random
  set_seed(args.seed)
 
@@ -41,7 +40,7 @@ def train(args):
  agent = DQNAgent(
  num_actions=env.num_actions,
  state_shape=env.state_shape[0],
- mlp_layers=[64,64],
+ mlp_layers=[64, 64],
  device=device,
  save_path=args.log_dir,
  save_every=args.save_every
@@ -55,8 +54,8 @@ def train(args):
  agent = NFSPAgent(
  num_actions=env.num_actions,
  state_shape=env.state_shape[0],
- hidden_layers_sizes=[64,64],
- q_mlp_layers=[64,64],
+ hidden_layers_sizes=[64, 64],
+ q_mlp_layers=[64, 64],
  device=device,
  save_path=args.log_dir,
  save_every=args.save_every
@@ -106,6 +105,7 @@ def train(args):
  torch.save(agent, save_path)
  print('Model saved in', save_path)
 
+
 if __name__ == '__main__':
  parser = argparse.ArgumentParser("DQN/NFSP example in RLCard")
  parser.add_argument(
@@ -163,13 +163,13 @@ def train(args):
  type=str,
  default='experiments/leduc_holdem_dqn_result/',
  )
- 
+
  parser.add_argument(
  "--load_checkpoint_path",
  type=str,
  default="",
  )
- 
+
  parser.add_argument(
  "--save_every",
  type=int,
@@ -179,4 +179,3 @@ def train(args):
 
  os.environ["CUDA_VISIBLE_DEVICES"] = args.cuda
  train(args)
-