To pass the next challenge, our GameAgent must defeat (or at least tie) the CS188 GSIs at three games of great skill: Tic Tac Toe, Nim, and Othello. Judicious planning and the best of heuristics will be necessary to avoid defeat.
In this project, you will get to build general game-playing agents and test them on on at least three different games: Tic Tac Toe, Nim, and Othello. Part A of the project will consist of implementing Minimax, Alpha-Beta pruning, and a graph version of Minimax. Part B will focus on implementing and testing heuristic evaluation functions, using your search algorithms from Part A.
The code for this project consists of several Python files, some of which you will need to understand in order to complete the assignment.
games.py | Generic code for defining and running games. All you need to know about this file is explained in the GameState API page. |
tictactoe.py, nim.py, othello.py | Definitions of Tic Tac Toe, Nim, and Othello in the framework defined by games.py. All you need to know about these files is explained in the GameState API page. |
util.py | Utility functions, one of which may or may not be useful to you when implementing versions of your search algorithms. |
gameagents.py | Definitions of game agents. You need to understand everything in this file, since it is where you will be adding all of your contributions. See the GameAgent API for more information. |
Important notes: Update: you make work individually or in pairs. While much of the
implemented code is
more general, you can assume that your contributions only need to work
for
2-player, zero-sum games. To introduce some variety into your
games while testing, you may want to try calling gameState.shuffledSuccessors() rather
than gameState.successors().
However, please don't call gameState.shuffledSuccessors()
in your final submitted code.
What to submit: You will edit
gameagents.py during the assignment. You
should submit it along with a
text or pdf file containing transcripts of your
code working and answers to the discussion questions, using the command "submit hw4". Directions
for submitting and setting up your account are on the course
website.
To create an initial Tic Tac Toe game state, you can type
>>> import tictactoe
>>> initialTTTState =
tictactoe.TicTacToeGameState()
To play a game against an agent that makes random moves, you can now type:
>>> import games
>>> import gameagents
>>> games.runGame(initialTTTState, {"X" :
gameagents.HumanGameAgent(),
"O" :
gameagents.RandomGameAgent()},
False, False)
At the prompt, specify your moves as (x,y) pairs. Hopefully you can win, or at least tie. If you want to quit a game early, press ctrl-c.
This runGame function is how you will be doing all of your testing. You pass it a game state, dictionary of GameAgents, and two parameters telling it how much information to print during the game (you will want to set these to True to profile your later agents).
Question 1 (2 points) In gameagents.py, the SimpleTTTGameAgent class currently describes an agent that attempts to play in the center square, or take the first available square if the center is taken. See if you you can beat it, by replacing RandomGameAgent with SimpleTTTGameAgent in the above command. Then, see how the SimpleTTTGameAgent performs against a RandomGameAgent.
Now, modify the SimpleTTTGameAgent so that it can play a decent game of Tic Tac Toe. Describe the algorithm you implemented in words, and measure its performance agains a RandomGameAgent (how many games does it win out of 10?). Your agent should be Tic Tac Toe-specific, and should not just call one of the search algorithms you implemented for a later problem.
Now that you've gotten your feet wet, it's time to write full-fledged generic game tree search agents!Question 2 (4 points)
Implement the minimax algorithm in the Minimax class
in gameagents.py. Test your code
against your SimpleTTTAgent as
follows:
>>> minimaxAgent =
gameagents.UtilityDirectedGameAgent(gameagents.Minimax())
>>>
games.runGame(initialTTTState, {"X" : minimaxAgent,
"O" :
gameagents.SimpleTTTGameAgent()},
True, True)
Your agent should be able to prove that the initial position
is a
draw (i.e., has value 0); this may take a little while (e.g., from 10
seconds to 10 minutes). Does it beat your
SimpleTTTAgent? Why or why not?
Now, try testing your minimaxAgent
against a RandomGameAgent
a few times. Does it ever lose? How often does it
win? Does it ever make moves that
seem less than ideal? If so, how can you explain
this fact? Hint: your SimpleTTTGameAgent
will probably win more often against a RandomGameAgent
than your minimaxAgent
does.
Question 3 (2 points) Implement a graph version of the minimax algorithm by filling in the GraphMinimax class in gameagents.py. Test your code against some other agents. Does it make the same choices as your Minimax agent in the same situations? How much faster does it run? How many fewer successors does it generate? Do some rough calculations to explain the difference in runtime (or lack thereof) you observe.
Question 4 (4 points) Implement tree search minimax with alpha-beta pruning in the AlphaBeta class in gameagents.py. Test your code against some other agents. Does it make the same choices as your previous two agents in the same situations? How much faster does it run? How many fewer successors does it generate? Why might you prefer this algorithm to GraphMinimax, and vice-versa?
Question 5 (1 point) Test out your search algorithms on Nim (see GameState API for a description), for a variety of different pile sizes. For example, to play against your AlphaBeta agent on a game with piles (1,2,3), run:
>>> import nim
>>> agent =
gameagents.UtilityDirectedGameAgent(gameagents.AlphaBeta())
>>>
games.runGame(nim.NimGameState([1,2,3]),
{"1" : agent,
"2" :
gameagents.HumanGameAgent()},
False, True)
How do your agents scale with varying numbers of piles and pile sizes? Can you explain the behavior you observe?
In this part of the project, you will use the search agents you wrote in part A along with new heuristic evaluation functions to improve performance for Nim, and scale to a new domain: Othello. We have streamlined this process, so you should not need to modify any of your code from Part A to complete this section.
To get started, try playing a game of Othello:
>>> import othello
>>>
games.runGame(othello.OthelloGameState(),
{"B" :
gameagents.HumanGameAgent(),
"W" :
gameagents.RandomGameAgent()},
False, False)
To make your experiments go faster, you can create a smaller
version of Othello by passing in the edge size: othello.OthelloGameState(6)
makes a 6x6 board. Just make sure the size is even.
Question 6 (3 points) Test
out your
GraphMinimax agent
from
the last section on 4x4 Othello. What is the value
of the
starting position? Can it choose a move for 8x8 Othello in a
reasonable time (< 5 min)? Now try running it with a
simple heuristic evaluation function that counts piece differences, and
a fixed search depth of 3-ply:
gameagents.FixedDepthHeuristicGameAgent(gameagents.GraphMinimax(),
>>> agent =
gameagents.pieceCountOthelloHeuristic,
3)
>>>
games.runGame(othello.OthelloGameState(),
{"B" : agent,
"W" :
gameagents.HumanGameAgent()},
True, True)
How does it perform, both in terms of search efficiency and move quality? How does changing the search depth affect its performance?
One problem with the pieceCountOthelloHeuristic
is that it values all squares equally. Fill in the edgeCornerSquareValue()
function in gameagents.py
so that it values corners and edge squares more than interior squares,
and test your function by running with the edgeCornerSquareHeuristic.
Can you find appropriate values for edge and corner squares
so
that your agent with this heuristic can consistently beat an agent
searching to the same depth with the pieceCountOthelloHeuristic?
How often can you (as a HumanGameAgent)
beat
an algorithm searching to depth 5 with this heuristic?
Question 7 (2 points) Implement
the perfectNimHeuristic
in gameagents.py,
which should return the true minimax value of a state for state.currentPlayer(),
in constant time.
Your solution need not be more than 5 lines. If you get
stuck, see here
for a big hint. Test out this heuristic with one of your
search
agents at a search depth of 1. For which initial states can
you
(as HumanGameAgent)
beat it? Now
try searching to a higher depth. Is
this better than searching to depth 1? Why or why not?
Extra Credit (2 points) Design, describe, and implement an Othello agent that can consistently beat your edgeCornerSquareHeuristic + 5-ply Alpha Beta agent, without being too much slower. You may use any techniques you wish. If there is sufficient interest, we may hold a tournament to see who can design the best agent; stay tuned for more details.