Research idea:  Gary Tesauro applied temporal difference (TD) learning to backgammon with spectacular success.  Can we apply TD learning successfully to poker?

Articles that we will read and discuss:

1. Temporal difference learning and TD-gammon, by Gerald Tesauro.
2. Practical issues in temporal difference learning, by Gerald Tesauro.
3. Learning to predict by the methods of temporal differences, by Richard Sutton.
4. Using probabilistic knowledge and simulation to play poker, by Darse Billings, Lourdes Pena, Jonathan Schaeffer and Duane Szafron.
5. Using selective-sampling simulations in poker, by Darse Billings, Denis Papp, Lourdes Pena, Jonathan Schaeffer and Duane Szafron.
6. Opponent modeling in poker, by Darse Billings, Denis Papp, Jonathan Schaeffer and Duane Szafron.
7. Representations and solutions for game-theoretic problems, by Daphne Koller and Avi Pfeffer.

Schedule:  Each section has two groups.  Each group meets twice a week.
Group 1 will meet on Monday & Thursday, except as noted.
Group 2 will meet on Tuesday & Friday, except at noted.

1. Seminar overview (all, Monday).
2. TD learning (Tesauro:  backgammon), presentation.
    
3. TD learning (Tesauro:  backgammon), presentation and discussion.
4. Poker (Schaeffer et al:  Loki), presentation and discussion.
    
5. Poker (Koller & Pfeffer:  Gala), presentation and discussion.
6. Summary discussion.  Begin developing the project details.
    
7. Complete development of project details.
8. Team meeting (work on project).
    
9. Status report.
10. Status report.
     
11. Status report.
12. Final reports (oral presentation)
    (all, Thursday and Friday, both 8th and 9th periods where possible).

Paper presentations:  the team of six will be divided into three pairs.  One pair will present TD learning, one Poker (Loki) and one Poker (Gala).

Final reports:  the team of six will be divided into two trios.  One trio will deliver an oral presenation, while the other delivers a written presentation.