IMGD 2905 Project 4

Drift and Sounder* Analytics

Due date: Saturday, April 20th, 11:59pm

(* A group of young pigs is a "drift", and a group of older pigs is a "sounder".)

The goal of this project is to analyze some simple games of chance using theoretical and empirical methods, and exposure to basic simulation as a tool. You start with a simple coin-flip game, progressing on to single-die and multi-dice games. For each, you analyze elementary and compound probability events and run repeated games with different game strategies, recording the outcome. You will modify basic scripts that are provided to provide data needed for analysis, using analytics skills to create statistics and charts, with appropriate explanatory text. Results are presented in a report.

Part 1 - Piglet

Piglet is a one player game where the goal is to get 10 points in a few turns as possible.

Rules

• Each turn, the pot starts empty
  • Player choice: go or stay
  • If stay, player adds pot to points and turn ends
  • If go, player flips coin:
    • Tails busts and the pot empties and the turn ends
    • Heads add 2 to pot and go back to the choice
• Get to 10 points to win
• A lower number of turns is better

Find a coin and play Piglet (with a friend) to get a feel for the game.

Questions

Answer the following questions:

1. What is the probability a player wins on his/her first turn?
2. What is the expected number of points for the first flip? Hint: The expected value is what you could expect in the "long run", after many trials, obtained by multiplying each event by its probability. Consider the number of points if successful (heads), the number of points if bust (tails) and the likelihood of each.
3. Complete the following table for 10 rows, and also filling in all the "Expected Points" columns.

      Flips  Chance Not Busting  Pot    Expected Points
      1      0.5                 2      ???
      2      0.5^2               4      ???
      3      0.5^3               6      ???
      ...

4. If a player flips a coin only once and then always passes, after 20 turns what is the probability that s/he has 4 or fewer points? Hint: think of the kind of distribution this is and try out a formula.

Experiments

Copy and paste the below Python script (into a file or a Jupyter notebook) and run it. Study what it does. Run it some more until you mostly understand it.

#
# Piglet - a single-player coin-flip game.
# v1.0
#

import random

# Game settings.
WIN = 10 # Number of points to win one game
OUTCOMES = 2 # Number of outcomes (e.g., 2 for coin, 6 for d6)
NUM_GAMES = 1 # Number of games to play

#######################################
# Pick random integer from 1 to OUTCOMES.
# Add number to pot.
# If pick 1, pot empties.
# Return pot.
def go(pot):

  number = random.randint(1,OUTCOMES)

  if number == 1:
    print("Bust!")
    pot = 0

  else:
    print(number)
    pot += number

  return pot

#######################################
# Players - Return True if go, False if stay.

# Human has choice.
def HUMAN() :
  while True:
    choice = input()
    if choice == '1':
      return True
    if choice == '2':
      return False

# Brave bot always goes.
def BRAVE_BOT(points, pot) :
  if points + pot > WIN:
    return False
  return True

# Timid bot always stays.
def TIMID_BOT(points, pot) :
  if pot == 0:
    return True
  return False

#######################################
# Play one game.
# Return number of turns it took to win.
def play():
  turn, pot, points = 1, 0, 0

  while points < WIN :

    print("Turn:", turn, " ", end="")
    print("Points:", points, " ", end="")
    print("Pot:", pot, " ", end="")
    print(" 1-Go or 2-Stay? ", end="")

    ## Uncomment one of the below players.
    choice = HUMAN()
    #choice = TIMID_BOT(points, pot) 
    #choice = BRAVE_BOT(points, pot)

    if choice == False:
      print("STAY")
      points += pot
      turn += 1
      pot = 0
      continue

    if choice == True:
      print("GO: ", end="")
      pot = go(pot)
      if pot == 0: # Bust
        turn += 1
      continue

  return turn-1

########################################

# Play game(s).
game = []
for x in range(NUM_GAMES):
  print("------------------")
  print("Playing game:", x)
  game.append(play())

# Print results.
print("------------------------------------")
for x in range(NUM_GAMES):
  print("Game", x, "turns:", game[x])

Experiments

The game defaults to being played with a person (HUMAN). However, there are two different bots (TIMID_BOT or BRAVE_BOT) that can play the game instead. Playing the game with a bot provides for a way of simulating gameplay without having to do the same testing with people.

Query: Which bot is better, TIMID_BOT or BRAVE_BOT?

• Run experiments that gather data needed to answer this query.
• Analyze the results, draw charts, providing statistics and describing the analysis.
• Provide a conclusion that answers the query from the data gathered.

Part 2 - Pig

Pig is like Piglet, but instead of flipping a coin, players roll one 6-sided die.

Rules

• Each turn, the pot starts empty
  • Player choice: go or stay
  • If stay, player adds pot to points and turn ends
  • If go, player rolls die:
    • 1 busts and the pot empties and the turn ends
    • 2-6 add number of pips to pot and go back to the choice
• Get to 50 points to win
• A lower number of turns is better

Find a die and play Pig (with a friend) to get a feel for the game.

Questions

Answer the following questions:

1. What is the probability of not busting on a single roll?
2. What is the average pot if the player does not bust on single roll?
3. How many ways are there to get exactly 10 points on one turn? (Optional - what is the probability of doing so?)
4. What is the expected number of points a player gets for a single roll?
5. Draw a chart depicting the expected number of points versus number of consecutive rolls. Hint: refer to the table for Question #3 in Part 1. Explain how the chart was computed and interpret the results.

Experiments

Modify the Piglet Python script from Part 1 to play Pig.

Query: Which bot is better, TIMID_BOT or BRAVE_BOT?

• Run experiments that gather data needed to answer this query.
• Analyze the results, draw charts, providing statistics and describing the analysis.
• Provide a conclusion that answers the query from the data gathered.

Develop a BETTER_BOT, one that wins in fewer rolls than either TIMID_BOT or BRAVE_BOT.

• Once developed, describe the logic behind your bot.
• Run experiments that gather data needed to evaluate your BETTER_BOT compared to the other two bots.
• Draw charts, provide statistics and analyze the data to support your comparison.

Part 3 - Swine

Swine is like Pig, but instead of rolling one die, players roll two 6-sided dice.

Rules

• Each turn, the pot starts empty
  • Player choice: go or stay
  • If stay, player adds pot to points and turn ends
  • If go, player rolls 2 dice:
    • "Snake eyes" (two 1's) adds 25 points to pot
    • Any other doubles busts and the pot empties and the turn ends
    • Any other combination, add sum of pips to pot and go back to the choice
• Get to 100 points to win
• A lower number of turns is better

Find two dice and play Swine (with a friend) to get a feel for the game.

Questions

1. What is the probability of not rolling any doubles with a single roll (2 dice)?
2. What is the probability of rolling snake eyes with a single roll?
3. What is the probability of not busting with a single roll?
4. What is the average score obtained from a single roll if the player doesn't bust? Hint: Break the computation into two parts, weighting each part by the probability.
5. Draw a chart depicting the expected number of points versus number of consecutive rolls. Hint: refer to the table for Question #5 in Part 2. Explain how the chart was computed and interpret the results.

Experiments

Modify the Pig Python script from Part 2 to play Swine. Compare performance of TIMID_BOT, BRAVE_BOT and BETTER_BOT.

Query: Which bot is best?

• Run experiments that gather data needed to answer this query.
• Analyze the results, draw charts, providing statistics and describing the analysis.
• Provide a conclusion that answers the query from the data gathered.

Bonus

Provide Python code for a BEST_BOT - a Bot that wins in the fewest number of turns on average.

Describe the logic behind your BEST_BOT.

Provide experimental results as per above that show BEST_BOT performance.

Part 4 - User Study

Play

Participate in a user study.

There are two options - you must participate in one (but it would be appreciated by your fellow students if you did both):

https://docs.google.com/spreadsheets/d/1uvQeRNc4Iah9PD4jole-IQIcUGFbOjMhO_4UaoKIms0/edit#gid=0

https://docs.google.com/spreadsheets/d/1TJNv_zGQoBiRYW6hTbcbWJsgFc8U3l3YW-HgmLV5YwA/edit

Grab a timeslot and participate in the study.

Data

Data is provided here periodically. Here are the cumulative data sets:

• April 16, 2019
• April 19, 2019

Download the latest data set.

Analyze

Use Lag as the independent variable. Analyze a dependent variable for some or all of the data (e.g., Time Survived versus Lag) for some or all of the data. Draw an appropriate chart, with a brief interpretation.

Note, there are two kinds of lag compensation algorithms being tested in the Is Using Lag Compensation column. This may matter for the analysis.

Note: for this part of your report, you do not need to describe the user study methodology. However, you should provide a few summary statistics (e.g., number of games/players) about the data set you analyze.

Writeup

Writeup a single, short report for all three sections.

Include appropriate descriptions of your methodology, avoid redundancy by separating out methodology that is common to each section.

Be sure to include responses to all required questions, indicating how you came up with your answers.

Be sure to include conclusions on all queries, providing appropriate analysis of the game data. You do not need to include your complete code listing. However, you should include the BOT functions you have written.

All dissemination guidelines for presenting and describing charts should be adhered to.

Submission

The assignment is to be submitted electronically via Canvas by 11:59pm on the day due.

The submission is a report in PDF, named:

    proj4-lastname.pdf

with your name in place of "lastname" above, of course.

To submit your assignment (proj4-lastname.pdf):

Open: IMGD2905-D19-D01
Navigate to: Assignments -> Project 4
Click: Submit Assignment
Click: Choose File
Select the pdf file: proj4-lastname.pdf
Click: Submit Assignment

Important - you must click the Submit Assignment button at the end or your file will not be submitted!

When successfully submitted, you should see a message similar to:

Submission
- Submitted!
Apr 20 at 10:50pm

Bonus - if you did the bonus part (BEST_BOT) in Part 3, in addition to including the code in your report, email professor Claypool just your bot code so he can run it.

Grading

All accomplishments are shown through the report. The point break down does not necessarily reflect effort or time on task. Rather, the scale is graduated to provide for increasingly more effort required for the same reward (points).

Breakdown

Part 1 (Piglet) - 35% : Analysis of the Piglet game.

Part 2 (Pig) - 30% : Analysis of the Pig game.

Part 3 (Swine) - 25% : Analysis of the Swine game.

Part 4 (User Study) - 10% : Participating in the User Study.

Bonus extra 1% : Providing a working, optimal BEST_BOT for Swine.

Rubric

100-90. The submission clearly exceeds requirements. All Parts of the project have been completed or nearly completed. The report is clearly organized and well-written, with questions answered correctly with work shown. All charts and tables are clearly labeled and described, with measures of central tendency and spread properly computed and explained.

89-80. The submission meets requirements. Parts 1-4 of the project have been completed or nearly completed. The report is organized and well-written, with questions answered mostly correctly and with most work shown. All charts and tables are labeled and described, with measures of central tendency and spread properly computed and explained.

79-70. The submission barely meets requirements. Parts 1-2 of the project have been completed or nearly completed, and some of Part 3, and maybe not Part 4. The report is semi-organized and semi-well-written, but some question answers are incorrect and/or work is not shown. Charts and tables are mostly labeled and described, but parts may be missing or unclear. Measures of central tendency and spread may not be always computed or adequately explained.

69-60. The project fails to meet requirements in some places. Part 1 of the project has been completed or nearly completed, and some of Part 2, but not Parts 3 or 4. The report is not well-organized nor well-written, charts and tables are not labeled or may be missing. Many question answers are incorrect and/or work is not shown. Measures of central tendency and spread may not be always computed or explained or may even be misused. Messages are not always provided for the analysis.

59-0. The project does not meet requirements. Many parts of the project are incomplete or poorly done. The report is not well-organized nor well-written, charts and tables are not labeled and/or are missing. Many answers to questions are missing or incorrect with work not shown. Measures of central tendency and spread are missing of, if in place, are misused.

Postmortem Feedback on Graded Projects

The comments below are in response to graded projects. They are not provided in any particular order.

• Most of the probability analysis was correctly done and clearly explained. Well done.

• The basic expected value tables and charts for the 3 games were correctly interpreted. This provided reasoning for making a "Better Bot", most of which worked quite well.

• Remember, inferring behavior (e.g., effectiveness of a Bot or play strategy) is best done over many trials. At least 30, then the central limit theorem applies and statistical significance can be determined. So, have a sample size of 30+ if possible (for this project, run at least 30 games for a given bot).

• Also remember, that the standard error (and confidence intervals) get smaller with an increasing sample size (n). So, if possible, run more games than 30. For this project, it is pretty "cheap" to run 1000 games for each bot. That's an advantage of automated game playing - it is relatively easy to collect data compared to a users study.

• Having a chart with the game number on the x-axis is not particularly useful. There is no dependence between games, so the x-axis value has no meaning. Yes, the games could be ordered high to low, in an attempt to show a distribution. But even here, while it could be useful to see spread across games, there are better ways of doing this (e.g., histogram or CDF).

game-number

• (Almost) anytime you have a mean value from a sample, you must show the spread! Remember, you are using the sample mean to try and infer a population parameter. Typically, you should show the spread with error bars. Most of the time, the error bars should represent that standard error. You should indicate what they represent in the description.

No error bars

• If you have numeric variables (e.g., lag in part 4), these typically are not categorical. This means representation is often better done without a bar chart (probably using a line chart). Also, numeric variables should be space proportionally to their values in the axis (e.g., proportional spacing for each).

Lag not categorical

• If you use histograms to show a distribution (say, of turns taken for a bot) and want to compare bots (say, timid bot versus brave bot), do not have the histograms on separate pages. It is too hard on the reader to try to compare visually this way. Better, consider doing your comparison with a CDF or box-and-whiskers plots.

Return to the IMGD 2905 home page