CS/IMGD 411x Project 4

Most multiplayer network games are played over the Internet, where dealing with latency is a reality. In this project, you will gain experience with gameplay and Internet latency, build techniques that compensate for latency in a multiplayer Internet game, and briefly assess those techniques.

Details

You will add features to your Fruit Ninjas game to experience and then compensate for network latency.

Simulating Latency

You will add artificial latency to network messages sent from server to client or client to server in order to simulate latency. This will enable players to experience high latency gameplay without necessarily finding network conditions with the same.

There are many ways to add artificial latency to client-server communication. However, the suggestion is to add latency when messages are sent in order for the game code to be mostly unchanged when simulating network latency.

Tip: To simulate symmetric network latencies (i.e., latency from the client to the server is the same as latency from the server and the client), latency can be added on both outgoing links (i.e., add latency at the client when sending to the server and add the same latency at the server when sending to the client).

Built-In

If you use the built-in NetworkManager code in Dragonfly, latency can be added to outgoing network messages by setting the delay for the socket:

  // Set delay amount added to outgoing messages (in ticks).
  // If sock_index is -1, apply to all connected sockets.
  // Return 0 if ok, else -1.
  int setDelay(int delay, int sock_index);

This method adds at least delay ticks of latency to all outgoing messages on the socket specified by sock_index. As an example, a Server may want to add delay to all outgoing messages to a Client after an initial connection.

  int Server::handleAccept(const df::EventNetwork *p_en) {

    int sock_index = p_en -> getSocketIndex();

    // Set delay for all outgoing messages.
    NM.setDelay(DELAY, sock_index);

    return 1;
  }

The units for the delay parameter are in ticks of the game loop. Since Dragonfly defaults to 30 Hz, this means each tick is about 33 ms of delay (e.g., 2 ticks of delay results in about 66 ms of delay for each outgoing message). Note, all messages are delivered and message sending order is still preserved.

Custom

If you use your own NetworkManager code (e.g., as implemented in Project 2), the suggestion is to intercept outgoing messages before they are sent (i.e., in the NetworkManager::send() method). Then, if there are artificial delays, the messages are not sent but instead queued, storing all the parameters so the message can be sent later, and include the time to send it.

  // Keep track of delay to add to each socket index.
  std::vector<int> m_delay;  

  // Define delayed message as tuple: when, message, bytes.
  using delayedMessage = std::tuple<int, char *, size_t>;

  // Define delayed queue as queue of messages tuples.
  using delayQueue = std::queue<delayedMessage>;

  // Keep track of delayed messages per socket index.
  std::vector<delayQueue> m_delay_q;

For the latencies, actual wall clock time (i.e., system time) can be used, but it is easier to use virtual time (i.e., game loop time). In Dragonfly, virtual time (in ticks of the game loop) can be obtained via GM.setStepCount().

For a new connection (via connect() or accept()), there would be no delay (the default) and an empty queue.

  // New connection.
  m_delay.push_back(0);
  delayQueue q;
  m_delay_q.push_back(q);

  // Add to queue for later sending.
  int delay = GM.getStepCount() + m_delay[sock_index];
  m_delay_q[sock_index].emplace(delay, buffer, bytes);

Then, each tick, the Sentry (see Project 2 would see if it is time to send the first message in the queue (the first message is always the next to send):

  // Check message in front for time.
  delayedMessage message = m_delay_q[sock_index].front();
  int time = std::get<0>(message);
  if (time <= GM.getStepCount()) { // Time to send?
     char *buffer = std::get<1>(message);
     int bytes = std::get<2>(message);
     sendNow(buffer, bytes, sock_index);
  }

External

Network latency can be simulated outside of game code and game engine code in several ways. At the end hosts (e.g., in the operating system) there are software tools that simulate latency (and other network conditions). There are many commercial tools (i.e., you have to pay for them), but some free and open source tools as well. The exact tools that can work depend upon the platform.

Note, however, it can be less convenient for testing to have to change and re-configure an external artificial latency system rather than doing so in your own game and game engine.

Try it out!

Test out your Fruit Ninjas game, even with just one player, where a player has high latency. Try a bit of added latency, say 100 ms, and see if you notice. Then try 250, 500 ms and even 750 ms of latency and see how it feels (and you definitely should feel it). For two players, you can configure your game so that the players can have different amounts of latency. For example, the first player that connects gets 250 ms of added latency, but the second player does not.

Ping

A common method of helping the player deal with network latency is to show it to them, literally displaying the network latency as a number on the screen. For example, the below screenshot shows a ping display with the latency value (here, in milliseconds) in the upper right corner of the screen:

You will add this functionality to your Fruit Ninjas game. Basically, this requires two components: 1) code that measures the round-trip time for messages sent from the client to the server and back, and 2) code that displays the measured values.

Tip: Developing Ping in parallel with custom simulated network latency can be helpful for verifying the simulated latency code is working.

Measuring Latency

The latest (most recent) measured latency value is sometimes known as the "instant" latency and is sufficient for this project. However, beyond instant latency, sometimes a window of time is used and the average or median latency computed over this window in order to: 1) allow latency to adjust over time, but also 2) smooth out latency spikes.

For Fruit Ninjas, the client should send a custom "ping" message based on game ticks (e.g., every 10-15 ticks of the game loop, send out a "ping"), where the message data only has the current time. The server gets this packet, then immediately echoes (sends) it back to the client. The client, upon receiving the echoed message, extracts the time and determines how long it took (i.e., the difference between "now" and the original time). That time is the ping time.

Note, if using the built-in Dragonfly NetworkManager, a df::NETWORK_CUSTOM_EVENT is generated when a NetworkNode (e.g., the Server or Client) gets a custom message. The message body would then be parsed to: 1) determine it is a "ping" packet, and 2) get the time.

Note, the ping value here (and in most network games) includes more time than just the network latency. It also includes all the software processing by the OS, game engine and game code on both the client and server in addition to any network latency.

Displaying Latency

The easiest way to display latency values is using a ViewObject in a fashion similar to that for the Points in Fruit Ninja. This means creating a Ping class derived from ViewObject and setting the location and view string. The color of the display can be changed based on the reported latency value: under 100 ms is green, 100 to 300 ms is yellow, and greater than 300 is red. This is most easily done by overriding the Ping setValue() method and changing the color based on getValue(). Important! Make sure to call the parent setValue() method (i.e., ViewObject::setValue()) or the display numbers won't change.

Note, the latency measurements likely use game tics as their units. Players have an easier time with "wall clock" units, like milliseconds. Game tics can be converted to wall clock units by multiplying by the frame time (i.e., the milliseconds per frame). In Dragonfly, this is available in the GameManager getFrameTime() method. For example, if an event happened at virtual time X ticks (i.e., the game loop has iterated X times), the latency between then and now in milliseconds would be:

  int event_time = X;
  int latency_ticks = GM.getStepCount() - time;
  int latency_milliseconds = latency_ticks * GM.getFrameTime();

Ping Summary

Try it out!

Test our the Ping display for no added latency. When run on localhost, the latency without any added delay is the base latency of the client-server system. Test out your Ping display by using your artificial latency for 50, 250, and 500 ms of latency. You should see the artifical latency plus the base latency in the Ping display.

Sword Prediction

You will implement a self-prediction latency compensation technique. Specifically, the client will act upon the mouse movement and move the sword without waiting for confirmation of the action from the server.

The server is still the authority on all final positions and game state resolutions. So, in this case, the client still sends the mouse movement coordinates to the server, just as in the basic, non-compensated Fruit Ninjas game. However, the client itself moves the sword (and draws a trail), too, in response to changes in the mouse position instead of waiting for the server to move and then synchronize the sword.

This means that the client's Sword needs to be differentiated from the Swords controlled by other clients. This can be done by:

The server no longer synchronizes Sword position changes with all clients. Specifically, when the Server gets the mouse movement position in a message, it changes the corresponding Sword position normally. The next tick, since the Sword position has changed (i.e., p_o -> isModified(df::ObjectAttribute::POSITION) is true) the Sword is synchronized (SYNC_OBJECT) with all clients except the client that controls that Sword.

Note, the client moves its own Sword but does not determine if a fruit is sliced. That functionality still resides with the server. This means sword movement will appear immediate to the player that moved their mouse - even with high amounts of network latency - but any slicing actions are still delayed.

Tip: For evaluation, you will need to play your game with and without sword prediction. This can be done by building different versions, setting up code in #ifdef guards for code that only runs when prediction is enabled versus code that only runs when prediction is not enabled. e.g.,

#ifdef PREDICT_SWORD  
  // Client moves Sword based on mouse event.
  ...
#endif

Try it out!

Now, when there is added artificial latency (say, 500 ms and verified with your ping display), the Sword should still be responsive (as responsive as for 0 ms of latency) for a client, moving with the mouse as if there is no latency. Fruit slicing will still look delayed (i.e., the exploding fruit happens later than when the mouse appears to go through it), especially noticeable at high latency.

Fruit Prediction

If you try out the implementation so far with sword prediction in place, the sword movement feels responsive even with high network latency. But slicing will still be hard because the player has to slice where the fruit is not. Specifically, the position of the Fruit objects on the server will be ahead of that on the client by the latency amount and the velocity. This means what looks like an effective slice on the client right through a fruit, may be a miss since the fruit has moved past that point by the time the message reaches the server. This is especially noticeable for small, fast fruit.

To help with this, you will implement an extrapolation latency compensation technique. Specifically, the client can predict the position of fruit based on the measured latency. For example, if a Fruit is at (10.0, 10.0) with a velocity of (0.25, 0.5) (i.e., it moves 0.25 spaces to the right and 0.5 spaces down each tick) and the latency is 2 ticks, the client can predict the Fruit to be at (10.5, 11.0) (i.e., apply the velocity to the position twice, once for each tick).

Since the client is already predicting Fruit positions based on the starting position (i.e., computing new positions based on velocity), the prediction position with latency can be done by modifying the start position on the client when it is spawned based on measured latency.

To implement this, the latency measured via "ping" messages should be saved in the client (e.g., in a Client attribute, say m_latency). Then, when a Fruit is created, after synchronizing, the starting position is adjusted by adding the velocity to the current position once for each "tick" of latency. This can be done manually or via the Object predictPosition() method as in the below code example:

  // Adjust position for each tick of latency.
  for (int i=0; i < m_latency; i++) {
    df::Vector new_pos = p_o->predictPosition();
    WM.moveObject(p_o, new_pos);
  }

In order to perform an action when an Object is created based on network synchronization, in Dragonfly a game object (e.g., the Client) should register for interest in a df::NETWORK_CREATE_EVENT:

  // Client handles create, adjusting Fruit position based on lag.
  registerInterest(df::NETWORK_CREATE_EVENT);

  if (p_e -> getType() == df::NETWORK_CREATE_EVENT)
    // Adjust fruit position based on measured latency

Try it out!

With fruit prediction enabled and high latency (e.g., 400 ms), the sword should no longer need to "lead" the fruit by slicing ahead of it. Instead, the sword should be able to slice through the fruit where it appears on the screen. The slice itself, however, will still be delayed.

Evaluation

Important! Up to here, the project should be done solo. But for this part of the project (i.e., for the evaluation) you must do this in a group of 2 (or even a group of 3). That means collaborating on at least running the experiments. In fact, everyone can collaborate on the report results and analysis, too (see below) and turn in the same document.

Upon building artificial latency and latency compensation, you will evaluate your Fruit Ninjas, assessing player performance and experience.

You first do scenarios (A)-(D) above with one-player (i.e., a single player game) each player using their own game. Then, you assess two player games where one player never has added latency. You can use either player's game for the multi-player experiments, but use the same game for all scenarios and conditions.

For (B)-(D) you can test with only one latency condition -- try to pick a latency high enough that it makes an impact (i.e., you should definitely feel it with no latency compensation) -- and use the same latency across scenarios so you can make comparisons. You should do several (e.g., 3) runs for each test condition.

Player performance can be assessed by their scores. Player experience can be assessed by a simple MOS-type question after the game.

Tip: the score for each player can be written to the logfile (or a separate file) for ease of analysis by the server when the game terminates. The built-in LogManager method writeMyLog() can be used to write to a custom logfile (e.g., one with player data only).

Your evaluation will culminate in a brief (1-2 page) write-up with the following sections:

Results and Analysis - Depict your results clearly using a series of tables or graphs. Provide statistical analysis where appropriate (e.g., average). Interpret the results. Briefly describe what the results mean, including the effects of latency and potential benefits of latency compensation.

Be sure to include a title and the names of the students in the group. Each person should turn in the report as part of their own assignment. It can be the same report -- that's not a problem -- as long as the names for all are listed.

Submission

Your assignment is to be submitted electronically (via Canvas) on the time and day due. You must hand in the following:

This will remove all the temporary (compiled) files. Failure to do may mean you file is too big to submit!

Important - you must click the Submit Assignment button at the end or your file will not be submitted!

Grading Guidelines

Breakdown

Simulating Latency - 5% : Adding the ability to simulate network latency demonstrates a game systems-type feature in game code and/or game engine code is worth about half a letter grade. Demonstrating control of artificial latency outside of code is also possible and demonstrates learning a third-party system.

Ping - 20% : Adding round-trip client-server latency measurements and then displaying latency on the screen shows you can implement a widely-used latency compensation technique (exposure), as well as some understanding of the game engine and network programming. Thus, it's worth about one fifth of the grade.

Sword Prediction - 25% : Acting upon mouse movement immediately and moving the Sword for a client, but still synchronizing them across all players demonstrates mastery of a self-prediction technique, worth one-quarter of the grade.

Fruit Prediction - 25% : Adjusting the Fruit position in response to the measured latency upon client spawn demonstrates mastery of another self-prediction technique, worth one-quarter of the grade.

Evaluation - 20% : the design and execution of experiments and analysis in a report that describes and illustrates performance with latency is worth a fifth of the grade. The experience with latency and latency compensation with different network latencies is the important part, and the report itself is the means of showing this experience.

Documentation - 5% : Not to be overlooked is including the documentation provided, as well as having that documentation be clear, readable and pertinent to the assignment. This primarily includes the README described above but also any needed supporting information. Having well-structured, readable and commented code is part of Documentation, too. Getting in the habit of good documentation is important for large software projects, especially when done in teams.

Note, bugs and/or code that has not been tested can result in points taken off for any of the areas above.

Rubric

100-90. The submission clearly exceeds requirements. The game can be setup to be played with artificial latency. The ping display shows the latency, colored according to severity. The sword's movement is self-predicted, being responsive even with latency. The fruit movement is also self-predicted, making it easier to slice. The evaluation report clearly and concisely describes the design, results and analysis. Documentation is thorough and clear and code is well-structured, commented and readable.

89-80. The submission clearly meets requirements. The game can be setup to be played with artificial latency. The ping display shows the latency, mostly colored according to severity. The sword's movement is self-predicted, being responsive even with latency and mostly works. The fruit movement is also self-predicted, generally working well and making it easier to slice. The evaluation report describes the design, results and analysis. Documentation is clear and code is well-structured and readable.

79-70. The submission barely meets requirements. The game can be setup to be played with artificial latency. The ping display mostly works and shows the latency, but may be lacking color. The sword's movement and/or the fruit movement predictions are both attempted, but may not fully work. The evaluation report has a design, results and analysis. Documentation is intact and code is well-structured and readable.

69-60. The project fails to meet requirements in key places. The game can be setup to be played with artificial latency with difficulty. The ping display may not completely work and/or lack color. Either the sword's movement and/or the fruit movement predictions are not working completely. The evaluation report is incomplete and/or missing large parts. Documentation is intact but code readability and structure needs work.

59-0. The project does not meet many of the key requirements. The game cannot be setup to be played with artificial latency easily. The ping display may not work and lacks color. Both the sword's movement and/or the fruit movement predictions are not working completely. The evaluation report is quite incomplete or completely missing. Documentation is sparse or missing and code difficult to read and/or poorly structured.

Latency Compensation

Overview

Details

Simulating Latency

Built-In

Custom

External

Try it out!

Ping

Measuring Latency

Displaying Latency

Ping Summary

Try it out!

Sword Prediction

Try it out!

Fruit Prediction

Try it out!

Evaluation

Submission

Grading Guidelines

Breakdown

Rubric