What makes a successful system for scoring Diplomacy?

This post was inspired by a post on the Diplomacy Reddit. The post was entitled Binary Scoring System. As I was thinking about a comment on the system itself, I thought I may as well write a blog post about it. And then I realised it would make for a decent example on system for examining what a good system for scoring Diplomacy would be.

Let’s have a look at how to examine a system first. I think there, perhaps, five questions to use when looking at this:

1. What is the system designed to do? In other words, in what context is the scoring system operating?
2. Is the system effective in differentiating between players and results? Does it, within the context, meaningfully rank players?
3. What are the objectives of the system? This is about considering what types of play the system seeks to reward. Does it affect the way games are played?
4. Are the objectives consistent with the design of the Diplomacy? As the system will impact on how competitors play games, is the style of play consistent with the way the game was designed to be played?
5. Overall, is the system a ‘good’ system? This is subjective in that it will reflect your idea about how Diplomacy should be played, but it is also objective in that it will reflect about whether it is successful in meeting its objectives.

Let’s have a look at each question in turn and use the Binary Scoring System as an example. There are better examples, and I’ll do further posts on this, but for now, remember, I’m responding to a specific post. First, though…

What is the Binary Scoring System?

Pretty simple, really. The systems scores 1 point for a win, 0 points for anything else. This is, simply, a Unusist system where only the solo is a meaningful result. There are distinct problems with this as far as playing Diplomacy is concerned, as we’ll see below, but if that is the goal of the system, then it’s effective.

The biggest example of this scoring system was the Diplomacy World Cup. I actually played in the ‘pool stages’ (qualifying) for this tournament in the second tournament and realised that the games were poor quality because of the scoring system. There is a current version of the tournament which is hosted on webDip but it isn’t the same tournament at all, in reality. That’s not a comment on the authenticity of the tournament, just a note.

1. What is the scoring system designed to do?

This is important. What kind of play is the system aiming to encourage? A system, such a Supply Centre Scoring system encourages players to amass SCs; a Draw-Based Scoring system encourages players to play for a win or a draw.

An SCS system is designed to reward SC count. This isn’t what Diplomacy is about. It’s use as a tournament scoring system is more understandable (see below) but not as a ratings system (something used for scoring a long series of games). SC count doesn’t matter in a regular, one-off game of Diplomacy and there’s no reason it should matter over a series of games.

A DBS system is designed to reward game results. A solo scores more than a 2-way draw, which scores more than a 3-way draw, etc. This is more in line with the way Calhamer designed the game, as can be seen by reading the rules, and in line with the scoring system Calhamer designed.

With the Binary Scoring system, the aim is to reward solos only. This is even more problematic. While it encourages playing to win, it ignores the draw, which is a secondary objective and, very often in a tournament, a decent (if not good) result.

2. Is the system effective in differentiating between players/results?

An SCS system will mean that there is more differentiation between results. Because games are scored on SC count, rather than on the outcome of the game, it means that scores in a 4-way draw, for instance, will vary. You can be part of a 4-way draw on 17 SCs or on 1! Obviously, the player who controlled 17 SCs at the end of the game will be clearly differentiated from the player holding on by the skin of their teeth!

A DBS system doesn’t achieve this: a 4-way draw is a 4-way draw – everyone scores the same result. This means that there will be a number of tied scores over a tournament, although this differentiation will grow over a longer series of games. DBS is better as a ratings system, then, rather than a tournament system. In a tournament, you’re likely to need to secondary scoring system to effectively differentiate.

The Binary System is even worse at differentiating. Although a solo is scored, nothing else is. This effectively promotes playing for a win but ignores that, certainly in a tournament, solos will be few and far between. The majority of players will end on 0 points. And, even when you solo, there is no way of differentiating between solos unless players solo more than once. For tournaments, this is ineffective; for a ratings system, it is less effective than a DBS system.

3. What are the objectives when playing to the system?

Under SCS systems, the objectives are to solo, then to draw. However, the objective in a draw is to amass Supply Centres and, as a lot of games (especially in a tournament) will be draws, this becomes the key objective. The idea is that it encourages more dynamic play, with players taking risks to increase their scoring potential.

With a DBS system, the objectives are to solo, then to draw. The objective is not to grab SCs but, if you can’t win, the overriding objective is to prevent another player from winning. It’s argued that DBS scoring therefore encourages negative play – just be in the draw. This is no more or less the case than under SCS. However, it will encourage less risk-taking because that risk – grabbing an SC or two more – might well see your survivability threatened.

The Binary Scoring system is only about winning. If you can’t win, there’s no reason to play for anything else. You may as well drop from the game. This is completely against the way the game was designed – if you can’t win, prevent someone else from winning. While this isn’t completely negated under the Binary Scoring system, as preventing a win is still preferable to losing because it prevents another player from scoring at all, it means you don’t have a chance to score – there’s nothing in it for you! It could encourage more cutthroat play, of course, in the pursuit of victory.

4. Are the objectives consistent with the design of Diplomacy?

Diplomacy has two objectives: win or draw. The solo is the primary objective but, if you can’t solo, you play to survive into the draw and to prevent someone else from winning. At the end of the game, the number of SCs you have only matters if you own 18 or more SCs, or if you’ve been eliminated. That’s it.

The SCS system is therefore not in line with the design of Diplomacy. It doesn’t matter how many SCs you have at the end of the game. A draw is a draw. That’s it.

There is a caveat to this. For a number of years, the rules suggested an alternative way to play, known as the Short Game rules. This had it that, before play started, the players should agree to end the game at a certain point. At this point, the player with the most SCs was declared the winner.

This is where SCS scoring comes into consideration. It is common with online tournaments especially for games to end at a certain point – the Game End Date. It would fit in with this approach to use an SCS system to score the game, in line with the Short Game rules. However, there is no indication anywhere that this was in Calhamer’s mind. This rule was in place when he wrote his article “Objectives other than Winning” and, here, he completely ignores the rule, stating that the draw is the only other objective in the rules.

DBS scoring is in line with Calhamer’s design. Game outcomes are a win, a draw or a loss. Nothing about SC scoring, as I’ve said. In fact, the Calhamer Point system is based on DBS: one point for a win, 1/n for a draw, where n is the number of players in the draw.

Perhaps the way the DBS system is utilised is not quite as Calhamerian as it could be. What can happen is that players agree to draw, rather than try to win. This may lead to draw-whittling, the removal of smaller powers to increase the number of points available in a draw. Playing for draw is not without its risks, of course.

The Binary Scoring system is consistent only as far as winning goes. Because a draw is worth nothing, it makes the draw a negative result. This certainly isn’t something Calhamer designed into the game. It’s a lesser result than a win, but it’s a better result than a defeat. A scoring system that doesn’t differentiate between a loss and a draw is far from Calhamer’s design.

5. Overall, is the system a ‘good’ system?

It’s impossible, really, to judge SCS and DBS systems in general with any accuracy. Each system within these categories should really be judged on it’s own merits. However, in general…

SCS systems allow for better differentiation between players in a tournament without requiring a secondary system. They might encourage more dynamic play, especially towards the end of a game, but this doesn’t necessarily mean they result in better or more exciting games. The main problem is that they involve a system of scoring that has no real place in Diplomacy.

DBS systems are best matched to a longer system of games because to effectively differentiate between players in a tournament or short series of games you will almost always require a secondary system of scoring. They produce more cautious play, in general, because taking risks reduces the chances of scoring. This, however, doesn’t mean that this play is less exciting. You are more likely to see players combine in longer-lasting alliances, and draw-whittling is common.

The Binary Scoring system is easy to define because it is a single system. Because it only rewards solos it means players have much less incentive to stay in the game if they’re struggling or to play to prevent a solo. Indeed, it can encourage players to throw games. It does not effectively differentiate between players and it’s hard to see that a secondary system wouldn’t be a better primary system! Because it is purely Unusist it is not part of the design for Diplomacy that Calhamer had in mind, even though it makes solos the only aim of the game.

---

POSTS IN THIS SERIES

• What makes a successful scoring system for Diplomacy?
• Sum of Squares system
• Tribute scoring system
• Hurst Tournament Scoring system
• Calhamer Point system
• A Basic DBS system
• Detour98F
• C-Diplo
• DC(C) system
• Mystery Scoring system
• Diminishing Value DBS system
• Mean Comparison Scoring system