Just before rehearsals started, the EBU announced a change in the way the jury rankings would be calculated this year. After explaining those changes in a previous article, we have now calculated alternative results for the 2016 final. So, what would change?
First, if you don’t understand how we came up with the results, don’t forget to read our article explaining the new system. For the most curious and/or the most mathematically able, the actual formula used to calculate juror scores is written down at the end of the article you’re reading.
The results
First, here are the results of the new jury voting:
As you can see, the gain and loss of points can be considerable: Israel lost 16 points and three places, for example.
However, if the difference has strong effects in the jury voting, adding the televote changes many things, and “smoothens” the results to make the scoreboard closer to the actual results. The difference in points remains the same (since nothing changes with televote points), but the rankings, as you can see below, are not that different.
As you can see, only songs below the Top 10 are affected. Ties have been resolved using the official rules (who has the most televote points is ranked higher).
Interpreting the results
Rank swapping isn’t a big thing
At first sight, the results don’t seem too different. Changes in jury points don’t have much impact on the general scoreboard, and none on the Top 10. Why is that? In general, because only the Jury votes change, and even though the loss of points can be big (Israel), it is still not much once the televote kicks in.
As for the Top 10, two reasons.
First, “top scores” (8-10-12, the Top 3 of a jury) don’t change much. In fact, only 3 of the 42 juries has had their favorite song change (Croatia, Denmark and Italy).
Then, there are usually bigger “gaps” between songs in the Top 10. The many “moves” between the 11th and the 18th place can be explained by the short gaps between countries in that region of the scoreboard. In the real results, The Netherlands, Malta and Austria scored respectively 153, 153 and 151 points. The tiniest change easily affects that. It’s harder with Bulgaria and Sweden (307 and 261 in the real score). Even though Bulgaria lost many points (11) and Sweden gained almost as much (9), it’s not enough to have them swap.
Still, any short gap gives the potential for swapping. It worked with Spain and Croatia. It didn’t with Sweden and France. In both cases, the difference was of 4 little points. Croatia gained 6 points while Spain didn’t gain any. France and Sweden gained points, but Sweden gained more than France, so the gap is actually bigger.
Also, as you can see, the Ukrainian victory is now a bit larger. Ukraine gained 8 points, while Australia lost 4 points. In the actual results, a single change in San Marino’s jury of Italy’s televote could have changed everything: both gave 12 points to Ukraine and none to Australia. Swap those, and Australia would win, one point ahead of Ukraine. But now, with a wider gap, you’d need more than one change.
Who wins? Who loses?
One of the reasons justifying the system change, is that the new rules will normally reward risky songs, and “punish” songs that would be too safe. Now that we have actual results, are the expectations fully met?
One thing is true: as expected, songs that divide jurors strongly are rewarded. Songs that brings what we’ll call a “medium consensus” (everyone agrees it’s good enough), without it being fantastic nor bad) lose points. However, are they alwas risky or safe? And how exactly have the winners and losers gained/lost points?
As a reminder, here is the weight of each rank:
| Juror rank | Juror Score |
| 1 | 12 |
| 2 | 10.50 |
| 3 | 9.17 |
| 4 | 8.01 |
| 5 | 6.99 |
| 6 | 6.10 |
| 7 | 5.32 |
| 8 | 4.64 |
| 9 | 4.05 |
| 10 | 3.54 |
| 11 | 3.09 |
| 12 | 2.69 |
| 13 | 2.35 |
| 14 | 2.06 |
| 15 | 1.80 |
| 16 | 1.57 |
| 17 | 1.37 |
| 18 | 1.20 |
| 19 | 1.05 |
| 20 | 0.92 |
| 21 | 0.81 |
| 22 | 0.71 |
| 23 | 0.62 |
| 24 | 0.54 |
| 25 | 0.48 |
| 26 | 0.42 |
Let’s look at a jury to better understand
You can always check the French example on the previous article. It is an interesting example of how songs can enter the Top 10 with the new system.
While we made the calculations, several jury had interestingly different results. Here, we will only show one of them, which is representative of most changes of the new system, and of how the winners (Sweden, Austria, Ukraine) and losers (Israel, Bulgaria, Malta) gained/lost their points.
Here comes Macedonia.
In red, we have what we could objectively call the losers of the system for this jury. In green are the winners. The two countries with light colours (France and Lithuania) are a loser and a winner, but their gain/loss in rank being out of the Top 10, their score did not change.
Macedonia had in 2016 one of the most spread jurors rankings of the final, hence a very strong sensibility to the new system.
The Winners
Russia and Sweden are good examples of how a very good ranking from one juror can get you in the Top 10. However, this couldn’t happen without at least a second or eventually a third “good” or “medium” rank (for Russia, the very good rank is 1st from Juror A, and the “support rank” is 9th from Juror B: had it been a 19th, Juror A’s ranking wouldn’t have been enough ; for Sweden, the very good rank is 1st from Juror D, supported by a 10th and a 13th). Lithuania is also the case where the good juror rankings give a better rank overall, but not enough to get into the Top 10.
Serbia and The Netherlands are good examples of how the new system reduces the influence of a minority of bad rankings. Serbia has one juror ranking it 20th, while the others all have it in their Top 3. The Netherlands have two jurors ranking it 24th, with the three others ranking it 8th, 5th and 2nd. In both cases, the lower ranking have less weight than the better rankings, which “lift” the songs higher.
The Losers
First things first: in every cases, the losers are never really “brought down” or “killed” by their scores in the new system. Instead, they are victim of the winners: a song will get a lower rank than in the old system if it’s not strong enough to keep its original rank against songs that go up. The less strong it is, the more it will fall.
Latvia is a very good example of where the winners’ advantages stop, to become drawbacks. Latvia’s juror rankings are spread, but not as much as Russia’s. Indeed, Latvia’s 5th, 6th and 10th are not enough to “lift” the song. They are more like the “support” ranks. And even though it has less low rankings than Russia (only 22nd and 17th, when Russia had 22nd, 19th and 18th), this does not make a lot of difference. Indeed, low rankings have a very light weight, so they don’t influence the results much. The lifting comes from the high ranks, and Latvia’s are not high enough. As such, Sweden and Russia jump and take the 8th and 9th place, pushing Latvia down at the 10th place.
Bulgaria is an example of a “kinda-good consensus” not exactly strong enough to maintain its rank. Being ranked 3rd, 4th, 4th, 6th and 7th, it created a consensus among jurors: everyone likes it, and as such it was 2nd in the actual results. But everyone really liked Serbia, except Juror D. Their “20th” ranking is ‘cancelled’ and Serbia is lifted. Bulgaria, however, is a bit weaker than Australia, which is ranked 3rd by three jurors. This is just enough for Australia to compensate a 9th and be a little bit stronger than Bulgaria. Thus, Australia stands its ground (stay 3rd), but Bulgaria falls more (from 2nd to 4th).
Azerbaijan is the example of the “free-falling medium-low consensus”: 8th, 10th, 12th, 16th and 21th was enough to get into the Top 10, but in the new system, let’s see it fall. Latvia and Israel, from the Top 10, fall into 10th and 11th place but are still stronger than Azerbaijan. Poland is strong enough to keep its 12th place, Lithuania’s two “5th” rankings lift it up just under Poland, at 13th. Armenia (15th in the old system) and its “4th” ranking allows it to swap with Spain (14th in the old system), which is still stronger than Azerbaijan (with a “6th” and a “9th”). The falls stops at 16th place. That’s where France’s fall starts.
Finally, Israel is a bit all at once. The ranks are less spread, but it’s not exactly a consensus. A 6th and a 7th are not enough to stay 6th in general, and the 12th, 14th and 15th are not enough to stop the fall before it gets out of the Top 10, but they still stabilize Israel at the 11th place.
And in general?
In general, the Macedonian jury shows most patterns excellently.
Sweden‘s gains mainly come from entering Top 10’s. When they were already high, they usually stood their ground without moving. Being outside of the Top 10 was due to very low ranks pulling it down, despite one or two Top 3 rankings. Those higher rank are now lifting Sweden up.
As for Ukraine and Austria it’s a combination of higher ranks lifting them more (meaning three jurors at least had ranked them low), or lower ranks being less “heavy” on the final ranking (meaning one or two jurors had ranked them low). Obviously, the logic is similar, but usually, in the former case, the song didn’t rank well originally and now ranks quite well (Russia in the Macedonian example), in the latter case, the song ranked very well, and now ranks fantasticaly (Serbia in the Macedonian example).
The losers share the same story: fragile consensus. Malta and Israel had “middle-of-the-Top-10” consensus helping them to get… in the middle of the Top 10. However, without any very good rank coming from jurors, they just fell, sometimes dropping out of the Top 10. The Macedonian example shows it very well with Israel.
Bulgaria is a very interesting case, and yet again, Macedonia shows it very well too. Bulgaria brought a “very-good consensus” in most juries, and got into a lot of Top 3. However, this type of consensus was not solid enough, and Bulgaria dropped of several Top 3. And that is even more penalizing, since dropping from 2nd to 3rd means a loss of 2 points (10-8). Bulgaria was penalized because, with some juries, it was very good and yet not extremely good for any.
So, risk pays?
This is a bit of a subjective field, hence why the following explanations (coming from me, with some ideas given by our editors Yassia and Miki) are not the absolute truth.
Malta and Israel certainly had “jury-friendly” songs, but certainly not friendly enough. That’s possibly the definition of “safe”. Bulgaria however doesn’t seem that “jury-friendly”, and looks more like “risky-but-not-too-risky”. Few people expected such a high score (even though the 4th place comes from the televote, the juries gave Bulgaria the possibility to reach the Top 5) in 2016. The song worked better than expected, but not solidly enough for some juries.
Austria and Sweden don’t look that “risky”, but one could argue they are “original” for the contest. Hence, divisive. “Loin d’ici” being in French, and being quite fairy-talesque in its staging might have put off some jurors. However, the 4th “winner”, The Netherlands, could be considered risky with its soft country sound, and that risked paied… a bit.
Finally, it is difficult to ignore the political context when interpreting Ukraine‘s gains. Jamala’s song was seen as being political by many, and this offers two factors of division. First, those who “agreed” with the message, and those who “disagreed” (rare, but possible). And secondly, those who might have ranked it low because they don’t want politics in the contest, versus those who ranked it high because they didn’t care and found the performance and the song very good.
What do you think of this new system? Now that we have actual examples, does it seems fairer? Why do you think the “winners” were so divisive? And who do you think might benefit (or on the contrary, be harmed) by the new system tonight? Tell us more on the comments below, or on social media at @escxtra !
What formula was used?
We used the mathematical formula of the exponential smoothing, with a smoothing factor “a” that we have estimated at around 0.1305.
Hence, the formula is a sequence u(p), with p = “rank by a juror”, and u(p) = “juror score for rank p”.
u(1) = 12
u(p+1) = a*(1/(p+1))+(1-a)*u(p)
(we use 1/(p+1) instead of p+1 to make the sequence decreasing)
The estimation of “a” was made by trying the formula with different valors, and checking the value of u(4), since the 4th rank, according to the official EBU graph, has a jury score that equals 8 or is very very close to 8. The rest of the results can be considered close enough to the graph, so this estimation is, in our opinion, the most reliable, as long as the graph is exact.
