Article

Judging the Bias

Whether we know it or not, everyone plays favorites, even judges

| posted on April 26, 2012

Kelly Slater, throwing up a hail mary in an incredibly close final against Mick Fanning at Bells Beach. Photo: Joli

If you were born in America, odds are you grew up pledging allegiance to the flag, have at least a mild affinity for “Subterranean Homesick Blues,” and feel an inexplicable draw toward In-N-Out (or Five Guys, for you East Coasters) whenever returning from abroad. I would assume that every nation has their versions of Bob Dylan and a Double-Double with cheese (in their minds, at least), and that shared experience gives nations a sense of unified identity. This is all well and good, but what happens when we drag this baggage into the world of professional surfing? More specifically, what happens when we drag it into the judge’s booth?

Recently, University of Illinois PhD student Breno Sampaio finished his dissertation in the field of forensic economics, in which he discussed a bias that exists in the judging of professional surfing events. His research, which was conducted using a sample of wave scores from five events in 2010, showed a trend suggesting that ASP World Tour judges tend to give their fellow countrymen preferential treatment, but not in the way you might think.

Although the knee-jerk reaction is to rally as a community, light our torches, and sharpen our pitchforks for a good ol’ fashioned ASP lynching, they may not quite have earned it in this case. According to the study, ASP judges aren’t over scoring their compatriots but instead the study finds that they are underscoring the foreign opposition. The research found that on average, judges underscore the opponent by 0.133 for all waves, with the average rising to 0.3 on waves that will actually be counted toward their heat total. If their compatriot is winning, however, this negative effect disappears. In other words, judges aren’t handing heat wins to their countrymen on a silver platter, they just want to keep them in the game if they fall behind.

For those of you who don’t have a basic understanding of ASP judging, it goes something like this: Each heat has five judges to score waves, with a Head Judge supervising the panel. The panel is chosen from a pool of international judges who rotate throughout the event. During each heat, all five international judges contribute scores, but the highest and lowest are dropped, allowing the average of the middle three to make up each wave score—which the ASP contends is enough to eliminate any potential bias. Unfortunately, according to the study, a competitor still stands a better chance of winning with a compatriot in the booth even after dropping the outside scores.

When the ASP was pointed to the research and asked for comment, the feedback was surprising. “I don’t rate any of it of any value,” replied ASP Head Judge Richard Porta. “We have a guy that sits in the middle of America and plays with figures and formulas. Me, I have more important things to worry about than if a judge is 0.8 off the mark in a handful of heats out of the 510 mens heats that will happen this year. Plus the 91 women’s events and the five Prime events I will be Head Judge at this year.” Co-ASP Head Judge Pritamo Ahrendt was more pragmatic in his response to the numbers: “I believe that we have the best system in place to curb bias,” he says. “We choose judges that are surfers and that love surfing as a sport and understand that being biased is the worst possible trait that any judge could possibly have. If we notice national pride or biased judging or just bad judging they will no longer work for ASP.” It’s clear that this is a topic they don’t take lightly, but neither would admit that there is even a possibility of bias within the judging. The numbers from Sampaio’s study, however, certainly beg to differ.

You may feel that these less-than-a-point discrepancies are trivial. But before you discount the research entirely, think about the final heat between Mick Fanning and Kelly Slater at the 2012 Rip Curl Pro at Bells Beach. That final was decided by less than a point, and it was a divisive win, with pros, fans, and pundits aligning themselves on either side of the debate. In a recent interview with STAB Magazine, Josh Kerr lent these words about the Bells final: “For Mick, I’m sure it helped that it was down at Bells and in Australia. I have a feeling if that final had been at Trestles, Kelly would’ve won.”

Perhaps it’s time that we find a neutral party—surfing’s Switzerland, if you will. We train those mountainous folk to judge surfing from a live stream in a cozy little cabin in Zurich, and as long as they don’t produce a world-class surfer themselves, our sport is safe. But even then we would not be out of the woods. Judges have to be lifelong surfers with a deep understanding of aquatic idiosyncrasies to be able to know for certain that a tail-high slob is far superior to a double-grab wheelie. Unfortunately, those individuals are and must be part of a developed surf scene—a scene that produces surfers they will more than likely be biased toward. In the end, no matter what we do in the pursuit of objectivity, surfing will never be the cut-and-dry sport of absolute winners and losers.

Watch Mick defeat Kelly by less than a point during the Bells final:

  • Pedro Lima

    No system is perfect. The ASP can also improve the collaboration of all.

  • J

    Kelly and Sally are consistently over scored

  • Rob

    Kelly is consistently under scored

  • bacuri

    kelly first 8.07 was totally underscored

  • Elliot Bates

    Kelly is constantly overscored. By at least 0.5 of a point.

  • Guille

    Show this video to anyone and will tell that kelly is the winner. Mick is a great surfer but contests are this way.

  • http://glas.com Dave Schauber

    I judged and head judged a few events at the WQS level, worked a few N. American CT’s and a few International events as well judging the Big Wave World Tour almost every Mavericks contest ever held so I don’t care what any judge or head judge says in regards to there being no bias at times because if they deny it they are all full of shit.

    I will be the first to admit that there are definitely some better judges and head judges than others. They are the ones that actually care about the sport and judge heats based on performance. Then you have what I call the groupie judges. These are the guys that sit up there on the panel to feel like they are in some way special or part of the pro surf scene. All they really care about is being on a first name basis with or to rub shoulders with the best surfers in the world. These guys usually have their hand up asking for an average or a replay after every single wave and need to babysat through the entire event.

    Quite frankly in reality most of these guys can’t even surf, they couldn’t make it as a pro and have found their way on to the tour as judges mostly on a who you know basis. In North America we have people that move here from other countries such as Brazil, S. Africa and Australia to live so the the ASP can portray them as International judges for bigger events and not have to fly a mixed panel in from their respective countries. The flip side is these very same judges then fly to S. Africa, Australia, or Brazil and can work as a North American judge or a judge from their country of origin which quite frankly is unfair.

    The whole system in which you become a judge is flawed. I have long felt that the top amateur or pro am judges from each country who makes it as an ASP judge should be ranked by performance and according to who is the most consistently closest to the average score over the course of an event or series of events and the Beach Byte computer is already set up to give this information.

    Judges should start out doing lower star rated events and as he or she progresses and they are scoring well according to the computer average, they will then progress up the list of potential judges for each event. It is simple if you perform well you will be at the bigger events where the surfers and the surfing are of a higher caliber. If your scores are off the average your ranking will drop and you will be doing lower star events again until you improve your rating or are asked to leave. This gives judges the incentive to score and be rewarded or perform poorly and lose your job

    Also there should be a judges oversight committee where in there are cameras place at 3 angles in the judges booth with video and audio. This way if there is ever an instance where there is a questionable situation or score we can then review the tape to watch and listen to what was occurring in the judges booth during the wave in question.

    At this point no judges or head judges are held accountable for their actions and in reality have no one to answer to or oversee their performances. I do like the advent of the multiple head judging system at the Championship Tour and Prime Event level and see this as a positive change in the right direction.

    In reality a judge should be there for one reason and one reason only and that is to throw the correct score at whatever color surfs the best on a wave based off their performance according to the ASP’s current criteria. Nothing more nothing less.

    I don’t think that partiality lies in the direction that Aussie guys judge Aussie guys higher or Americans judge Americans higher at every event etc. What I have seen personally is that the partiality varies from event to event and is dependent on the current title situation, the vibe, and the electricity going down on the beach. If an event sponsor has a surfer in the event that is local and they find themselves in a close heat they almost always get the nod. I think it is more subliminal than blatant but it definitely does happen quite frequently.

    There is a lot that can be done, it all seems fairly simple but what people out there need to understand is that pro surfing is a big corporate money machine manipulated and controlled by the media and a hand full of million dollar corporations who’s main interest is making money not getting a proper winner. As long as the judges are “ASP judges” instead of an independent panel of experienced dedicated people elected to do a job based on a revolving door of performance , then nothing will ever change.

    At the current rate I don’t even see the ASP being a viable part of surfing’s long term future.The big 5 companies and the top surfers themselves have figured out that they are carrying the ASP and that the ASP could not survive financially if they pulled their sponsorship dollars or quit attending their events.

    If an organization made up of established trustworthy leaders within our sport can obtain enough capitol over a 5 year period from resources outside the surf industry to sustain a long running and mutually profitable tour, the ASP is doomed. It’s just a matter of time.

  • Fishboard Twin

    Does anyone here really believe in professional surfing? If it was a serious sport, Kelly would not be 11x world champion. How many times he was overscored in decisive moments that allowed him to move on? Americans and Australians often win by small margins. Especially when they are home. Funny how no one says anything. But now what? Kellys loses in a final… is something wrong? Who appears in the media, who sells more magazines, always wins by an extra point. Moreover, the problem of bias is also in commentators. It’s annoying how the media’s favorite win many accolades during transmission. The guy may not be surfing anything but be praised until the end. Fuck that! What should count is the performance showned in the moment of truth, and nothing else. The lack of professionalism in the surf amazes me. I root for big brands such as Nike entering the surf. Maybe then the sport can truly become professional and go global.

  • barry

    I so feel that normally Kelly is over scored but honestly at Bells I honestly don’t know how he didn’t win !!

  • Billy Dora

    The fact that surfing judges are biased is a long understood reality of the shitty ASP system and it would be a non-topic if surfing did as, lets say gymnastics has done. Give all tricks and all possible maneuvers a degree of difficulty rating and a possible point total with a plus or minus of say .5 points in either direction depending on how well the trick or move was done. That would take out the human element and simplify the scoring numbers by establishing a consistent point value for every move. Then any one of us with an understanding of the point system could keep score and quickly know if anyone is being over scored. Simple. But alas the ASP is too stupid to figure out a consistent scoring system let alone keep an accurate tally of competitors total scores.

  • Kelpy Slaughter

    Kelly has consistently surfed better than his opponents over the years. He has a game plan, a formula, tactics, and has trained his mind and body to achieve his goal. His goal is to win and be world champion. He has risen to the challenge and beaten 4 generations of surfers fair and square. He approaches the sport with mature and dedicated effort. He has taken the time to practice and master the waves on tour. He didn’t take for granted his natural ability. He is a master of his equipment. He is a nightmare for his competition. He is not a god. Don’t hate on him…beat him…if you can. No conspiracy, no favoritism, no corporate agenda. Just domination. Effort, imagination, passion, fitness, health and desire that is his formula. Good luck unraveling all that.
    Go Ke12y!

  • kev

    Mick won the heat. Best two scores on two waves. Kelly got a 10 on one wave. Time to move on.

  • the spanner

    watching that footage it is impossible to know who won that final. Cutting all the waves to pieces, water angle, beach angle, zoomed out, zoomed in. We are all being lounge chair judges. Sick final and awesome for surfing.

  • quiet one

    I noticed the sample was from 2010. How long has this been sitting in SURFER’s vault waiting for an appropriate moment to publish, when an American loses a close final. Wasn’t that back when there were 48 surfers on tour and the system was all different. What about all the heats when country men are against country man which is a huge amount of heats, are they then biased against goofy footers. When a sport is as subjective as surfing you will always be able to find discrepancies that you can see as bias. Do the study again to see if it is relevant now.

  • tracky

    Who is Dave Schauber thats one long boring message. Sounds like a disgruntled ex judge. He was probably biased and got the boot

  • Patty

    I sat on the beach with all my mates and we all thought that Mick won fair and square. I only realized there was any controversy when I checked all the American surfing sites. The heats on demand does no justice to what really went down. oh well didn’t everyone love the epic moment in surfing

  • Frank

    This is another take on judging and bias and the Bells Final.

    http://surfinglife.com.au/news/asl-news/6809-by-the-way-it-doesnt-matter-who-sponsors-you

  • Jeff

    I am a huge fan of surfing and think the judges do a good job over all. Resently I have seen some scores that were questionable like like the heat at Pipe Masters with Kelly and John John. I was there and John John was robbed. And the final at Bells Kelly was under scored on one of his waves. Sally is always over scored as well. I know that judges can make mistakes but with all the technology they have I think they could do a little better.

  • Steve

    I think maybe some of the conspiracy readers should read nick Carroll’s article titled by the way that he had in asl April 11th

  • Hendo

    Wah. Wah. Wah.
    Someone call the Wahmbulance.

    Slater lost by a point in 2012 and you have data from 2010 that proves he was cheated – come-on Surfer Mag, pick your game up.

  • Dropknee McGee

    bias, no bias, judges do this, Kelly does that…
    Has world gone bananas?
    Watch this clip:

    http://www.youtube.com/watch?v=ST3OgExVqCg

  • http://www.yankaus.com Mik

    @Dave Shauber: thnx for a well though out comment. Accurate. Still, it is what it is, and what it is is rad. Watching the best surfers in the world blowing-up i all kinds of surf conditions has changed my whole approach to surfing, and my appreciation of what is possible. Love the commentators, in general. Not into bias, disappointed often, but totally grateful to the surf companies who have stepped up to put these events on. It’s allot of work, allot of money, and hopefully it is returned by me/us via brand loyalty. I definitely buy ASP sponsor goods. So much cooler than the NFL, MLB, NBA or FIFA!

  • d money

    How about how surfers with less media exposured get underscored consistently? There have been many unnoticed 3rd or 4th round heats over the years where kellys opponents get underscored enough to let kelly pass and as the contest progresses kelly rips his way to another win. Getting a bad score halfway through the heat changes the psychology of the heat, guys will try too hard and bog, knowing they have to push it to compensate for the underscore. I’ve long felt that underscores, sometimes blatant, demoralize their victims, who have to fight an uphill battle against their opponent and the judges.The judging seems pretty good, but timely lapses in their judging consintency make the system inherently unfair.

  • Jay

    The only over scoring going on in bells was for parko!!!!! Take a look at his heats and scores…… Complete waves yes, critical no!!!! Kelly even bought this up on live interview. I feel mick surfed to the criteria that the judges expressed they wanted at bells ( full long rail turns) but Kelly in terms of general surfing just amazing and should have been a clear point different winner. I feel regardless of the location the surfer who surfs the best should win???

  • http://surfer Kimbo

    please Dave since when did we start qouting Josh Kerr? Kellys best mate ! . Throughout his career Kelly has been the recipricant of overscored heats in his favour . This debate is history , the fact that Kelly has taken it upon himself to openly judge others results ( PARKO) is just bad manners . I think we are experiencing a similar situation to Tiger Woods , No person is bigger than the sport .

  • Rikki

    Fair call Kimbo but there IS something about the homeboy hero bias syndrome that often the judges can’t seem to deal with….maybe its the home crowd noise factor that influences them…or is it subliminally keeping the locals/hosts/ happy ?
    eg these results
    Jordy v Fanning at JBay
    Taj v Adriano/ Jordy/ Ace at Snapper
    Simpo at Huntington
    Adriano at Rio
    Fanning v Kelly at Bells
    Kelly v anybody at Trestles ( Taj, Phil Mac et al)

  • BenG

    “There is no true objectivity, only degrees of subjectivity”

  • William

    It’s very clear that Slater is the best and should win. Fanning was way overscored at Bells
    As history has shown American surfers are better than Austrailans. Hawians are second best and then Aussies third. All the other countries equal at fouth.

  • Stucco

    Past contests aside, the world outside Australia (and MOST inside Australia) will concede that Kelly won THIS contest. Clearly. Period.

    I wonder how many people who think Kelly has been overscored in the past have seen these contests in person. In person, the differences between Kelly’s surfing and the rest of the guys becomes so much more obvious. Kelly is STILL, far and away, the best surfer in the world. But before the era of roughly Andy Irons, the gap was even wider, so I don’t know what the hell people are talking about. Go watch Slater at Trestles or JBay or any real performance wave IN PERSON, and tell me he’s not the best all-around surfer in the world, by far.

  • jim samuels

    “Judges have to be lifelong surfers with a deep understanding of aquatic idiosyncrasies to be able to know for certain that a tail-high slob is far superior to a double-grab wheelie”

    Umm… you guys do realize the current judges can’t tell the difference between this right???

    Guys do stalefishes and get the exact same scores as doing a straight air reverse.

  • jbinsb

    Hey Quiet One, It takes time for a study to get completed. The fact that the sample was in 2010 and the study is out now is perfectly normal. Stop looking for conspiracies. I gotta agree with Spanner. Kelly’s air in the final was the sickest, by far, ever. And he started doing airs way after all the hot young air corps who are currently on tour. That said, he bogged after his carving 360, and I thought Mick just completely annihilated his tight turns high in the pocket. It was so close. And yeah, the fact that it was Bells and Mick probably gave him an edge. That edge is part of the human element, and bias should be minimized, but if you want machine judging, you’re also going to need machine surfing. As for the guy who brought up gymnastics, they have had some pretty severe judging scandals despite having values assigned to every move.

  • Dom

    9 qualified judges on the panel, computer randomly selects 5 after each wave scored, drops high and low and averages out the middle 3… I’m sure thats how its done in olympic skating now. No olympic sport has been as scrutinized as skating after all the judging controversy over the years. obviously the BIG issue here for the ASP is the additional costs, but what is an ASP event win worth if its not seen as credible.

  • Big fan

    Mick won, Kelly won, no Mick won, no Kelly won. Isn’t this why we have professional judges to make these decisions. If you look at all the msgs under all contest reports everyone is debating all the results even the ones that are really clear. I’m sure the judges know what they are doing more than all us punters. Judges and referees are always the ones that cop all the grief but they are also making the hardest decisions and in real time. Mick definitely won hahaha. Pro surfing is awesome this year

  • Ben

    Kelly lost the battle, but his surfing will always win the war.

  • s4p

    And that’s why I care less scores and have lost enthusiasm for pro surfing… Surf for yourself… Gain inspiration from friends/pros… but in the end, surf for yourself.

  • Dewey

    Someone should ask the ASP what type of international training system they have in place for their judges. I will give you the answer, there is none. Every member country does things differently. Until the ASP initiates a standard of training for all judges, in all member countries that is the same, there is no judging standard. The statistics don’t lie.

  • Christian

    Whats wrong about judges promoting their own / their mother countries’ individual style of surfing? The more styles the merrier the tour.
    It is competitive surfing… It will never fully make sense.

  • Rod Carver

    I was an international traveling judge and I would agree that it is really hard to keep bias out but in the end you have such a mix it works.

    Worse than national pride though is a preference to style. I judged when Curren won his last title, then Potts and Duma. Curren had such a grip on style it was hard not to judge him high for just a few small carves.

  • liam mcnamara

    Well as far as i saw in the final kelly should have won!!! Kelly did the heaviest air ever done in competition in the final and it was for sure a 10.His other waves both came in at 8.07 (i think and they were both underscored).Or Micks waves were over scored.Yes Mick caught the better waves and surfed them great but the degree of difficulty was not there .What he did guys have been doing for the past 20 years .He surfed them with good flow and good moves but in the end Kelly’s degree of difficulty was on a whole different level.I think the main problem is none of the judges can do a air and have no idea of the degree of difficulty ! The only way the judging will improve is to get guys on the panel who understand what it takes to do the high risk moves.The ASP needs to get former pros and pay them enough to be judging on tour ,Then we will see real outcomes in more events.

  • Wade

    Kelly’s clearly the best surfer we’ve ever seen but does get over scored a lot. Last year at trestles he done 5 big cut backs from start to finish one after the other & was awarded a 9!! I thought the new judging criteria was NOT to reward repetition?

  • Speedo Gonzales

    Dropknee McGee that’s a horror movie. I would rather give up surfing than turn into a goofy foot. If Kelly Slater ever does turn goofy I would expect someone to do the appropriate thing with a wooden stake so he doesn’t have to exist in that tormented state. Right foot forward – it’s just plain weird! I say there should be a systematic under-scoring of goofies to slowly eradicate them from competition, the sport, and eventually society.

  • Nick

    I thought we cleared this up Years ago with the Kelly vs Taj Final at Lowers. 4 turns or in Mick’s case 3 turns to Innovative maneuvers (New scoring) system. Mick killed it. Sick turns, done em before, plenty more on the Reg. Kelly’s maneuvers. So sick. So not on the reg for Anyone. Should have given it to Kelly for the freak factor. Classic turns Mick. but new judging system.

  • http://surfer Kimbo

    theres only one area of surfing that is never overscored and thats good old USA bias.

  • googoo

    Rod Carver, is your son’s name “meat”?

    liar, pants on fire.

  • Steve Riggs

    I just re-watched the final on demand to see what all the hype is about. I really don’t give a shit who wins or loses, but for the life of me I CANNOT see how Slater didn’t win that final. That was one of the worst examples of poor judging yet. Worse than Adriano’s Floatergate, worse Taj over Adriano in the contest prior, worse than anything I can remember. I just can’t see how Slater’s 8.07 (the first one) wasn’t a higher score than Mick’s 9.7, let alone 1.63 points lower. He did 3 different kinds of turns and then FINISHED the wave on the inside. Very complete wave with an exciting finish. Mick does three VERY nice turns on the outside, but basically the SAME turns, and then falls on the inside. Even the commentators all agreed Kelly won that exchange. Friggin’ weird.

  • dropknee mcgee

    I really think all you guys are looking into a bit much.. surfings surfing, couple of blokes decide who wins a few contests, so what, they go to the next place surf couple of blokes decide again. If they dont like the way the system works they leave ie Bobby..life goes on. Someones got to win 33 have to lose thats the way it goes. Not happy tune out and go surfing yourself.. its more fun.

  • GJB

    This are the comments that should be look in consideration to improve this dilemma even though surfing was never meant to be judge but the animal race always wants to prove who is better no matter what:

    Winners:

    1- Billy Dora- The way surfing is now they need to have scores already appointed for every move and depending on how it was perform take or give .05

    2-Dave S- Big fucking comment but you said it like it is, judges need to be proven and need to know how to surf, pay the ex-pros and let them judge.

    3- Kelly Slaughter- Kelly you said it best, Kelly is a freak of surfing, health and life.

    4-D-money- you got a point, that I agree, I am sure many of us commentating here have been thorugh that jajaja.

    5-Jay- Parko is a super great surfer and human being but they are pushing him lately as the big money corps that control the ASP want to give him a championship…

    6- Stucco- you are right Fanning is a great surfer but watching Kelly surfing in person is a whole different thing than anyone else on earth period. Like Curren before him nobody misses watching his heats!

  • Mort

    Maybe this article should be about bias journalism, your king kelly lost, get over it! So many have gone his way but not this one booohoooo!

    While im on the subject how did your boy Kolohe rate higher than medina in your hot 100? he won two asp events and finished equal with Jon Jon at pipe? Kolohe has done nothing in the big leagues..

  • Max

    Love both Mick and Kelly’s surfing. I would rate Mick’s carves and cutbacks slightly ahead of Kelly and Kelly did got cuaght behind on his carving 360. But the key area to compare is the airs, where Kelly’s airs were far superior and air tricks are the future of surfing. Compare Kelly’s airs in every aspect and he blew doors off. Kelly’s airs were higher, longer, full-rotation, totally hand’s free and he did huge airs compared to Mick’s single low-risk low height air. Kelly’s airs were full tilt full commitment and got the crowd totally psyched.

  • adam may

    slater won; no brainer..

    agree 100% with Steve Riggs comment; couldn’t have said it better.. my point exactly.. slaters 8.07 should have been a higher score than fannings 9.7 – shocking.. robbed..

  • ginz

    to all those armchair critics who think they can judge a heat via a web-cast you will never get a true reflection of the whole wave and therefore unable to correctly determine the tight heats, i would love to put you morons on a panel and see how lost you get after a couple of heats, the majority of the WCT judges we have today are by far the best the ASP has ever had and granted occasionally we do see a couple of sus heats but writing off the judges seems unfair considering they are officiating the most difficult sport to judge. So just relax , sit back and enjoy one of the coolest spectator sports around and if it pisses you off change the channel ,wrestling and darts will surely satisfy your intellectual capacity

  • surferguy2014

    Kelly is so underscored this year. I think everyones on the Medina bandwagon.. even the judges.