Editor's note: Paul Gift is an associate professor of economics at Pepperdine Graziadio Business School and has formerly provided expert witness support for antitrust cases. He has covered the UFC antitrust case since its inception.
U.S. District Judge Richard Boulware’s stunning decision last week to not accept the settlement between the UFC and fighters who alleged antitrust violations appears to have been based on his acceptance of dubious economic modeling by Dr. Hal Singer, the plaintiffs’ primary expert witness.
The plaintiffs’ entire case hinges on a statistical model that doesn’t mesh with the reality of UFC contracts and Singer’s testimony on the subject may have played a role in Boulware, who admitted he has little experience in antitrust cases, rejecting the settlement.
Boulware made the surprising, and rare, move to deny approval of the UFC’s $335 million antitrust settlement with former fighters last week. He tentatively set a revised trial date of Oct. 28.
The $335 million settlement figure was higher than I expected when it became public in March. My stance was the opposite of Boulware's, who in a June settlement conference expressed “serious concerns” over the amount of monetary damages going to the Le class of fighters. Le fighters fought in the UFC or other Zuffa-owned MMA promotions from December 2010 through June 2017. Johnson fighters fought in the UFC from July 2017 through the present.
In a follow-up July conference, Boulware pivoted to plaintiffs attorneys’ conflicts of interest with respect to the Johnson class of fighters as well as the time fighters would have to potentially opt out of the proposed settlement.
As the only observer of the June conference, my statement to plaintiffs attorneys at the conclusion of the hearing was, “You guys did too good of a job on him,” referencing the fact that Boulware seemed to be the only person in the courtroom who thought the plaintiffs’ case was strong.
“…we can talk about this, right, an assumption that the trial case had real issues and might not have prevailed,” Boulware said in the June conference. “I found based upon what was credible evidence that the damage award you put forward, above what they actually received… for the Le class they were paid 550 million, but I certified damages above that for 850 million to a billion.”
Then in July Boulware noted, “…if this case were to go to trial for the Le class, it's not as if I haven't seen all of -- we had five days of expert testimony. I would be shocked if there's somehow some evidence that has suddenly appeared in the trial that was unknown to me in a fundamental or categorical way…”
Lead plaintiffs attorney Eric Cramer attempted to explain the settlement’s significance to Boulware as “one of the first cases ever on behalf of workers to get as far as it did, let alone settle for multi-hundred million dollars.” He’d later note, “Your Honor, I've been doing this for 30 years. This settlement in a Section 2 case is the highest share of single damages in any Section 2 case I've ever seen or known about.”
What Cramer didn’t fully mention in open court – but danced around – was that his case isn’t as strong as Boulware seems to think it is.
“I'm in a difficult position,” Cramer said, “because if Your Honor does not approve the settlement, I have to go back and try the case against the defendant. And so I don't want to talk down my case too much.”
“I'm in a difficult position because if Your Honor does not approve the settlement, I have to go back and try the case against the defendant. And so I don't want to talk down my case too much.” -- Eric Cramer, plaintiffs attorney
While Boulware hasn’t yet issued his written justification for denying the settlement agreement, it appears he feels the Le case is too strong to warrant what would’ve ultimately been a net $193.5 million payment to its fighters, based on his language and demeanor in both settlement conferences.
Fact and expert discovery haven’t yet begun in the Johnson case, so Boulware’s denial of that portion of the settlement likely stems from perceived conflicts of interest with plaintiffs attorneys, the ability of fighters to opt out or challenge the settlement, and the rights certain fighters would be giving up.
“But, again, for me, one of the fundamental questions I had related to the Johnson class and the adequacy of the settlement really related to the question about giving up certain rights for $3,000 or $6,000,” Boulware noted.
The UFC probably isn’t terribly concerned with Johnson, supported by the fact that it asked the judge to initiate discovery for the July 2017-to-present time period while plaintiffs opposed it. Additionally, many Johnson fighters have agreed to class-action waivers and arbitration clauses. Certain UFC contract terms have been eliminated or shortened in response to the original Le lawsuit, competitors such as PFL and One Championship have taken in hundreds of millions of dollars in investment capital and expanded their operations, and plaintiffs expert witnesses would arguably be locked-in to a statistical methodology that showed serious flaws in the Le case and may not provide the results they need to effectively litigate Johnson.
As for Le, while Boulware may be shocked if evidence “suddenly appear[s],” he’s been seduced by what seems to be sophisticated hand waving with the evidence in front of him. Here’s why.
A “Pathbreaking” Lawsuit
Originally filed in December 2014, the Le case was certified last year as a class action of over 1,200 fighters suing the UFC for the alleged anticompetitive acquisition and maintenance of monopsony power in the market for elite, professional MMA fighter services. Put differently, the UFC purportedly maintained dominance in the MMA industry by excluding rival promoters through acquisitions, long-term exclusive fighter contracts, and coercive tactics.
The heart of the case was the second element – long-term, exclusive fighter contracts. If a competing MMA promoter doesn’t have access to fighters, they don’t have a viable promotion. And a lack of viable promoters could dampen competitive forces for fighter services and ultimately reduce fighter pay from what it otherwise would have been.
That is the story plaintiffs had to credibly tell via their primary expert witness, Dr. Hal Singer.
In the lead up to trial, almost every Boulware ruling went the plaintiffs’ way. The last straw leading to a settlement was that the UFC would be excluded from calling five fighters, five managers, and three employees as witnesses at trial and evidence in the trial would completely cut off at June 30, 2017, as if the MMA world was frozen in time for the past seven years.
Yet even with these rulings, winning an antitrust case isn’t easy.
To prevail at trial, plaintiffs need to prove the UFC possessed monopsony power; willfully acquired or maintained that power with anticompetitive conduct, and caused an antitrust injury to its fighters. It’s a difficult task made even harder by the fact that their direct evidence of the UFC’s alleged monopsony power, antitrust injury, and damages calculation of $894 million to $1.6 billion all hinged on one single, solitary regression.
If that regression model had flaws, the plaintiffs’ case would be in serious trouble. If the regression model was nonsensical, the expert’s work would typically be excluded and the case wouldn’t even make it to trial.
The problems with Singer’s regression have been detailed multiple times at Forbes Sports. Essentially, he tried to measure how a fighter’s pay as a percentage of the UFC’s event revenues (wage share) was explained by other variables, including a purported measure of the UFC’s dominance called “foreclosure share.”
The idea was to show that as the UFC’s alleged dominance increased, fighter pay went down. If Singer performed his analysis with wage levels (i.e., a fighter’s pay in dollars) his results would show $0 damages, which is a huge problem. So he took the unusual step of analyzing pay as a percentage of revenue instead of pay in dollars. But to carry any evidentiary weight, a regression needs to yield results that make sense. This is why academicians must justify their work under peer review and conduct sensitivity analyses on their findings.
According to Singer’s model, a UFC fighter does not earn more money when they win their fight. This contradicts one of the most prominent facts that came to light during the case: When a fighter wins their fight, they double their base pay 94 percent of the time. And that’s just doubling. Sometimes fighters have a win bonus in their contract that pays extra for winning, but not double. So, any model that doesn’t show a statistically positive relationship between winning and fighter pay is completely unreliable and shouldn’t be taken seriously, especially as the basis for claiming up to $1.6 billion in damages.
Antitrust liability and damages have to be proven with reputable and reliable evidence.
Singer was asked about the troubling result from his model at an evidentiary hearing in 2019. Since so much of the plaintiffs’ case hinged on the reputability of his statistical work, the entire case was on the line at that moment. He attempted to resuscitate his model by testifying that the nonsense result from his regression was due to a correlation problem. Singer’s variable for whether a fighter won the current fight in question (Win Flag) “turns out to be very highly correlated” with another variable representing the fighter’s prior wins in Zuffa-owned MMA promotions, according to Singer’s testimony.
The problems with his explanation are (1) win bonuses are so fundamental to UFC contracts that a statistician should be able to throw anything, including the kitchen sink, into a regression model and still find a statistically positive relationship between what’s being called Win Flag and fighter pay, and (2) the actual correlation between Win Flag and a fighter’s prior wins in Singer’s analysis was 0.20, according to a source with knowledge of the matter.
A correlation of 0.20 is generally considered weak or negligible by statisticians. Yet Singer testified to Judge Boulware that the two variables were “very highly correlated” and he presented this purported high correlation as justification for Boulware to keep faith in his model used to estimate up to $1.6 billion dollars in damages. “This is a very common – a common occurrence in econometrics,” Singer testified, “and it would – there would be no reason to have to lose faith in the overall regression.”
“Does anything about Dr. Topel’s, quote/unquote, wrong signs argument undermine your regression in any way?,” Cramer asked Singer later.
“No, they do not,” Singer responded. Dr. Robert Topel is a UFC expert witness.
If the newly revived case ends up making it to trial, it can’t be overstated how massive of a problem this could be for Singer’s credibility and the plaintiffs’ case. I’ve described their case in the past as optically strong and fundamentally weak. Singer’s model is part of that fundamental weakness and plaintiffs’ claims of foreclosure effects and suppression of fighter pay both rely on it.
If a jury of everyday folks doesn’t end up appreciating this important detail at trial, an appeals court certainly will. The UFC knows this and its acceptance of a $335 million settlement was likely the company attempting to avoid the headaches and expense of potentially going down the appeal route. It was a win-win for both sides. Plaintiffs had taken a situation where the only reliable statistical analysis returned zero in damages and turned it into $335 million while the UFC gained certainty and closure over two lawsuits, avoiding the time and resources needed to see the cases through to their ultimate conclusions. Or so we thought.
Singer’s History
In the same discussion about Win Flag in the 2019 evidentiary hearing, Singer testified to Boulware, “Your Honor, remember, we have hundreds of controls and thousands when you consider the fighter fixed effects. And, so, the notion that -- that we're not going to get the precise sign as one might expect for each variable and statistical significance for all of our controls is not as -- is not as important as what -- what we're -- again, what I'm trying to focus on, which is what is the relationship between foreclosure and wage share, controlling for all other things.”
It's disheartening that a Ph.D. economist and seasoned expert witness would testify in such a manner. Singer is just flat out wrong there. A statistician’s model being grounded in reality is far more important than any estimated relationship between two variables of interest. If a model doesn’t comport to the real world, all of its supposed findings are meaningless.
Just last week, U.S. District Judge Phillip Gutierrez overturned a $4.7 billion jury verdict against the NFL in its Sunday Ticket litigation because plaintiffs’ expert opinions were found to be “unreliable,” there was “no other support” for an antitrust injury, and for one of the experts, there was “simply too great an analytical gap between the data and the opinion proffered.”
This is almost the exact situation facing the UFC with Singer – an enormous analytical gap between the real world and his expert work. The testimonial hand waving of what the judge should “focus on” is reminiscent of what has landed him in hot water in other lawsuits.
In a 2020 price fixing case, he was described as doing an “end run” around the judge’s ruling. After questioning by the judge, a particular chart of Singer’s was excluded with the judge stating, “I told you not to do that. You assured me several times that had not happened.” The exchange continued with an attorney from Singer’s side, who also worked on the UFC case, where the judge concluded, “You misrepresented this to me. That was a flat-out misrepresentation… You are suspect in my eyes now.”
The UFC has shared this and six other situations with Boulware where Singer’s work was allegedly excluded for reasons relating to reliability or “untenable” assumptions. Such information should lead a judge to approach Singer’s work with great skepticism, but we haven’t really seen that yet from Boulware.
Judge Boulware’s antitrust inexperience
When Singer finished his Win Flag testimony at the 2019 evidentiary hearings, he left the witness stand and sat next to me as I immediately whispered in his ear “I call BS. I call BS on Win Flag being highly correlated with [prior] wins.” His testimony had set off alarm bells in my head. But for Boulware, it seemed to be business as usual, not showing indications anything was awry.
When an expert witness says “…we're not going to get the precise sign as one might expect for each variable… what I'm trying to focus on… is the relationship between” two particular variables, it’s a red flag that something might be hidden or obscured. A seasoned antitrust judge would take notice. Yet Boulware has repeatedly appeared ill-equipped to preside over a case of this magnitude when it comes to the economics and statistics involved.
Confirmed by the Senate in 2014, he disclosed that “developing an expertise in civil litigation will present the greatest challenge for me.” Then by June 2015, he was the presiding judge in this major class action, monopsony case.
Boulware recently admitted he hasn’t overseen “multiple cases involving hundreds of millions of dollars of antitrust classes settling… this is my first antitrust monopsony class action,” and it shows.
From asking “What’s that red line?” when looking at a basic economics diagram to misunderstanding a sensitivity analysis criticism, Boulware’s comments and opinions on economics and statistics topics have often been bizarre.
When discussing wage share in his class certification order, Boulware listed no academic support for its use in compensation regressions to assess monopsony power and seemingly ignored the testimony of Dr. Paul Oyer, the former editor-in-chief of the Journal of Labor Economics, when Oyer testified, “…there are sports examples and there are articles that discuss the wage share in those articles, but they never -- they never run a regression the way Singer did, never. Literally never.”
He also continued a misunderstanding of monopsony and wages made all the way back in 2019 and argued that UFC experts Oyer and Dr. Robert Topel had themselves utilized wage share to evaluate monopsony power. To support his claim, Boulware cited two academic articles in which neither was an author along with a report Topel co-authored as part of a consulting project related to whether the NFL could afford a particular salary cap. It had nothing to do with assessing monopsony power.
Bizarre.
Boulware’s also seemingly ignored evidence of a fraudulent journal, which was the basis for arguing fighters are the product and therefore that wage share is an appropriate metric. In his summary judgment order, he concluded that a median fighter career length of 0.82 years was an “appropriate empirical finding,” yet apparently still thought the UFC could disadvantage rival MMA promoters when careers last less than a year? And he allowed Singer to assume foreclosure effects on UFC competitors rather than prove them.
It's all been very puzzling. And that was before last week’s news that Boulware and UFC CEO Dana White apparently went to high school together.
Plaintiffs surely appreciated Singer’s weaknesses and Boulware’s unusual decisions or they wouldn’t have settled both cases – one with the potential for over $4 billion in treble damages and another whose potential is yet unknown – for $335 million. Yet it’s a sum that appears too small for Boulware since in his world Win Flag is “very highly correlated” with prior wins and it makes perfect sense for a model to estimate over $1 billion in damages while also supporting the peculiar notion that fighters don’t earn more money when they win.
Will Boulware’s rejection of the settlement end up extracting more money from the UFC? That’s yet to be seen, and the way this case has progressed so far, who could possibly know? But it also may cost fighters the entire sum, especially the Johnson class – a group of fighters who are separated from Le simply because of a “particular random point in time.”
When reached for comment on the correlation between Win Flag and a fighter’s prior wins, Cramer disputed the 0.20 correlation number from this article. When asked a more detailed follow-up if 0.2024 was the result obtained by using the “correlate” statistical command on the Win Flag and prior wins variables from the dataset in Singer’s Table 6 regression in his expert report dated Aug. 31, 2017, and to comment with specifics on any factual inaccuracies, Cramer did not respond.