What Do We Really Know About the Link Between Loot Boxes and Gambling?

By Rich Stanton on at

Last week a House of Lords report called for immediate regulation of aspects of the UK games industry. This report was based on a range of academic papers, as well as oral evidence provided by one Dr David Zendle, a lecturer in computer science and media effects researcher at the University of York. After reading the report and the academic work cited, I had some questions about what exactly we do and don't know when it comes to loot boxes, and Dr Zendle was kind enough to answer.

I'll preface this with my own lukewarm take on the loot box question, and the reason I was keen to speak to an expert, which is that the facts often get obscured by opinion. There's a number of folk who live and die by the slogan 'loot boxes are gambling', and brook no dissent from that line. There are people who want loot boxes banned. There are people who want the more extreme examples to be regulated. There are people who think it's up to industry to self-regulate, where both internal analysis and consumer reaction dictate what is acceptable and what crosses the line. Some think it should be the Wild West and, at the moment, it probably is.

There are also a lot of people who don't engage much with the issue, but buy loot boxes. And there is a lot of money in loot boxes and micro-transactions. We don't know how much, because the industry doesn't tend to break down its revenues in a granular enough manner to say exactly what money is coming from where. But if we take an analyst like NewZoo's 2019 estimate for global games industry revenue, which is $152.1 billion total, we might note that mobile alone (a market where we can confidently say the revenue is overwhelmingly coming from micro-transactions) is responsible for just under $70 billion of that number. I'm not going to parse the figures any further, because it's an estimate and that's the problem, but it gives some idea of the scale we're talking about when it comes to in-game purchases. This is not about billions, globally speaking, but about high double-digit billions of dollars.

Loot boxes have thus been a political issue for many years, with Asian countries far ahead of the curve, and the UK Parliament is far from the first national legislature to examine the issue: the most prominent European result thus far has been Belgium's Gaming Commission declaring that loot boxes are games of chance and so should be subject to gambling law, which among other things led to EA stopping the sale of FIFA Ultimate Team points in the country. This link to gambling is easily the most contentious issue with loot boxes, for the simple reason that we know problem gambling has the potential to ruin lives, and this is why the gambling industry (in the UK at least) is subject to strict regulatory oversight.

Dr David Zendle

So are loot boxes gambling? Do they have a link to gambling? Do they lead to gambling? As well as Dr Zendle's oral evidence to the House of Lords, the report references this academic study (co-authored by Dr Zendle), the results of which suggest there is a correlation between people who problem gamble and people who spend money on loot boxes. In my questions I referred to this correlation as 'minor', which Dr Zendle was quick to clarify is not the case.

"The first thing I'd unpack is the term 'minor'. The correlation that we see tends to look like a situation in which 5-10% of variance in problem gambling can be predicted from knowing approximately how much money someone is spending on loot boxes. This might not seem like much – but in social science terms it's larger than a lot of links we see. So many things can predict an outcome (for example problem gambling) that seeing this much signal under the noise is uncommon. In the study of media effects in general, this is so chunky it passes a bunch of fairly stringent checks for importance of magnitude. Back when talking about video game violence was all the rage, sceptics put formal cut-offs into place in the literature which basically say 'look, any relationship smaller than this you can pretty much ignore'. The loot box correlation leaps and bounds over those cut-offs."

Last year Rocket League joined a host of other major titles in ditching loot boxes

But correlation is not causation, and the suggestion of a causal link creeps in everywhere around the loot box question (the word 'link' is used repeatedly in the HoL report). Is there anything in the literature that does suggest a causal link between loot boxes and problem gambling?

"So the causality is completely unclear. We have literally no clue whether it's a situation in which loot box spending drives problem gambling, or problem gamblers are drawn to spend money on loot boxes. At this point, I wouldn't hazard a guess either way. Anyone who tells you otherwise (at this point) is overconfident. Or they have data that I don't.

When it comes to the existence of a 'link': Two things being linked, at least in my neck of the academic literature, is bidirectional: Instant noodle consumption can be linked to being a student, it doesn't mean that eating noodles will get you a degree! We use 'related' and 'predicts' the same way. In terms of causal links: technically, there has got to be a cause-and-effect relationship in the mix somewhere. Conceptually, everything is caused by something else. So here we have a situation where causal direction is fuzzy: either whatever makes you a problem gambler causes you to buy loot boxes; or buying loot boxes causes you to become a problem gambler. Again – the direction of the link is totally unclear."

The problem here is that, as Dr Zendle explains, reasonable people will think very differently about this connection depending on which direction it is coming from, and we simply don't yet know the answer.

"Both situations are very different in terms of effects: Loot box spending causing problem gambling is something that I think people would universally want to prevent; problem gamblers being differentially drawn to big spending on loot boxes I think would meet a mixed reception. Some would want to regulate to protect them – others wouldn't. [To be clear] I don't have any data on how people feel about regulation in these situations – that's my personal feeling on the matter."

Read More: PEGI to Warn of Loot Boxes on Game Boxes

We had a brief aside about some of the wilder claims around loot boxes, such as one survey respondent claiming to have made a million dollars from them, which seemed to both of us highly implausible. "More broadly," says Dr Zendle. "I don't think anyone outside of industry has a real handle on the prevalence and scope of cashing out, but I'd hazard a guess it's less prevalent and profitable than people might think."

So we know there is a correlative link between people who are problem gamblers also spending money on loot boxes, and social scientists would consider it to be a substantial one. The gaming industry itself knows this, and over recent years some companies have voluntarily distanced themselves from the loot box model, both because there are these moral questions where the answer is fuzzy, but also because there are plenty of other ways to make money: Fortnite launched with what were definitely loot boxes but, when it became a phenomenon attracting a mostly very young audience, Epic soon changed the loot box model into micro-transactions that are clear about what you're paying for (thus removing the element of chance). Behaviour like this doesn't necessarily need applauding, but it should be acknowledged.

With that said, loot boxes are still everywhere, they're still making god-knows-how-many billions, and certain types of loot box lean into the element of chance much more than others.

Self-regulation, in other words, doesn't seem to be happening in any kind of coherent manner across the industry: individual developers and publishers are making their own decisions. I asked Dr Zendle for his perspective on regulation.

"When it comes to regulation – I think there's a hierarchy that you could spin out here. The ideal is self-regulation: The industry takes its responsibilities seriously, and is responsive to consumer protection issues. Industry collaborates on research with independent academics, opens up data to independent scrutiny, and acts strongly and decisively on the basis of credible issues raised. I don't think this is happening.

Next in my hierarchy is for games to have their own bespoke regulator. Something that is sensitive enough to their specific context and business needs, but also with enough force behind it to make companies take consumer protection issues seriously. I wrote a blog on this topic for the BMJ a while ago. However, there doesn't seem to be any appetite for this – a fall-back option would be to disperse regulatory concern for the games industry amongst relevant pre-existing bodies (for example Ofcom).

At the bottom of our hierarchy we have a situation in which there is something worrying going on in video games, but industry stakeholders are rarely showing signs of taking this development seriously. I would exclude TIGA from this – I think they've been refreshingly open, possibly because of the composition of their membership. On the AAA front – I think 'surprise mechanics' and Kinder Egg analogies did a lot more harm to government's patience than many people realise.

So the world we live in isn't a paradise of self-regulation; or a neat universe in which we have a bespoke regulator set up and running. This means that we're stuffed into this situation where the most credible course of action is classing loot boxes as gambling. And I can't oppose that because all other credible options aren't on the table."

With the House of Lords acting on his advice by calling for regulation, I ask Dr Zendle what kind of regulation he would like to see. There's a long answer to this question which I'm going to slightly break up, because it first expands on a previous point about why the industry needs to wake up about self-regulation.

"My dream world is self-regulation. Industry works alongside independent researchers to identify potential sources for concern, and rigorously investigates the potential for harm present in them long before any external bodies think to get involved. If something is determined to be potentially problematic, industry itself imposes rules for what is acceptable and isn't acceptable in an evidence-based manner – and follows them.

If this can't happen, and I think we are reaching a point of no return, then we reach a situation where the industry is forced to abide by external input: Bodies from the outside reach into the games industry and tell it what it can/can't do. This is not optimal. And it is still I think preventable.

The reason why this situation is particularly sub-optimal is in regard to specific regulation. A lack of industry cooperation, data sharing, and general interest means that most questions regarding loot box effects are pretty much entirely unanswered. Academia also bears some culpability here – people have been asking the wrong questions about games for decades."

This argument is, it seems to me, pretty irrefutable. To anyone who isn't a games developer working on products that include loot boxes, the data and the truths it contain are an unknown quantity. It will remain that way to the committees who may regulate them, perhaps overly harshly or clumsily, until there's either an industry-wide awakening about Doing The Right Thing by opening up to independent scrutiny of its practices, or failing that a future combination of whistle-blowing and legislation forces the issue.

To rather over-simplify Dr Zendle's point, the industry examining itself is always going to produce more effective results because the people within the industry are best-placed to know what to look for. It's self-evident.

But in a world where FIFA Ultimate Team is responsible for over a billion of EA's revenue, companies inevitably drag their feet. And that's why we're here. The problem, to my mind, isn't whether loot boxes are banned for a link to gambling. It's whether loot boxes are banned for a link to gambling that is not evidenced and not understood well enough, because that reflects badly on the gaming industry, it reflects badly on legislators, and it ultimately is avoiding a question rather than answering it.

"In such a fuzzy world, how do we enact evidence-based policy? In the UK, we often treat various activities (including gambling) under the precautionary principle: this refers to the idea that the introduction of new products whose effects are unknown should be resisted. Under this model, the uncertain effects of loot boxes would demand a precautionary approach: and the most precautionary you can get isn't making odds visible, or setting spending limits in place. It's putting the whole thing under the heading of 'gambling' and exposing the industry to the extremely restrictive legislation that this entails.

Of course, this would seem like a crazy thing to do if we had a decent evidence base showing either that (a) loot box spending didn't lead to problem gambling (b) people didn't tend to spend more than they can afford on loot boxes (c) limit setting works... (etc etc etc yada yada yada). And again – this evidence is something that a functional self-regulating industry would have generated. The reason why we don't have it (in my opinion) is that self-regulation broke a while back for games, and we're now feeling the shockwave of that."Fortnite retrospectively removed loot boxes

To return to the perception issue of 'surprise mechanics' and Kinder Egg analogies, one of the most ridiculous moments in the loot box conversation came when Hawaii Democrat Chris Lee launched a push for legislation against "predatory behaviour", in which he identified Star Wars : Battlefront II as a "Star Wars-themed online casino, designed to lure kids into spending money” before another suit compared it to Joe Camel (a character used to sell cigarettes to children). You don't have to like EA or Battlefront to see that describing the game as a 'casino' or comparing the micro-transactions to cigarettes as a public health issue is a gross misrepresentation. That's a bad example but there are countless other mischaracterisations of what games and loot boxes are, and some of them stick.

Still, guess we shouldn't complain. Other companies like 2K seem determined to reinforce the casino rhetoric.

So here's where we are in the UK. You combine a limited evidence base with a Gambling Commission that is, in my opinion, not informed about the topic it is supposed to be regulating, and it should be clear there is the potential for the kind of regulation based on extreme examples and poorly understood research that would probably help no-one. But should things go this way, the industry has only itself to blame. In the above linked article I take aim at the quality of questions that industry reprentatives had to handle from our elected representatives; it's equally true that those industry people were not as revealing as they could have been, and that was how their employers wanted them to act in that scenario.

Self-regulation is the right idea, but the industry got it wrong. It has not yet proven itself capable of effective concerted action: individual companies do deserve credit for pro-actively moving away from certain business models, but many more are happy to push it to the limit.

In the absence of self-regulation, it is inevitable some form of external regulation will follow. Indeed, the Conservative 2019 manifesto contained a pledge to "tackle issues around loot boxes." The games industry has had every chance to bring its own perspective and evidence to the table. It has failed to do so. And if the games industry won't take responsibility for themselves, well, they can hardly complain when someone does it for them.

Read more: NHS Boss Accuses Loot Boxes of "Setting Kids Up For Addiction"

The DCMS Committee Says Loot Boxes Are "Games of Chance"