The User Is Always The Boss

The desktop Linux world was in for a surprise when Epic Games announced they brought their storefront to Linux, and its flagship live service "Fortnite" will soon follow.

Wow. It took them a decade to finally vibe code a shopping cart. Amazing.

I'm not sure how vibe-coding is a flex? This reads like begging aimed at shareholders. We use AI, we're hip, we're the future! We're going places and we're valuable!

It is as clear as ever that Tim Sweeny continues to completely fail to understand why Valve has such a loyal following from gamers everywhere. The overhead of running Steam is well worth the conveniences it provides that competitors don't offer. If you disagree, then you aren't using any of the other launchers either - they all center around some social network or some other reason to get you to use their software, all of them are hefty, and most of them are proprietary. Well, okay, there ARE very vocal weirdos (astroturfers) who use a centralized storefront that isn't Steam and try to argue that Steam alone is "bloat". I think it's also curious that Epic never once brings up the Steam Deck or SteamOS, which people will undoubtedly use the new Epic store with.

But I digress. There's bigger issues with this that people are not talking about.

After years and years of Linux not being able to run some mainstream live service games, the following two are, predictably, the predominant opinions you'll see from Linux users:

1. Who the fuck cares? I don't play Fortnite and I never will.
2. Finally! This will get Fortnite players to use Linux, and convince other developers/publishers to adopt Linux as well!

It's understandable. We've conditioned ourselves to want support from proprietary software vendors because it's long been the thing holding us back in the eyes of the mainstream. And if you're already using Linux, you're happy with what you have. Fortnite is not a good game either, speaking from thousands of hours of hands-on experience. There are good things about it, but it's been swallowed by the live service forever-death.

I don't want Fortnite to come to Linux. The problem is not really about Fortnite, but one of Fortnite's anticheats: EasyAntiCheat.

See, the reason we've needed support from developers to make these games work on Linux despite not getting an official port is because they use kernel level anticheat. This has a lot of problems which I'll touch on in a later section, the biggest of which is that The User Is Always The Boss. For now, all you need to know is that Linux and Windows have different kernel APIs and different architecture. WINE handles most of these, but privileged anticheats make a lot of assumptions about the software you're running that don't hold up under WINE. No amount of coercion will fix this without neutering the anticheat entirely, which is a cat-and-mouse game as hackers attempt the same thing, and anticheat vendors push updates to delay them further.

Valve asked developers to make a compromise. With the developer's permission, EAC and Battleye will use a special, unprivileged anticheat for Linux when they detect that they are running under Proton or WINE. Obviously, this defeats the whole point of having an anticheat in the kernel. If a cheater cannot crack a kernel module, they can install Linux instead.

You might then be surprised to learn that, today, almost all cheating happens on Windows. As it turns out, kernel level anticheat is total bullshit (The User Is Always The Boss). Also, pasters buying cheat software from reverse engineers don't want to switch operating systems just to play a video game. Nobody does.

Epic is not content with accepting this reality. They want their kernel level anticheat on Linux. Which means they need to trust that the kernel has not been tampered with by the user. As Windows and macOS have shown us, this is literally impossible, but that has not stopped them from trying. EACv2 is Epic's shot at bringing kernel anticheats to Linux. It will require a verified kernel running in secure boot, signed by a key that the anticheat trusts.

(I am aware that EACv2 technically does not run in the kernel and does not carry all of the security implications that entails, but the security requirements go that far, so I am calling it a kernel level anticheat.)

This is not only going to get people to install malware on potentially millions of Linux systems in the future, but it actually hurts Linux adoption.

What is Secure Boot?

Secure Boot is a feature on many platforms, including most UEFI-based IBM PCs, that prevents the computer from booting if the EFI executable is not signed by a trusted party. The firmware has a list of trusted public keys which it will use to verify the signature of the software it runs. Linux and Windows NT are just EFI programs, as are their bootloaders.

The original purpose of this was to deter rootkits. If you, the user, want to know that you're actually booting your operating system and not some fake chainloader that'll infect you, this is great. You can self-sign your kernel and bootloader, or use a key from a vendor you trust who signs them for you (Microsoft, Google, or Apple, in most cases). Secure Boot is a big part of anti-forensics technology on smartphones and Mac computers.

However, many developers of proprietary software have turned this around and tried to use it as a tool to verify system integrity against users. You need secure boot and a proprietary, well-known operating system from a "trusted" vendor to enjoy things like Widevine, Vanguard anticheat, et cetera. This doesn't work because The User Is Always The Boss. He or she has the physical hardware and can, sans some kind of exploit, easily ensure that it only runs the code they authorized. As an outside third party, you can't. You just have to hope that the computer reports its own condition honestly and faithfully, and that whatever walls you put up will annoy people enough they won't try to change that. Which, as history has taught us, has never worked - not for long anyway.

Treadmill Work

In the 1999 Christmas season, Eric Raymond and John Carmack both theorized in an essay on Raymond's website that in the future, we would reach a point where anticheat software will have to rely on observing player behavior in-game, and we will have to accept that closet cheaters go undetected if we hope not to ban innocent players. The point of the essay was about security design in general and how it relates to the Free Software movement, not really about DRM in particular, but it's still very relevant and I would encourage everyone to read it.

We have already reached the point where people are buying external cheat devices, that use either AI vision systems or PCI DMA to cheat in games. The latter might be deterrable in part through repeated obfuscation of software updates (treadmill work), as we already do now. But many developers, such as Valve, are taking the machine-learning route. And before Valve's VACNET, they had Overwatch (the CS:GO human-moderated anticheat, not the Blizzard game). Overwatch was a system in older versions of CS:GO that sent anonymous match demos from reported players to volunteers, and those volunteers would vote on whether or not the player in the demo was cheating. VACNET was trained on years of Overwatch case files. Gamers like to whine about Valve and say they "don't care about CS", despite the obvious evidence to the contrary, and pretending they know everything without having original ideas is something the CS fanbase is truly the best at. But Valve knows what they are doing. There is a reason that the old school client-side VAC is only a userland sigchecker, and not a heuristic-scanning kernel module like EAC. They caught onto this inevitability years ahead of the rest of the industry, and are going the behavioral analysis route, as we knew would happen back in the Quake era. AnyBrain, another anticheat vendor, is rolling out a similar solution. One of my favorite shooting games THE FINALS currently uses it, with questionable success.

For anyone who hasn't read the above essay or doesn't have a solid grasp on computers, the reason we can't "just fix the cheater problem" is simply this: The User Is Always The Boss. As I've been saying throughout this article, they control their computer, and you don't. And if you think you, the developer, control their computer with your software, you are begging for a hacker to prove you wrong. Once a hacker does it, they spread the knowledge and the tools to the less dedicated users. Because a video game requires participation from the user, meaning it needs data from their machine, it is fundamentally impossible to guarantee the data they give you came from where you think it did. The only way to almost promise this is if they're playing the game entirely on your computer, such as with cloud streaming. "Trusted Computing" as we've come to know it today through closed platforms and DRM is a fucking circus act. We will go as far as electrically glitching the chips to make them more vulnerable if we have to.

It's not like Epic Games does not know better. They even admit this, though far less bluntly, in a "we know what we're doing is wrong but we're going to say it like it's not" kind of way:

I trust my own judgement as a programmer, and of the many hackers before me, way more than I trust gamers and salesmen. And I have a theory. It's a bit radical, I admit, but hear me out.

Anticheat Software is not Real

I firmly believe that anticheat solutions are scams.

Think about it. The smartest engineers in the world, trying to stop hackers using methods that make absolutely no sense. It is an impossible problem that people throw millions of dollars at hoping it will magically be fixed. We've begun marketing games on their anticheats now. Players obsess over them, they crave them because we've normalized it. They don't have the prerequisite knowledge to understand that it's bullshit.

The developers of these anticheats do, though. They have to, cybersecurity is their job. They should know better than anyone that putting their networked code in the kernel is a security nightmare, and that it's all for nothing. Many of them are formerly cheat developers. No self-respecting software engineer with that kind of talent would bother.

Think of all of the money studios make, and how much of that goes to DRM and anticheat. Imagine how much better games could be if we stopped wasting all of that time and money on Treadmill Work. The anticheat industry is in the business of selling snake oil to developers, and convincing players that a multiplayer game cannot be fun without it. It's devious, sneaky shit.

A Real Solution?

Before live service games and centralized social media, community servers were the norm. You would find a group of strangers to play with, maybe join them regularly and become friends. This has bred a lot of braindead power-tripping admins and echo-chambers, but the flip side of this is that cheaters were a non-issue for most players. The server you were comfortable playing on had its way of dealing with them, and it was effective, because everyone looked after their own players. And if there was a disagreement, or the admins were pulling some shenanigans, you formed your own splinter group. This still happens, believe it or not. Some of my friends reading this might remember our time playing on Q3Retro, before the drama ensued.

Many of these games also had a wonderful "voting" feature, where you could vote to temporarily ban a player from the current match. This gets abused sometimes, but it's a godsend when it counts.

When I play Xonotic, cheaters don't bother me. When I play TF2, cheaters don't bother me. When I play Unreal, cheaters don't bother me. When I play CS:Source, cheaters don't bother me. When I play DOOM, cheaters don't bother me. And so on.

At worst, they are a temporary nuisance and a funny story to tell, and at best, they're really entertaining encounters. In games with awful security, like the older Call of Duty series, they're more silly than serious. In other games, it's fun to call them out and watch them have an emotional meltdown. I don't get mad or think "this game has a cheating problem", because I know that I can join any other server if I don't like it, and all is right in the world.

Cheaters only become a serious issue in games that promote long-term commitment or feel like you've wasted your time. I'll bring up THE FINALS again as an example. Even in casual modes, you're ranked on your performance. If you play against a cheater, or someone just playing the game unfairly, you don't progress as fast as you should have. If you want that Emerald World Tour badge in a reasonable amount of time, having fun is not enough. Some players don't care about these things, but they are there. An even better example would be a game like Tarkov or RUST, where a single death can straight up erase all of the fruits of your labor. The answer is not "just have fun, it's just a game", because fun can be different things to different people. If you like a longer challenge that rewards you, then that is fun. Trying really hard is not just something miserable people do to waste time, there's nuance to it.

The other big environment where cheaters are actually a serious issue is in esports, where money is on the line. But cheating in esports looks a bit different from cheating in the general gaming population for various reasons, and the anticheat software is less relevant.

Client-sided anticheat software is not the way. We need better moderation systems and healthier game design.

What does this have to do with Linux?

Earlier, I teased that this would hurt Linux adoption. Here's the part where I finally elaborate on that.

Remember Secure Boot? Did you know it is enabled by default on almost every PC motherboard when it leaves the factory? But without keys, it won't work. So most firmwares come with Microsoft's public keys automatically trusted to keep Windows users happy. Are you seeing the bigger problem now?

First of all, EACv2 can't accept self-signed kernels and bootloaders, that would defeat the purpose. This means that, in order to play Fortnite on Linux, you will be forced to use a distribution that Epic approves of, with a bootloader that Epic approves of. This almost guarantess that anyone not running RedHat software is shit out of luck. But it gets worse.

Operating systems cannot automatically enroll their own keys to be trusted at installation. If they could, any program with the right access would be able to, and that's a security risk. It would, again, defeat the purpose of secure boot. So to enroll a public key, the user has to go into their firmware settings and put their motherboard into a special mode of secure boot that will accept the new key. This is power user stuff. The kinds of Linux users that want to play Fortnite don't want to do this, they understandably will want a GUI that does it for them or some other handholding.

This leaves only one option: sign the operating system with Microsoft's public key. This means that Microsoft gets to decide what Linux distributions will run Fortnite. And every other game that uses the new EACv2 will be caught up in this too.

Microsoft, the company that, has throughout its entire history, thrived on first controlling the competition to destroy them. They even coined a term for it: "Embrace, Extend, Extinguish". It won't just be RedHat trying to do this anymore through systemd and FreeDesktop, it'll be real. The renaissance of Linux gaming will be over.

That's one hell of a monkey's paw. You will get Fortnite on Linux, but in return, no more freedom in Linux. You might as well install Windows at that point. It's also worth pointing out that EACv2 expects systemd. To what extent, I don't yet know. Maybe any service manager will work, but if it depends on libsystemd to function, that makes the problem even worse. In the best case scenario, this does nothing for Linux except maybe the corporate distros with OEM contracts. New users already get funneled into RedHat/Canonical land and this certainly won't help. Worst case scenario, Microsoft controls Linux and the sun explodes.

Say no to EACv2 before it's too late. Don't play any games that have it. Do your part, and let developers know this is not okay. The User Is Always The Boss, let's stop pretending otherwise.