Slowness after updating Fatdog

In late 2019 / early 2020, a CPU vulnerability was found that supposedly can leak data. You may have heard of it. It is called by its popular name, SPECTRE.

Because this vulnerability happens in the CPU, it cannot be patched and fixed. Instead, it has to be "worked around", that is, software must learn how to not use certain CPU features which supposedly can lead to the vulnerability being exploited.

The software in question is mainly the kernel of the operating system. This also includes Linux kernel, and Linux kernel releases after that have "workarounds" of these kinds; so that vulnerabilities can no longer be exploited.

Problem solved?

Not quite. The CPU features which are supposedly vulnerable, are those that have powered the spectacular compute performance leaps in the last decade or so. Disabling / not using these features is the same as undo-ing all those performance improvements.

How much performance loss are we talking about?

Depending on which features you want to avoid, it could be anything between 1% to 30% - give or take a few percents. To make things worse, after the first CPU vulnerability was found in early 2020, more vulnerabilities of similar nature were found; and "securing" them meant disabling more CPU features or adding awkward program constructs which (guess what) reduce performance by a lot.

But is countering these vulnerabilities worth such performance degradation?

The answer is, of course, it depends (with lots of caveats).

Most CPU vulnerabilities found are weaknesses in terms of "timing attack". By careful timing of certain CPU instructions, one can guess what data are driving those instructions. In theory, this can be used to, say, read a protected region of memory, which perhaps contains a password - and then the password can be stolen.

Is this really a big deal? Well, the answer is, yes or no. It depends. For example, if you are running a server with 100 VM in it, you'd probably be not smart to ignore this (one of the VM could be used to guess the hypervisor password). Or if you know that you're targeted by a national security agency of your country for whatever reasons, perhaps it pays to be paranoid as well.

But otherwise, there are so many ifs before this can happen. Plus, perhaps there is an easier way for people to lose their password (just ask them nicely for it? Many people fall for phishing attacks - e.g. a fake email asking them to reset their password ...)

I won't go further into the merit of why you should or should not ignore these weaknesses, because it obviously depends on your situation.

Instead, I will just say this. Linux prefers to err on the side of caution. And this means that by default, the workarounds - or mitigations as they are called - are ALL enabled by default to protect you from these vulnerabilities, at the cost of lower performance.

Every kernel released after 2020 has mitigations to various degrees, newer kernels obviously have more mitigations and are thus slower.

This is one of the most prominent reasons why newer Fatdogs (which use newer kernels) are slower.

Now, if you like the warm feeling of knowing that you're safe from all the prying eyes; and you don't mind the slowdowns, you can sit back and relax, there is nothing more you need to do.

If you, however, prefer to live on the risky edge and strive for the best performance at the cost of being vulnerable, then you can choose to disable these mitigations and claim back some of the lost performance.

There are many vulnerabilities, and Linux kernel gives you the option of fine-tuning which mitigations you want to enable/disable. You pass these options on the kernel command line (similar to how you pass the boot parameters). For the details you can see them here. Just search for "mitigations" and you will find them.

Alternatively, you can just disable ALL mitigations and get back the most possible performance. This is done by passing mitigations=off into the boot parameters.