CPU Isolation & CPU affinity In Linux

In Multi-processor architecture processors is directly get compared with processing power. Common notion is more processors means better performance but that is not always the case. In Most of Embedded chip-set multiple processing units are merged so one single chip can give you multiple processor functionality, processing units often referred as cores. To get an clear picture let’s take an example. In general 4 core machine we can see below cpu arrangement.

machine is having 2 cores and 4 logical processing unit which mean 4 parallel thread can run.

When we say CPU Isolation then we are isolating one or more cpu.

CPU isolation:

CPU isolation simply means isolating one of CPU from any process to be get loaded. To get this Part more clearly we will paint a scenario. Consider you have a embedded system/board which has multiple processors. If we took quad processor system as example each core has 4 cpu/processor unit in it. Considering a generic scheduler algorithm It is never defined like which process should be loaded on which processor/CPU. Every process can run on any processor until the specific process is pinned to the particular CPU. When any one processor is isolated it mean no other process or interrupt will be scheduled for that processor/CPU.

Before going further more details we need to understand how processors selects available process and load it and How isolation will be performed.

Every Processor is capable to load and execute operating sytsem core modules.

Every process descriptor is stored in process table with it’s current state. At the time of context switching scheduler module will be loaded on processor. Then scheduler takes control of processor and loads the ready to run processes from process table.

*It seems like cpu isolation can be achieved if some how we are managed to tell scheduler like do not load any process on this processor.

Scenario before CPU Isolation:

When scheduler code get loaded on processor it put a spin lock and search for process which is ready to execute. Once it found it will load that process and remove lock form process table. Like wise on every processor scheduler code will get loaded

Scenario After CPU Isolation:

If CPU isolation is activated for one of core then scheduler code will be never get loaded on isolated processor so no other process will get scheduled, until explicitly we tell to scheduler to load particular process on that processor no process will be loaded over it. Or in some architecture isolated CPU are removed/ Ignored in scheduler algorithms. it is basically Design dependent that how CPU is isolated from the process to be get loaded

How to isolate a Processor?

Experiment done on a generic Laptop/system where grub loader is there to load operating system.

1. To isolate cpu temporarily or for single boot we can do it by below steps,while Linux booting by grub loader user will see grub menu as shown in below picture.

2. Here we can press ‘e’ to edit the kernel boot command. Once we press ‘e’ a new window will appear. Which is actual grub script to load application. There generally you will see line like below.

linux bootvmlinuz{kernel-version} root=UUID=.... quite splash isolcpus=2

above command loads kernel image there we are manually passing “isolcpus=2” parameter to isolate CPU number 2. after this change we can boot the kernel by pressing F10 or as mentioned in grub note. After this once system boots cpu2 will be isolated and no user/kernel process is loaded. Only few cpu bounded process will run over it.

3. To check CPU isolation is done or not. Fire below command

cat /sys/devices/system/cpu/isolated Isolated CPU number you will get as output. If No Cpu is isolated then you will get an empty Output

Note: This change will be applicable for current boot only. Once system is rebooted settings will go away.

*To isolate CPU permanently for every boot done by grub follow below method.

1. In kernel boot options we can provide kernel boot parameter. “ isolcpus= ‘CPU Number’ ” In grub config we can mention this boot parameter. To update grub config mention this parameter in file “/etc/default/grub” mention parameter as isolcpus=2 in front of GRUB_CMDLINE_LINUX which says isolate cpu number 2.

Note: remember you can’t isolate all the processors.

For example: GRUB_CMDLINE_LINUX = “isolcpu=2” isolcpu=1,23 will pass as kernel boot parameter.

After changing above configuration update your grub to get updated Configuration by firing below command >> sudo update-grub2

Dedicating a CPU to a specific performance-critical process/task is desired.

Above Picture shows that CPU isolation is affected and cpu 1-3 are isolated.

Above is "top" utility view once cpu is isolated. All newly launched applications are getting tagged to processor 0 only. 1-3 processor will be only running idle_inject or cpu bounded processes only.

CPU affinity:

There are two types of CPU affinity. The first, soft affinity, also called natural affinity, is the tendency of a scheduler to try to keep processes on the same CPU as long as possible.scheduler, which has poor CPU affinity. This behavior results in the ping-pong effect. The scheduler bounces processes between multiple processors each time they are scheduled and rescheduled.

Hard affinity, on the other hand, is what a CPU affinity system call provides. It is a requirement, and processes must stick to a specified hard affinity. If a processor is bound to CPU zero, for example, then it can run only on CPU zero. It will not run on any processor Even if other CPUs are free and cpu0 is busy. It will wait until cpu0 gets free.

Benefit:

CPU affinity is optimizing cache performance.when processes bounce between processors they constantly cause cache invalidation, and the data they want is never in the cache when they need it. Thus, cache miss rates grow very large. CPU affinity protects against this and improves cache performance.Scheduling that process to execute on the same processor improves its performance by reducing performance-degrading events such as cache misses.

• CPU isolation can help you to observe your code performance and process switch latency on different processor scenarios.
• Once CPU is isolated, Isolated CPU is completely free to execute specific Pinned process. When we will pin a specific process to isolated CPU then cpu will execute solely pinned process only. Doing this will reduce the latency for pinned process and throughput will increase as no other process is there to disturb or take cpu away. For critical and important process we can do this way which will help to reduce latency and increase through put.
• CPU affinity is optimizing cache performance

Add on Information:

The CPU Affinity option in /etc/systemd/system.conf sets affinity for systemd itself, as well as everything it launches, unless their .service file overrides the CPU Affinity setting with its own value.

Configure the CPU Affinity option in /etc/systemd/system.conf.

#CPUAffinity=1 2

How to Pin a task to CPU:So that the process or thread will execute only on the designated CPU or CPUs rather than any cpu. This can be viewed as a modification of the native central queue scheduling algorithm in a symmetric multiprocessing operating system.

Use taskset command to pin the process.

The values in smp_affinity specify which CPUs handle that particular IRQ particular module irq handler can also be tagged. By changing value at proc/irq