[Research] ZRAM to use compressed swap in memory (increase performance when memory usage is high)

[openoms]

ZRAM is a ~3x compressed swapfile in memory. Helps to avoid using the swap on the disk which even with an SSD can lead to a complete halt when the computer is running out of memory.
Article:
https://haydenjames.io/raspberry-pi-performance-add-zram-kernel-parameters/
Generic install script:
https://github.com/foundObjects/zram-swap

Relevant for 1-2GB RAM models already now.

The best test would be to install ZRAM on a device which already reproducibly experiencing slowdowns and/or freezes and see if it makes a difference.

[GoetzGoerisch]

RPI3, HDD during IBD:

v1.7.2rc1

Before:

admin@192.168.178.5: ₿ free -m
               total        used        free      shared  buff/cache   available
Mem:             959         213          40           1         706         687
Swap:           3137         349        2788
admin@192.168.178.5: ₿ sudo cat /proc/swaps 
Filename                                Type            Size            Used            Priority
/mnt/hdd/swapfile                       file            1984508         354540          -2
/dev/zram0                              partition       1228548         4452            15
admin@192.168.178.5: ₿ vmstat -s
       982840 K total memory
       211788 K used memory
       206780 K active memory
       629852 K inactive memory
        52120 K free memory
         8592 K buffer memory
       710340 K swap cache
      3213056 K total swap
       361296 K used swap
      2851760 K free swap
        77607 non-nice user cpu ticks
            2 nice user cpu ticks
        66127 system cpu ticks
       573418 idle cpu ticks
       251014 IO-wait cpu ticks
            0 IRQ cpu ticks
         7894 softirq cpu ticks
            0 stolen cpu ticks
     19807001 pages paged in
      1990289 pages paged out
        33186 pages swapped in
       107756 pages swapped out
     33451365 interrupts
      6807495 CPU context switches
   1642783597 boot time
        62686 forks

After:

admin@192.168.178.5:~ ₿ free -m
               total        used        free      shared  buff/cache   available
Mem:             959         317          49           3         592         581
Swap:           3137         343        2794
admin@192.168.178.5:~ ₿ sudo cat /proc/swaps 
Filename                                Type            Size            Used            Priority
/mnt/hdd/swapfile                       file            1984508         0               -2
/dev/zram0                              partition       1228532         355328          15
admin@192.168.178.5:~ ₿ vmstat -s
       982828 K total memory
       323424 K used memory
       115460 K active memory
       536984 K inactive memory
        49592 K free memory
         3576 K buffer memory
       606236 K swap cache
      3213040 K total swap
       356352 K used swap
      2856688 K free swap
        37270 non-nice user cpu ticks
            4 nice user cpu ticks
        24942 system cpu ticks
        90050 idle cpu ticks
        51643 IO-wait cpu ticks
            0 IRQ cpu ticks

[openoms]

@GoetzGoerisch thank you for testing, but not sure what changed between the two blocks? There is zram set in both.
Did you experience any performance improvement with it?

[GoetzGoerisch]

@openoms
Before: zram was not active, although installed.
After: zram was active with the parameters according to the article above.

Difference is, that it does not need to write to swapfile:

admin@192.168.178.5:~ ₿ sudo cat /proc/swaps 
Filename                                Type            Size            Used            Priority
/mnt/hdd/swapfile                       file            1984508         0               -2
/dev/zram0                              partition       1228532         804956          15

Which was the case before, running of a HDD this was unusable, and crashing daily.
I'm trying this on a PI3 because this is the only device available for testing at the moment. Therefore I gave it a try and you mentioned a low end device, would be best for testing.

The PI3 is now usable and slowly but stable doing the IBD with approx 0,5-1block/s.

These are my initial observations, will continue to add my observations.

[openoms]

I have set this on RPi-s, on a Debian laptop, and a server with 32 GB RAM and all are working great.

Having the swap in the RAM does not only speeds things up, but reduces the wear in the disk.

@rootzoll I think this should be part of the default setup especially in the view of: #2859

The script uses the half of the RAM for the lz4 compressed swap.
On my 8GB RPi the 4GB zram gives a 10GB swap device which should be enough for all situations and there is still the additional 4GB swap on the disk, but set to low priority:

sudo cat /proc/swaps
Filename				Type		Size		Used		Priority
/mnt/hdd/swapfile                       file		4194300		867988		-2
/dev/zram0                              partition	10064516	0		15

For a quick install:
(note: needs a reboot after a kernel update to work)

# install zram to 1/2 of RAM, activate and prioritize
git clone https://github.com/foundObjects/zram-swap.git
cd zram-swap 
sudo ./install.sh

# make better use of zram
echo "\
vm.vfs_cache_pressure=500
vm.swappiness=100
vm.dirty_background_ratio=1
vm.dirty_ratio=50
" | sudo tee -a  /etc/sysctl.conf 

# apply
sudo sysctl --system

Check:

sudo cat /proc/swaps

After this the zram will be prioritised and the disk swap will gradually empty.

Can unload and switch it off completely with:

sudo swapoff /mnt/hdd/swapfile

To remove the zram:

cd zram-swap
sudo ./install --uninstall 
sudo rm /etc/default/zram-swap 

[rootzoll]

@openoms do you want to target this for soonish 1.8 release or push it up to v1.8.1?

[openoms]

Had no bad experiences with this so with a short install script it could be part of the deafult setup in the next release.

[rootzoll]

OK lets go for the v1.8 then :)

[openoms]

zswap can be a good addition also (paired with zram - not instead of), but still looking to find a good way to install it on the RaspberryOS https://docs.armbian.com/User-Guide_Fine-Tuning/#swap-for-experts

[rootzoll]

@openoms so you like to add ZRAM to the v1.8.0 build script? Would be ghood to have it in an early RC so that it gets some more test time before release. Otherwise we could add it as an SETTINGS option in v1.8.1 first.

[openoms]

added a simple install script which relies on the widely used script https://github.com/foundObjects/zram-swap

Would be ok to just activate it for Fulcrum, but low memory devices can benefit to be on by default.
Shall the blitz.zram.sh called from the build_sdcard.sh?

[rootzoll]

Would be ok to just activate it for Fulcrum, but low memory devices can benefit to be on by default.
Shall the blitz.zram.sh called from the build_sdcard.sh?

On building the sd card we dont know the memory of the future device - so if we dont want to make it default for all devices I would not put it in the build sd card yet. The better place would be in the provison process - detect low memory devices there and then trigger install - like here: https://github.com/rootzoll/raspiblitz/blob/55215299e1f9ab6c10e09b255594d4c3e36c176b/home.admin/_provision_.sh#L69

[openoms]

as discussed above all devices can benefit from the lower disk swapping so added ZRAM for all to _provision_.sh in #3102

[rootzoll]

PR merged. Setting to final testing.

[rootzoll]

So far no problems during testing ... closing issue for release.