Raspberry Pi setup power fail safe operation

About

Raspberry Pi is pretty famous for eating SD cards. While this can be annoying when it acts as media player, it can be costly and time consuming operation to replace when installed in remote location. If we assume that SD card is branded and good quality, most dangerous thing that could happen is unexpected power down. To avoid corrupt filesystem I suggest mounting SD card on Read Only mode. This is not perfect, but after testing for a few days I propose this solution.

Prepare

Prepare SD card. Format, flash latest Raspbian image and boot Raspberry. Find IP address and connect via SSH. You will need Internet connection to accomplish some steps.

• Format SD card with SD Formatter
• Download Raspbian Jessie Lite image (I used 4GB SD card)
• Flash downloaded image with win32diskimager
• Connect LAN, and USB cables and power up Raspberry pi
• Find out IP address with Advanced ip scanner
• SSH to Raspbery Pi

Setup raspi-config

$ sudo raspi-config

• 1 Expand Filesystem
• 2 Change User Password
• 5 Internatiolization options / I2 Chage Timezone / None of the above -> UTC
• 9 Advanced Options / A2 Hostname -> Kurokesu
• 4 Wait for Network at Boot -> Fast Boot Without waiting for network connection

Update OS

$ sudo apt-get -y update
$ sudo apt-get -y dist-upgrade
$ sudo reboot

Setup RO filesystem

Disable swap

$ sudo dphys-swapfile swapoff
$ sudo dphys-swapfile uninstall
$ sudo update-rc.d dphys-swapfile disable
$ sudo systemctl disable systemd-readahead-collect
$ sudo systemctl disable systemd-random-seed

Replace rsyslog with syslog

$ sudo apt-get -y install busybox-syslogd
$ sudo dpkg --purge rsyslog

Install UnionFS

$ sudo apt-get -y install unionfs-fuse

$ sudo nano /usr/local/bin/mount_unionfs
#!/bin/sh
[ -z "$1" ] && exit 1 || DIR=$1
ROOT_MOUNT=$(grep -v "^#" /etc/fstab | awk '$2=="/" {print substr($4,1,2)}')
if [ "$ROOT_MOUNT" != "ro" ]; then
/bin/mount --bind ${DIR}_org ${DIR}
else
/bin/mount -t tmpfs ramdisk ${DIR}_rw
/usr/bin/unionfs-fuse -o cow,allow_other,suid,dev,nonempty ${DIR}_rw=RW:${DIR}_org=RO ${DIR}
fi

$ sudo chmod +x /usr/local/bin/mount_unionfs

Edit boot script

$ sudo nano /boot/cmdline.txt

add ro:  ... elevator=deadline rootwait ro ...

Edit fstab

$ sudo nano /etc/fstab

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    ro                0       2
/dev/mmcblk0p2  /               ext4    ro,noatime        0       1
mount_unionfs   /etc            fuse    defaults          0       0
mount_unionfs   /var            fuse    defaults          0       0
none            /tmp            tmpfs   defaults          0       0

Prepare directories

$ sudo cp -al /etc /etc_org
$ sudo mv /var /var_org
$ sudo mkdir /etc_rw
$ sudo mkdir /var /var_rw

Create script to mount filesystem in rw mode

$ sudo nano /home/pi/rw.sh

#!/bin/bash

# remount root rw
mount -o remount,rw /

# prapare target paths
mkdir -p /chroot
mkdir -p /chroot/{bin,boot,dev,etc,home,lib,opt,proc,root,run,sbin,sys,tmp,usr,var}

# mount special filesystems
mount -t proc proc /chroot/proc
mount --rbind /sys /chroot/sys
mount --rbind /dev /chroot/dev

# bind rw directories
for f in {etc,var}; do mount --rbind /${f}_org /chroot/$f; done

$ sudo chmod +x /home/pi/rw.sh
$ sudo reboot

After completing all steps you should have safer system to work with. In order to make changes ensure power is stable and mount filesystem with script /home/pi/rw.sh. Reboot after you made all changes.

Issue #1 reboot

There are some issues with reboot script and after normal reboot, Raspberry Pi hangs with last log entry

[ OK ] Reached target Shutdown.

We have read only file system, so it is safe to reboot not nicely with command:

sudo reboot -d -f -i

Issue #2 crontab

Crontab is f*cked up after these modifications, so please make sure to implement your own. I ended with custom python script which is started from /etc/rc.local