Thursday, March 7, 2013

Low Memory Automatic Reboot

Below is my solution to the memory leak problem that I discovered in the frame buffer interface used by my digital picture frame.  My first thought was to simply schedule a reboot at midnight, but the Raspberry Pi has no internal clock.  My picture frame is not connected to the network, so it never really knows what time it is. Also, the low memory system hang may occur in less than 24 hours.

The better solution is to monitor the amount of free system memory and reboot when it gets too low.  I use the /proc file system to access the system memory status, the awk command to pull out the value that I need, and a bash script to tie it all together.

lowmemreboot.sh

#!/bin/bash
declare -i fmem
while [ true ]
do
        fmem=` awk '/MemFree/ { print $2 }' /proc/meminfo`
        if [ "$fmem" -lt "10000" ]
        then
                echo FREE MEMORY IS LOW----FORCING REBOOT
                reboot
                exit
        fi
        sleep 60
done


Note that the command that awk runs is surrounded with single quotes.  The entire command is surrounded with back quotes.  This causes the output of the command to be saved in the fmem variable.

This script is run from /etc/rc.local the same way that wait-for-gpio-trigger.sh is run.  When the free system memory drops below 10K, the system will reboot and all is well again for a while.  Now I can just let the picture frame run forever and not worry about it.

I would like to see someone write a real slideshow program that does multiple transition types and supports controls such as pause and reverse.  I have too many more interesting projects on my mind, but maybe someone will find the time to do this.


13 comments:

  1. For $8 you could add wifi, and have NTP sync'd time. :-)

    Regardless, your solution is definitely the better way to solve your problem.

    ReplyDelete
    Replies
    1. This particular frame hangs on the wall in my office at work. Corp policy will not allow me to attach a "non-approved" device to the network.
      Besides, there isn't really room for anything to plug into the USB port the way that I mounted the hardware. See the pictures in the post about the Jumbo Digital Picture Frame.

      Delete
    2. I am building a much nicer looking frame for home. (The wife wants one.) It will have an all wooden frame - none of the plastic from the original monitor will be used.
      That one will have WiFi just to make it easier to use. The picture directory will be shared using Samba, so I can just drag and drop pix.

      Delete
  2. You could just use monit so you don't have to run your bash script. Monit would allow you to restart your server if cpu is too high, not enough memory, etc.

    check system
    if loadavg (1min) > 4 then alert
    if loadavg (5min) > 2 then alert
    if memory usage > 75% then alert
    if cpu usage (user) > 70% then alert
    if cpu usage (system) > 30% then alert
    if cpu usage (wait) > 20% then alert

    Something like this, or exec 'reboot' instead of alert. Found your blog searching Google for connecting my new raspberry with an alarm system, to start video recording/motion only when turning the alarm on.

    Might not be good enough in electronic to actually build the full alarm system (easy, one door and one infra red detector).

    ReplyDelete
  3. Hi Ted
    As regards to your memory leak, instead of rebooting the Pi, why not find the source of the problem?

    Nice site b.t.w.

    ReplyDelete
    Replies
    1. The leak is in the Frame Buffer Interface application (fbi) and have no desire to debug it. That can be someone else's problem.

      And I came up with a much easier fix that should have been obvious from the beginning. I just run the fbi program in a loop. When it crashes, the memory is freed up and then it just runs again. It has been running non-stop for about three weeks now.

      Delete
    2. Hi Ted

      Care to share the "... run fbi program in a loop" for us newbs?

      Cheers

      Chris

      Delete
    3. Run it in a loop in a shell script. Like this:

      # /bin/sh

      while [ true ]
      do
      fbi -noverbose -m 1920x1080 -t 10 /boot/PICTURES/*.jpg
      done

      Delete
  4. I am making a RPI picture frame for my mom. Can you give me an 18mo later update? Does your setup continue to be rock solid? Are you still running fib in a loop? Do you have a reboot when memory low routine running in the background too? Thanks!

    ReplyDelete
    Replies
    1. The system is still running rock solid. I never did put fbi into a loop, and the low memory monitor is not running. I think an update the the fbi program may have fixed the memory problem. The system says it has been up for 21 days and it doesn't seem to be losing memory.

      Delete
  5. I have a question if you don't mind.

    I am kind of a newbie. I want to know where do you save the code for 'lowmemreboot.sh'?

    Do I just paste it in the /etc/rc.local file?

    I am running a different project and I think your solution will help my problem.

    If you could please explain, like i am a 5 year old, how you run this.

    ReplyDelete
    Replies
    1. Save the script as a file named /etc/lowmemreboot.sh and call it by adding the following to /etc/rc.local

      /etc/lowmemreboot.sh &

      The & tells the system to let the script run in the background. Without this, it would never return and rc.local would hang causing the boot process to hang.

      Delete
    2. Also, the script must be made executable by the following command

      chmod +x /etc/lowmemreboot.sh

      Delete