Jul 14, 2013

How to setup Raspberry Pi for Foscam IP Camera

I explained how to store video data from Foscam IP Camera on Raspberry Pi. Now I am going to show you how to wire them all together.

As I explained, Foscam IP Camera, FI8910W, has motion detection feature in it. When any motion is detected, it will either send an email or upload a few still images to FTP server. A problem is that it doesn't store video data. That's what I am trying to do in this article.

The trick is that on the camera side it will do the motion detection and uploading files to FTP. When FTP connection is requested, it actually interact with a fake ftp server and the fake ftp server will trigger the video recording with a script that I explained.

Here is a big picture of how whole thing works.
  1. It opens a fake ftp port and wait for FTP request.
  2. Camera request FTP login.
  3. Start recording video stream
  4. Wait for another FTP request to identify whether or not there is motion going on still.
  5. If FTP access is request within a given time, it keeps recording the video stream.
  6. If FTP access is not requested within a given time, it stops recording.
The setting on the camera side is already explained here.

A fake ftp server for shell script version is here:
#!/bin/sh
CUT=/usr/bin/cut
TR=/usr/bin/tr

if [ ! -x $CUT -o ! -x $TR ]; then
        echo "*** some of utilities are not found or not executable ***"
        exit 1
fi

echo "220 Welcome message."
read read_id
if [ ! "`echo $read_id | $CUT -d ' ' -f 1`" = "USER" ]; then
        echo "530 Invalid login information..."
        exit 2
fi
ID=`echo $read_id | $CUT -d ' ' -f 2 | $TR -d ^M`

echo "331 Please specify the password."
read read_pw
if [ ! "`echo $read_pw | $CUT -d ' ' -f 1`" = "PASS" ]; then
        echo "530 Invalid login information..."
        exit 3
fi
PW=`echo $read_pw | $CUT -d ' ' -f 2 | $TR -d ^M`

echo "230 Guest login ok, access restrictions apply."

read read_typei
if [ ! "`echo $read_typei | $TR -d ^M`" = "TYPE I" ]; then
        exit 4
fi
echo "200 Type Set to I"

echo ID=$ID >&2
echo PW=$PW >&2
I placed it here: /home/pi/script/foscam_fake_ftp.sh
But you can place it wherever you want.

In the script, a special character, "^M", is not two characters but one character. In VI, you can type in the character by pressing "Ctrl+V Ctrl+M". The special character was entailed when the camera send text message so I had to remove them in the script.

A difference compare to the previous Windows batch file version is that the ID and Password information given to the fake ftp server will be used as a login information to access the video data. This time, we don't care whether it is Passive or not.

Here is a configuration file that will be placed at /etc/foscam.conf
#!/bin/sh
fake_ftp_if=wlan0
fake_ftp_port=2122
foscam_fake_ftp=/home/pi/script/foscam/foscam_fake_ftp.sh
foscam_record=/home/pi/script/foscam/foscam_record.sh
foscam_listener=/home/pi/script/foscam/foscam_listener.sh
foscam_cron=/home/pi/script/foscam/foscam_cron.sh

record_basedir=/home/pi/camera
recording_ip_dir=/var/run/foscam.d
#recording_ip_dir=/home/pi/script/foscam/var

alarm_wait_interval=60
cron_sleep_interval=2
file_ext=.mkv

LISTENER_PIDFILE=/var/run/fld.pid
CRON_PIDFILE=/var/run/fcd.pid

WGET=/usr/bin/wget
GREP=/bin/grep
CUT=/usr/bin/cut
MKDIR=/bin/mkdir
DATE=/bin/date
GSTLAUNCH=/usr/bin/gst-launch-1.0

NCAT=/usr/bin/ncat
GREP=/bin/grep
CUT=/usr/bin/cut
SLEEP=/bin/sleep
PS=/bin/ps
KILL=/bin/kill
DATE=/bin/date
IFCONFIG=/sbin/ifconfig
TAIL=/usr/bin/tail
RM=/bin/rm

LS=/bin/ls
Now we need a listener script here:
#!/bin/sh
. /etc/foscam.conf

if [ "$foscam_fake_ftp" = "" -o ! -x $foscam_fake_ftp ]; then
        echo "*** fake ftp server is not found or executable: $foscam_fake_ftp ***"
        exit 0
fi
if [ "$foscam_record" = "" -o ! -x $foscam_record ]; then
        echo "*** fake ftp server is not found or executable: $foscam_record ***"
        exit 1
fi
if [ ! -x $NCAT -o ! -x $GREP -o ! -x $CUT -o ! -x $SLEEP -o ! -x $PS -o ! -x $KILL -o ! -x $DATE -o ! -x $IFCONFIG -o ! -x $TAIL -o ! -x $RM ]; then
        echo "*** some of utilities not found ***"
        exit 3
fi

fake_ftp_ip=`$IFCONFIG $fake_ftp_if | $GREP inet\ addr | $CUT -d ":" -f 2 | $CUT -d " " -f 1`
#echo fake_ftp_ip=$fake_ftp_ip

while [ true ]; do
        echo [`$DATE`] Waiting for a new request...
        ncat_result=`$NCAT -v -l -c $foscam_fake_ftp $fake_ftp_ip $fake_ftp_port 2>&1 3> /dev/null`
        if [ ! $? = 0 ]; then
                echo "*** Fake ftp cannot start: $fake_ftp_ip:$fake_ftp_port ***"
                exit 5
        fi

        #echo "$ncat_result"
        IP=`echo "$ncat_result" | $GREP "Ncat: Connection from " | $GREP ":[0-9]" | $CUT -d " " -f 4 | $CUT -d ":" -f 1`
        ID=`echo "$ncat_result" | $GREP ID= | $CUT -d "=" -f 2`
        PW=`echo "$ncat_result" | $GREP PW= | $CUT -d "=" -f 2`
        echo [`$DATE`] Streaming requested from $ID\@$IP...
        #echo PW=$PW

        info_file=$recording_ip_dir/$IP
        if [ ! -f $info_file ]; then
                record_result=`$foscam_record $IP $ID $PW`
                #echo "$record_result"

                alarm_upload_interval=`echo "$record_result" | $GREP alarm_upload_interval= | $CUT -d "=" -f 2`
                record_interval=`expr $alarm_upload_interval + $alarm_wait_interval`
                gstreamer_pid=`echo "$record_result" | $GREP pid= | $CUT -d "=" -f 2`

                echo "gstreamer_pid=$gstreamer_pid" > $info_file
                if [ ! -f $info_file ]; then
                        echo [`$DATE`] info file coundn\'t be created: $info_file
                        echo [`$DATE`] Recording streamming video from $ID\@$IP for $record_interval seconds...
                        $SLEEP $record_interval
                        $KILL $gstreamer_pid
                else
                        record_started=`$DATE +%s`
                        finish_recording_time=`expr $record_started + $record_interval`

                        echo "record_started=$record_started" >> $info_file
                        echo "alarm_upload_interval=$alarm_upload_interval" >> $info_file
                        echo "record_interval=$record_interval" >> $info_file
                        echo "finish_recording_time=$finish_recording_time" >> $info_file
                fi
        else
                more_recording_requested=`$DATE +%s`
                record_interval=`$GREP record_interval= $info_file | $CUT -d "=" -f 2`
                finish_recording_time=`expr $more_recording_requested + $record_interval`

                echo "more_recording_requested=$more_recording_requested" >> $info_file
                echo "finish_recording_time=$finish_recording_time" >> $info_file
        fi
done
I placed it at  /home/pi/script/foscam/foscam_listener.sh

In order to get this script working, you will need three other scripts: foscam_record.sh, foscam_cron.sh and foscam_fake_ftp.sh, which is shown above.

You can get foscam_record.sh from this previous article of mine. But you must get "gst-launch-1.0" running as a background process. It can be easily done by adding "&" at the end of the line.

Another script, foscam_cron.sh, will figure out when to stop recording. This script should be running all the time as a daemon or cron-job.
#!/bin/sh
. /etc/foscam.conf

if [ ! -d $recording_ip_dir ]; then
        echo "*** recording_ip_dir does not exist: $recording_ip_dir ***"
        exit 1
fi

if [ ! -x $GREP -o ! -x $LS -o ! -x $CUT -o ! -x $PS -o ! -x $RM -o ! -x $SLEEP -o ! -x $DATE -o ! -x $KILL ]; then
        echo "*** some of utilities not found ***"
        exit 3
fi

while [ true ]
do
        for info_filename in `$LS $recording_ip_dir`
        do
                info_file=$recording_ip_dir/$info_filename
                #echo info_file=$info_file

                gstreamer_pid=`$GREP gstreamer_pid= $info_file | $CUT -d "=" -f 2`
                if ! $PS $gstreamer_pid 2> /dev/null 1> /dev/null
                then
                        echo "*** gstreamer not found: $gstreamer_pid ***"
                        $RM $info_file
                        continue
                fi

                cur_time=`$DATE +%s`
                finish_recording_time=`$GREP finish_recording_time= $info_file | $TAIL -1 | $CUT -d "=" -f 2`
                if [ $finish_recording_time -lt $cur_time ]; then
                        echo Time to stop: $info_file ...
                        $KILL $gstreamer_pid
                        $RM $info_file
                fi
        done

        $SLEEP $cron_sleep_interval
done
Now I want it to start at boot time automatically.
You will have to make a new file at /etc/init.d/foscamd
#!/bin/sh

### BEGIN INIT INFO
# Provides:          wrice.blogspot.com
# Required-Start:    $network $local_fs $remote_fs
# Required-Stop:     $network $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: start Foscam daemons (fcd and fld)
### END INIT INFO

. /etc/foscam.conf

# See if the daemon is there
test -x $foscam_listener || exit 0

. /lib/lsb/init-functions

fake_ftp_ip=`$IFCONFIG $fake_ftp_if | $GREP inet\ addr | $CUT -d ":" -f 2 | $CUT -d " " -f 1`

remove_stale_pid_file () {
        pid_file=$1
        # Wait a little and remove stale PID file
        sleep 1
        if [ -f $pid_file ] && ! ps h `cat $pid_file` > /dev/null
        then
                rm -f $pid_file
        fi
}

case "$1" in
        start)
                log_daemon_msg "Starting Foscam daemons"

                log_progress_msg "fcd"
                if ! start-stop-daemon --start --quiet --background --make-pidfile --pidfile $CRON_PIDFILE --exec $foscam_cron
                then
                        log_end_msg 1
                        exit 1
                fi

                log_progress_msg "fld"
                if ! start-stop-daemon --start --quiet --background --make-pidfile --pidfile $LISTENER_PIDFILE --exec $foscam_listener
                then
                        log_end_msg 1
                        exit 1
                fi
                sleep 1
                if ! ps h `cat $LISTENER_PIDFILE` > /dev/null
                then
                        log_end_msg 1
                        exit 1
                fi
                log_end_msg 0
                ;;
        stop)
                end_msg=0
                log_daemon_msg "Stopping Foscam daemons"
                log_progress_msg "fld"
                if ! start-stop-daemon --stop --quiet --pidfile $LISTENER_PIDFILE
                then
                        end_msg=1
                fi
                echo | /usr/bin/ncat --send-only $fake_ftp_ip $fake_ftp_port
                remove_stale_pid_file $LISTENER_PIDFILE

                log_progress_msg "fcd"
                if ! start-stop-daemon --stop --quiet --pidfile $CRON_PIDFILE
                then
                        end_msg=1
                fi
                remove_stale_pid_file $CRON_PIDFILE
                log_end_msg $end_msg
                ;;
        reload|restart|force-reload)
                $0 stop
                sleep 1
                $0 start
                ;;
        status)
                status="0"
                status_of_proc -p $LISTENER_PIDFILE $foscam_listener foscamd || status=$?
                exit $status
                ;;
        *)
                echo "Usage: /etc/init.d/foscamd {start|stop|reload|restart|force-reload|status}"
                exit 1
                ;;
esac

exit 0

Once foscamd is created, you will need to run this command to generate symbolic links for each boot sequence:
sudo update-rc.d foscamd defaults
Now when you restart your computer the listener will start automatically.

9 comments:

Sérgio de Oliveira said...

Congratulations for this post. I'm searching a way to record videos from my Foscam cameras. Now, it's working.
I've tried to save in avi files, but it doesn't work. The saved videos only open in mplayer. i would like to open in Windows Media Player.
I have success to save in avi, using avcon, but without hw h.264 decoder. Avcon use about 35% of processor to encode in MPEG4 with 100K bitrate.
Regards!
Sérgio

Anonymous said...

ok I may be asking a dumb question here.. where do you pass the parameters to set the ip address of the camera, user name and password?

(bash script/raspberry novice)

Jay said...

I hate to say but the scripts inthis article is not very stable and if you don't understand the script, I don't recommend it.

Anonymous said...

THis is awesome ! Did you try to tackle issue of Fsocams not having secure protocol (http vs https). I saw some attempts to make pi act as secure proxy to unsecured foscams but depending on mobile software (e.g. foscampro) these fail due to unability to have extra // in paths.

Jay said...

I haven't thought about https. I am starting to think about it now... Thanks for the idea.

Anonymous said...

Hello Jay,

thank you for this great article! Everything works fine except the ftp connection:
When I try to test the ftp server from the webcam admin panel, I get the following message:
"Test ... Failed
Error in PASV mode. Please be sure the server support PASV mode"
But at the test time my raspberry starts to record the stream.
Will it work in alarm mode now? I wasn't able to test it.

Thank you very much!

Regards,

Manuel

Jay said...

Thank you for your interest on my article. but I am not maintaining this code anymore. I am not even using it. lol. I am planing to make a more stable program that does the same. But it may take some time for me to finish it because I am learning Python still.. lol

Jay said...

I made a new C++ program that does better job in this. You can find it from here: http://raspberrypiprogramming.blogspot.com/2014/09/foscam-video-capture-server-at-motion.html

Unknown said...

I most likely appreciating each and every bit of it. It is an incredible site and decent impart. I need to much obliged
see this