Jun 2, 2013

Recording surveillance video from Foscam IP Camera without iSpy

*Edit ( Aug/18/2014 ): I made a C/C++ program that stores the stream video from Foscam IP camera with libvlc. It may work for you better. Check out the article here.

iSpy is a cool open source software that records surveillance video. Although it has several features, the features I was using was recording video/audio when motion is detected. It had been working very well and I was enjoying it a lot.

A little problem was that this software has to be running 24/7 and all of the video data must be streamed from IP camera to the computer that iSpy is running even when there is no motions are detected. Because the motion detection happens on the computer side not on the camera side, the computer must download all the video data from the IP camera via WiFi. I figure the video data isn't too big. It was about 7.3MByte per minute, which is 122KByte/sec.

Today I found that my IP Camera (Foscam FI8910W Wireless IP Camera) has motion detection feature already in it. So it made me thinking that when IP camera detect any motions, it can let my computer know that the recording has to start. This way, the WiFi bandwidth is saved and CPU power that were supposed to be spent on image processing is saved. I don't think neither of them are big deal but I started implementing my idea for fun.

The idea was that my Foscam Camera can upload image files onto FTP server when motion is detected. But it doesn't have any ways to upload video streaming data. So I am recording video streaming data when the FTP connection is requested, while the actual image uploading from the camera will be ignored.
The working scenario is like this:
  1. Foscam Camera monitors the scene.
  2. Foscam Camera detect motions
  3. When any motions are detected Foscam Camera try to connect to FTP server and upload image files.
  4. On the FTP server side, it ignores whatever is sent from Foscam Camera.
  5. But at the time FTP connection is requested, VLC is launched and VLC start recording video streaming data directly from Foscam Camera.

Settings on Foscam Camera

 "FTP Service Settings" looks like this.
  • "FTP Server" is the IP number of the computer that is going store the video streaming data.
  • "FTP port" is a fake number of this specific task. I will use a number "2121" for this article but you can change it for any numbers.
  • "FTP User" should be "camera". It is case sensitive like most of FTP login ID.
  • "FTP Password" doesn't matter for now. But you can make it count by changing the fake ftp server batch file later.
  • "FTP Mode" should be "PASV" for now.

The way I am explaining here is not going to use a real FTP server. I made a fake FTP server that pretends to be a FTP server. So User/Password doesn't matter at all.

"Alarm Service Settings" looks like this:
  • "Motion Detection Alarmed" should be checked.
  • "Upload image on Alarm" should be checked.
  • "Upload interval (seconds)" can be bigger than 60. This number is something you can tweak later. I don't recommend any number below 60.

These are all settings on Foscam Camera side.
Now when it detect any motions, the Camera will try to upload to the computer. If you have a real FTP server, it will be working. But you will get only images not audio/video.

Now I will explain computer side. This is little hacky and hard to understand but there is no way to harm the computer at all. You will need to install NetCat for windows and VLC for windows. My implementation is all based on Windows but if you are familiar with shell script, you can translate it by yourself because NetCat and VLC should be working on Linux/Mac too.

You need to install

Let me show you how VLC can record video stream directly. ( This example is taken from here )
"C:\Program Files\VideoLAN\VLC\vlc.exe" http://xxx.xxx.xxx.xxx:####/videostream.asf?user=UserName&pwd=Password --qt-start-minimized --no-qt-notification --run-time=TimeInSeconds :demux=dump :demuxdump-file=MyCamera1.asf vlc://quit
Other URL commands for Foscam IP Camera is here.

Now let's have a batch file that know where to store the video stream data and where to get from.
Save this batch file as "c:\batch\record_foscam.bat"
@ECHO OFF
SET IP=xxx.xxx.xxx.xxx
SET ID=userNameForCamera
SET PWD=passwordForCamera
SET run_time_in_sec=60

REM *** PATH should not have quotation marks ***
SET VLC=C:\Program Files\VideoLAN\VLC\VLC.EXE
SET ASF_BASEDIR=C:\surveillance\video\Foscam
SET YEAR=%DATE:~-4%
SET MONTH=%DATE:~4,2%
SET DAY=%DATE:~7,2%
SET HOUR=%TIME:~0,2%
SET MIN=%TIME:~3,2%
SET SEC=%time:~6,2%
SET ASF_FILENAME=%ASF_BASEDIR%\1_%YEAR%-%MONTH%-%DAY%_%HOUR%-%MIN%-%SEC%.asf

ECHO Recording %run_time_in_sec%seconds of Video stream on to %ASF_FILENAME%...
REM "%VLC%" http://%ID%@%IP%/videostream.asf^?user=%ID%^&pwd=%PWD%^&res=8^&rate=6 --qt-start-minimized --noqt-notification --run-time=%run_time_in_sec% :demux=dump :demuxdump-file="%ASF_FILENAME%" vlc://quit

"%VLC%" http://%ID%@%IP%/videostream.asf^?user=%ID%^&pwd=%PWD%^&res=8^&rate=6 --qt-start-minimized --noqt-notification --run-time=%run_time_in_sec% --sout=#transcode{vcodec=h264,acodec=mpga,channels=1}:standard{dst="%ASF_FILENAME%"} vlc://quit

You will need to change IP, ID, PWD and ASF_BASEDIR.
  • IP is the IP number of your Foscam IP Camera.
  • ID is the user name for the camera that you use when you access Camera Web Interface.
  • PWD is the password for the user name on the Camera.
  • ASF_BASEDIR is where you want to store the video streaming data.

At this point, you should run this batch file and see if it does store the video data.
If you don't get any files saved, you will need check the id, password and VLC path.



Now here is my Fake FTP Server.
Save it as a batch file like "c:\batch\fake_ftp_server.bat"
@ECHO OFF
REM *** IP number of FTP server ***
SET ServerIP=xxx.xxx.xxx.xxx

REM *** Actual data transfer port number ***
SET ServerDataPort=2120

REM *** Case sensitive user name ***
SET ValidUserID=camera

REM *** Where is the batch file that triggers VLC to record video stream ***
REM *** Do no use quotation marks on the path
SET VLC_CAPTURE_BAT=C:\batch\record_foscam.bat


REM *** Do not change below ***
SET /a ServerPortHigh=%ServerDataPort% / 256
SET /a ServerPortLow=%ServerDataPort% - %ServerPortHigh% * 256

ECHO 220 Welcome message

SET /p user=
PAUSE > NUL
IF NOT "x%user%" == "xUSER %ValidUserID%" GOTO :ERROR_AUTH
ECHO 331 Please specify the password.

SET /p pass=
PAUSE > NUL
ECHO 230 Guest login ok, access restrictions apply.

SET /P typei=
PAUSE > NUL
ECHO 200 Type set to I

SET /P pasv=
ECHO 227 Entering passive mode (%ServerIP:.=,%,%ServerPortHigh%,%ServerPortLow%)
PAUSE > NUL

SET /P stor=
ECHO 150 FILE: no file will be stored but video recording will start...
START /B CMD /C CALL "%VLC_CAPTURE_BAT%" 2> NUL > NULECHO 226 Transfer complete.
PAUSE > NUL

SET /P next=
PAUSE > NUL
GOTO :END

:ERROR_AUTH
ECHO 530 Invalid login information...
GOTO :END

:END
Once you saved this batch file somewhere, you will need to modify two variables: ServerIP, ServerPort.

  • ServerIP is the IP number of the computer.
  • ServerDataPort needs little explanation. When FTP client access to a FTP server, it makes two connections. One is for the command communication and the other is for actual data transporting. They are traditionally port number 21 and 20 respectively. In this article, I am using 2121 and 2120 respectively. If you scroll up, you can see "FTP port" is set to be 2121 on the Foscam Camera and this fake_ftp_server is going to listen to the port number. Then this fake FTP server has to tell what port number is for the data transfer. So the ServerDataPort should be any port number that is not in use. In this article I will use 2120 for the data transfer.


Remember that this FTP server is a fake FTP server. It can check user name and password but for now it cares user name but not password.


The last step of this article is to have a network port listener.
NetCat will keep listening any network access from other machines.
Here is the batch file and save it as "c:\batch\fake_ftp_listener.bat"

@ECHO OFF
SET FTP_Port=2121
SET FTP_DataPort=2120

REM *** Do not use quotation marks on path
SET FTP_SERVER=C:\batch\fake_ftp_server.bat
SET NETCAT=C:\batch\nc111nt\nc.exe

ECHO [%DATE% %TIME%] Starting fake FTP server for IP camera...

:RESTART
START /B CMD /C "%NETCAT%" -l -p %FTP_DataPort% ^> NUL
"%NETCAT%" -l -p %FTP_Port% -e "%FTP_SERVER%"
"%NETCAT%" -z localhost %FTP_DataPort%
GOTO :RESTART
You will need to set the Path of batch file and NetCat; FTP_SERVER and NETCAT respectively.
Make sure the Port numbers are matching to other batch files; FTP_Port and FTP_DataPort.

Once you start this batch file, NetCat will start listening two network ports.
Now try to run the batch file.
To test whether it works or not, you can manually connect to the port while fake ftp listener is running.
telnet localhost 2121
If you see a welcome message, type in "USER camera".
If it asks password, then it is working.

As an optional step, you can start it as a hidden process when the windows start.
Save this line as a visual basic script file, "c:\batch\invisible.vbs"
CreateObject("Wscript.Shell").Run """" & WScript.Arguments(0) & """", 0, False
 Save this line as a batch file, "c:\batch\fake_ftp_listener_hidden_start.bat"
@"C:\Windows\System32\wscript.exe" "C:\batch\invisible.vbs" "C:\batch\fake_ftp_listener.bat"
 Now you can make a shortcut of the "fake_ftp_listener_hidden_start.bat" in start up folder of your windows.
It is usually C:\Users\[YourUserName]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup

31 comments:

Unknown said...

como lo puedo integrar a javascript?

Anonymous said...

Great work!

I followed your instructions and everything is working as instructed but I still don't get any videos saved. When I run "C:\batch\record_foscam" independently it saves with no issue. Any suggestions?

Jay said...

did u try "telnet localhost 2121" part and did it work?

Jay said...

did u try "telnet localhost 2121" part and did it work?

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

Rectification
la foscam est configurée & allumée
Taper dans VLC Flux réseau
http://localIP:port/videostream.cgi
http://192.168.1.6:80/videostream.cgi
entrer log & password
et enregistrer la video !
A+

Jay said...

Manually running "record_foscam.bat" saves a file of the cam. "Telnet localhost 2121" works, it records and saves a file. I can see in the "Task Manager" that two new 'nc.exe' processes have been started. But when I run the ftp listener, it does nothing.
I have setup the webcam config options for ftp and alarm services as per your instructions. I have disabled my firewall. Any suggestions?

Anonymous said...

Great trick. I would like to not loose the FTP server functionality and thus keep the pictures on my FTP server.
Can you think of a way to let the cam trigger the Fake FTP Server so that it starts the recording, but at the same time let it pass through all communication to a Real FTP server, so that it can record the image uploads ?

Jay said...

It sounds like your foscam camera is not making actual connection. It may be due to anti-virus software but I cannot say for sure.

Jay said...

Technically it should be possible but I haven't tried. I think you may be able to echo anything that NetCat listens to another real FTP server data port.

Pedro said...

hi,

when i run the record_foscam.bat,and i have a VLC error:

vlc media player could not start

either the command line option were invalid or no plugin were found


can you help me?

Anonymous said...

@Pedro,

The Pb is the --noqt-notification option. Just remove it from the script.

Anonymous said...

Hi there. I've got it all set up but it comes apart when I try to log into the fake FTP server. I see the welcome message, then type "USER camera" then it says "Connection to host lost." Any suggestions?

Jay said...

I think I had the same issue. The connection was lost after one line. I don't remember how I fixed it. It might be a problem on Set /P or Pause. Or maybe it was a client issue.

Anonymous said...

Well, it started working on its own, I don't know what I did specifically that fixed it :-P But there is a run on line in your fake_ftp_server.bat. Where it says something like "no file will be saved..." The START following that should be on the next line. Very cool idea, thanks for sharing :)

Eduardo said...

Hi Jay, greeting from spain!

I decided to make your guide, but I'm stuck at the first .bat file.


I have the rigth IP (I tried with Port Number and without PN).
I tried with diferents paths and folders to save file.
Even I tried to execute .bat with administrators privilege/level, but nothing works.

I check VLC path (vlc 2.1.2), and the bat file opens VLC, but VLC closes as soon as open. If I delete VLC quit, VLC doesn't open anything, even if I go to open the URL by myself. I should close VLC and then go by URL.

I can see the camera with url and also record in VLC, but not with the bat file.

Any idea?


Many thanks!

Jay said...

As the matter of facts, I tried my batch files yesterday and I found they don't work anymore. lol

I am not sure why it doesn't work but I think it may be due to the firmware upgrade.

I am planning to work on a new script or batch file but I cannot make any promises at this point.

Anonymous said...

Very good idea. I made a similar approach but on a Linux server. If you want to get the FTP file transfer and do a recoding at the same time:
sudo tcpflow -c port 21 | grep -q "USER ipcam" && bin/vlc_ipcam_record.sh

Unknown said...

Hello, I was looking for information on how to set the Alarm Upload Interval (in seconds). I noticed you said probably set it no lower than 60. Why is that? What is it's function? Help! Thanks!

Jay said...

This is "Blogger", which belongs to Google. I don't know more than that. lol

Unknown said...

Erm....... quite complicated, anyhow, thank you for sharing, perhaps I can tell my friend to learn recording surveillance video from you.

Unknown said...

free software downloads windows also latest updates.

German said...

Hello Jay, I know that this post is very old! But I only wanted to say thank you! This helped me to create a script that runs in my router. It was difucult at first but is working ok now. The only thing that I did not fixed yet is the time difference of the videos. Because the stream is not very good the video is in fast motion! But I will try to sove that too! Again Thank you. A very nice hack!

Unknown said...
This comment has been removed by the author.
Unknown said...

Thanks a lot for sharing about Recording surveillance video, it's a very useful post.
Full Version Software

Unknown said...

Great post! Thanks you so much for the share. Keep up the Excellent work,/we look forward to reading more from you in the future
honeywell | cctv

Wireless Camera in Lahore said...

It is okay to peruse such a pleasant composition. I have sought couple of days and discovered some fascinating composition on it. Be that as it may, it is the best of all. A debt of gratitude is in order for composing this. I truly value your composition.

Ram Chauhan said...

Good post....thanks for sharing.. very useful for me i will bookmark this for my future needs. Thanks.
New Branded Laptops and Desktops In Delhi

Security Product said...

Found your blog. Its really nice on Audio Surveillance. I appreciate your article. Its important to get quality security products. So thanks for sharing all that important information.

HD Wifi Camera said...

Exclusive blog post. Thanks for sharing with us.