Jun 17, 2013

Google AdSense rejected me for "copyright materials"

Today I remembered something that I was wanted to do before I come to America, which is 6 years ago. Web banner.

It was super easy to apply for Google AdSense; it took only 5 minutes to apply. And a few hours later, I got an email saying they rejected my application due to copyright material on my site.

First of all I was disappointed. I thought I can motivate my self for posting more quality articles on my blog by making some pennies. My excitement was changed to a big disappointment.

Second of all, I felt offended. I checked all the articles on my blog three times. Most of them are proudly my own thoughts or ideas. What they mean by "copyright materials" seems like some of illegal torrent link or video/audio files. I do have some video files uploaded but they are all my own videos.

It seems like it is not only me who had been rejected for the same reason. So I am feeling better after finding that there are more people who are like me.

I don't think there is any way to "prove" who owns a video file. I think they are trying to be risk-averse for unknown zones. None technical people may think my script files or game related posting some kind of "hacking" without knowing what those articles actually mean.


Actually, I think I am still feeling offended. But I guess that's what it is... No pennies for me from Google.

Jun 16, 2013

I was having a trouble connecting Wifi of my IP camera, FI8910W, to my new router, SBG6580.

I got the new router from TimeWarnerCable after I upgraded my Internet speed. The router was actually a good one; I like it a lot. It supports Dual band 5GHz 802.11n and it is a Modem and Wifi router combined device.

However, what I found is that my new router doesn't support old Wifi protocols, 802.11b/g. It only supports 802.11n. Also my IP Camera is not working well as 802.11n, although the tech spec stated "802.11b/g/n".

After wasting a few hours, I ended up using both Wifi routers together: one that covers 5GHz 802.11n and another that covers 802.11b/g/n. Since one of them is using a different frequency, 5GHz, I think they should work without much signal interfere.

Jun 5, 2013

Batch file that converts AC3/DTS/EAC3 movie files to MP3

I watch movies through iPad or iPhone. I connect them to TV so that I can enjoy the big screen on my sofa. The WiFi speed is fast enough to carry high definition movies but I found one small problem. Apple is banning AC3/EAC3/DTS movie players. Currently there is no movie play app on iOS that is able to play AC3/EAC3/DTS and access network shared video files.

I think it is some kind of license of patent issues that Apple is complying. The best way for me to work around is to convert those movie files to normal MP3 audio format. So I made a batch file that detect audio format and convert them if necessary.

Audio format can be detected by two softwares: ffmpeg and mplayer. I am using ffmpeg in this article.
There are several transcoding software but I decided to use HandBrake.

Here is my batch file. Save it as c:\batch\convert_audio.bat.
@ECHO OFF
SET ConvertedFolderName=Converted
SET ConvertedExt=mp4
SET VideoSpeed=medium

SET ffmpeg=C:\Utils\ffmpeg\ffmpeg-20130603-git-2976e2a-win32-static\bin\ffmpeg.exe
SET HandBrakeCLI=C:\Program Files\Handbrake\HandBrakeCLI.exe

REM Removing quotation marks from input file path...
SET InputFile=%*
SET InputFile=%InputFile:"=%
SET InputFile=%InputFile:"=%
IF "x%InputFile%" == "x=" SET InputFile=

IF NOT EXIST "%ffmpeg%" GOTO :ERROR_ARG
IF NOT EXIST "%HandBrakeCLI%" GOTO :ERROR_ARG
IF NOT EXIST "%InputFile%" GOTO :ERROR_ARG
REM SET DebugCmd=ECHO

ECHO === Checking audio type: %InputFile% ===
SET AudioProblemDetected=
FOR /F "usebackq tokens=*" %%A IN (`CALL "%ffmpeg%" -i "%InputFile%" 2^>^&1 ^| find "Audio: ac3" `) DO SET AudioProblemDetected=True
FOR /F "usebackq tokens=*" %%A IN (`CALL "%ffmpeg%" -i "%InputFile%" 2^>^&1 ^| find "Audio: dts" `) DO SET AudioProblemDetected=True
FOR /F "usebackq tokens=*" %%A IN (`CALL "%ffmpeg%" -i "%InputFile%" 2^>^&1 ^| find "Audio: eac3" `) DO SET AudioProblemDetected=True
IF "x%AudioProblemDetected%" == "x" GOTO :END
ECHO === Audio needs to be converted ===

SET DriveLetter=
SET BaseDir=
SET FileNameOnly=
FOR %%A IN ("%InputFile%") DO (
    SET DriveLetter=%%~dA
    SET BaseDir=%%~pA
    SET FileNameOnly=%%~nA
)

SET InputDir=%DriveLetter%%BaseDir%
IF NOT EXIST "%InputDir%" GOTO :ERROR_PARSE

SET OutputDir=%InputDir%%ConvertedFolderName%
IF NOT EXIST "%OutputDir%" MKDIR "%OutputDir%"
IF NOT EXIST "%OutputDir%" GOTO :ERROR_OUTPUT_DIR

SET OutputFile=%OutputDir%\%FileNameOnly%.%ConvertedExt%
IF EXIST "%OutputFile%" GOTO :ERROR_OUTPUT_FILE_EXIST

ECHO === [%DATE% %TIME%] Converting file to %OutputFile% ... ===
%DebugCmd% "%HandBrakeCLI%" -i "%InputFile%" -o "%OutputFile%" --encoder x264 --x264-preset %VideoSpeed% --aencoder faac --rate 29.97 --pfr
IF NOT EXIST "%OutputFile%" GOTO :ERROR_FAILED_HANDBRAKE

ECHO === [%Date% %TIME%] Convertion finished ===
GOTO :END

:ERROR_FAILED_HANDBRAKE
ECHO *** It seems like HandBrake didn't generate the file ***
GOTO :ERROR_OUTPUT_FILE_EXIST_DUMP

:ERROR_OUTPUT_FILE_EXIST
ECHO *** Output file already exists ***
:ERROR_OUTPUT_FILE_EXIST_DUMP
ECHO OutputFile = %OutputFile%
GOTO :ERROR_OUTPUT_DIR_DUMP

:ERROR_OUTPUT_DIR
ECHO *** Output directory cannot be created ***
:ERROR_OUTPUT_DIR_DUMP
ECHO OutputDir = %OutputDir%
GOTO :ERROR_PARSE_DUMP

:ERROR_PARSE
ECHO *** Parsing error ***
:ERROR_PARSE_DUMP
ECHO Drive letter = %DriveLetter%
ECHO Folder name = %BaseDir%
ECHO File name = %FileNameOnly%
ECHO InputDir = %InputDir%
GOTO :ERROR_ARG_DUMP

:ERROR_ARG
ECHO *** Incorrect arguments ***
:ERROR_ARG_DUMP
ECHO ffmpeg = %ffmpeg%
ECHO HandBrakeCLI = %HandBrakeCLI%
ECHO InputFile = %InputFile%
GOTO :END

:END

As an optional step, I also made another batch file that takes a folder name and convert every files under the subfolders. Save it as c:\batch\convert_audio_folder.bat.
@ECHO OFF
SET CONVERT_AUDIO_FILE=C:\batch\convert_audio.bat

SET BaseDir=%*
SET BaseDir=%BaseDir:"=%
SET BaseDir=%BaseDir:"=%
REM SET DebugCmd=ECHO

IF NOT EXIST "%CONVERT_AUDIO_FILE%" GOTO :ERROR_ARG
IF "x%BaseDir%" == "x=" SET BaseDir=
IF NOT EXIST "%BaseDir%" GOTO :ERROR_ARG

FOR /F "tokens=*" %%A IN ('dir /b /s /a-d "%BaseDir%"') DO @%DebugCmd% %CONVERT_AUDIO_FILE% %%A

GOTO :END

:ERROR_ARG
ECHO *** Incorrect arguments ***
:ERROR_ARG_DUMP
ECHO CONVERT_AUDIO_FILE = %CONVERT_AUDIO_FILE%
ECHO BaseDir = %BaseDir%
GOTO :END

:END

This converting seems to take forever and I will be running these batch files for a while... lol

Jun 3, 2013

How to delete oldest files when a folder size goes above limit.

I needed to delete surveillance video files once I have more than certain amount in my hard drive. Let's say I don't want to hold more than 2GByte of video files, which will probably cover  one or two month of surveillance video data.

What I need to do with a batch file is like this:
  1. Check how much video files are collected so far.
  2. Compare if the total size is bigger than the size I can afford.
  3. If it is under the space budget, then the batch finishes.
  4. If there are more than I can afford, I want to delete oldest files first.
  5. So I need to figure out what date/time is the oldest.
  6. I will delete files that are at the date/time.
  7. Go to the first step in order to see if I need to delete more.

This was little tricky without installing any external applications.

I had to make two separate batch files due to the syntax limitations on For loop.
One looks like this (c:\batch\delete_surveillance_find_oldest.bat):
@ECHO OFF
SET fdate=%1
SET fdate=%fdate:~6,4%%fdate:~0,2%%fdate:~3,2%
SET /a DATE_DIFF=%OLDEST_DATE% - %fdate%
IF NOT "x%DATE_DIFF:~0,1%" == "x-" SET OLDEST_DATE=%fdate%
It takes a Date value as an input argument.
It compares the date with an oldest date.
If it the date is older than the one that has been found, it updates it.

Another file is like this (c:\batch\delete_surveillance.bat):
@ECHO OFF
SET MAX_SIZE_IN_MB=1800
SET TargetDir=C:\surveillance
SET FIND_OLDEST=C:\batch\delete_surveillance_find_oldest.bat

IF NOT EXIST "%TargetDir%" GOTO :ERROR_ARG
IF NOT EXIST "%FIND_OLDEST%" GOTO :ERROR_ARG

REM SET DeleteCmd=ECHO
SET DeleteCmd=DEL /f

:RESTART
SET TotalSize=0
FOR /F "tokens=1,2,3" %%A IN (' dir /s "%TargetDir%" ^| FIND " File(s)" ') DO SET TotalSize=%%C
SET /a TotalSize=%TotalSize:,=%/1024/1024
ECHO Size limit: %MAX_SIZE_IN_MB%
ECHO Total size: %TotalSize% MB

SET /a SizeDiff=%MAX_SIZE_IN_MB% - %TotalSize%
IF NOT "x%SizeDiff:~0,1%" == "x-" GOTO :SPACE_ENOUGH

ECHO Space is not Enough...
ECHO Searching for the date of the oldest file...
SET /a OLDEST_DATE=99998877
FOR /F "tokens=1,2" %%A IN (' dir /s /a-d "%TargetDir%" ^| FIND "/" ' ) DO CALL "%FIND_OLDEST%" %%A
ECHO Oldest date found: %OLDEST_DATE:~0,4%/%OLDEST_DATE:~4,2%/%OLDEST_DATE:~6,2%

ECHO Deleting the oldest files...
FORFILES /p %TargetDir% /s /D -%OLDEST_DATE:~4,2%/%OLDEST_DATE:~6,2%/%OLDEST_DATE:~0,4% /c "cmd /c @FOR %%I IN (@path) DO @IF NOT EXIST %%~sI\NUL ( @ECHO @path & %DeleteCmd% /f @path )"
ECHO Let's see if we have enough space now.
ECHO.
GOTO :RESTART

:SPACE_ENOUGH
ECHO There are enough space.
GOTO :END

:ERROR_ARG
ECHO Input arguments or settings are incorrect.
ECHO TargetDir = %TargetDir%
ECHO FIND_OLDEST = %FIND_OLDEST%
GOTO :END

:END

This will follow the process I described above.

Jun 2, 2013

Recording surveillance video from Foscam IP Camera without iSpy

*Edit ( Aug/18/2014 ): I made a C/C++ program that stores the stream video from Foscam IP camera with libvlc. It may work for you better. Check out the article here.

iSpy is a cool open source software that records surveillance video. Although it has several features, the features I was using was recording video/audio when motion is detected. It had been working very well and I was enjoying it a lot.

A little problem was that this software has to be running 24/7 and all of the video data must be streamed from IP camera to the computer that iSpy is running even when there is no motions are detected. Because the motion detection happens on the computer side not on the camera side, the computer must download all the video data from the IP camera via WiFi. I figure the video data isn't too big. It was about 7.3MByte per minute, which is 122KByte/sec.

Today I found that my IP Camera (Foscam FI8910W Wireless IP Camera) has motion detection feature already in it. So it made me thinking that when IP camera detect any motions, it can let my computer know that the recording has to start. This way, the WiFi bandwidth is saved and CPU power that were supposed to be spent on image processing is saved. I don't think neither of them are big deal but I started implementing my idea for fun.

The idea was that my Foscam Camera can upload image files onto FTP server when motion is detected. But it doesn't have any ways to upload video streaming data. So I am recording video streaming data when the FTP connection is requested, while the actual image uploading from the camera will be ignored.
The working scenario is like this:
  1. Foscam Camera monitors the scene.
  2. Foscam Camera detect motions
  3. When any motions are detected Foscam Camera try to connect to FTP server and upload image files.
  4. On the FTP server side, it ignores whatever is sent from Foscam Camera.
  5. But at the time FTP connection is requested, VLC is launched and VLC start recording video streaming data directly from Foscam Camera.

Settings on Foscam Camera

 "FTP Service Settings" looks like this.
  • "FTP Server" is the IP number of the computer that is going store the video streaming data.
  • "FTP port" is a fake number of this specific task. I will use a number "2121" for this article but you can change it for any numbers.
  • "FTP User" should be "camera". It is case sensitive like most of FTP login ID.
  • "FTP Password" doesn't matter for now. But you can make it count by changing the fake ftp server batch file later.
  • "FTP Mode" should be "PASV" for now.

The way I am explaining here is not going to use a real FTP server. I made a fake FTP server that pretends to be a FTP server. So User/Password doesn't matter at all.

"Alarm Service Settings" looks like this:
  • "Motion Detection Alarmed" should be checked.
  • "Upload image on Alarm" should be checked.
  • "Upload interval (seconds)" can be bigger than 60. This number is something you can tweak later. I don't recommend any number below 60.

These are all settings on Foscam Camera side.
Now when it detect any motions, the Camera will try to upload to the computer. If you have a real FTP server, it will be working. But you will get only images not audio/video.

Now I will explain computer side. This is little hacky and hard to understand but there is no way to harm the computer at all. You will need to install NetCat for windows and VLC for windows. My implementation is all based on Windows but if you are familiar with shell script, you can translate it by yourself because NetCat and VLC should be working on Linux/Mac too.

You need to install

Let me show you how VLC can record video stream directly. ( This example is taken from here )
"C:\Program Files\VideoLAN\VLC\vlc.exe" http://xxx.xxx.xxx.xxx:####/videostream.asf?user=UserName&pwd=Password --qt-start-minimized --no-qt-notification --run-time=TimeInSeconds :demux=dump :demuxdump-file=MyCamera1.asf vlc://quit
Other URL commands for Foscam IP Camera is here.

Now let's have a batch file that know where to store the video stream data and where to get from.
Save this batch file as "c:\batch\record_foscam.bat"
@ECHO OFF
SET IP=xxx.xxx.xxx.xxx
SET ID=userNameForCamera
SET PWD=passwordForCamera
SET run_time_in_sec=60

REM *** PATH should not have quotation marks ***
SET VLC=C:\Program Files\VideoLAN\VLC\VLC.EXE
SET ASF_BASEDIR=C:\surveillance\video\Foscam
SET YEAR=%DATE:~-4%
SET MONTH=%DATE:~4,2%
SET DAY=%DATE:~7,2%
SET HOUR=%TIME:~0,2%
SET MIN=%TIME:~3,2%
SET SEC=%time:~6,2%
SET ASF_FILENAME=%ASF_BASEDIR%\1_%YEAR%-%MONTH%-%DAY%_%HOUR%-%MIN%-%SEC%.asf

ECHO Recording %run_time_in_sec%seconds of Video stream on to %ASF_FILENAME%...
REM "%VLC%" http://%ID%@%IP%/videostream.asf^?user=%ID%^&pwd=%PWD%^&res=8^&rate=6 --qt-start-minimized --noqt-notification --run-time=%run_time_in_sec% :demux=dump :demuxdump-file="%ASF_FILENAME%" vlc://quit

"%VLC%" http://%ID%@%IP%/videostream.asf^?user=%ID%^&pwd=%PWD%^&res=8^&rate=6 --qt-start-minimized --noqt-notification --run-time=%run_time_in_sec% --sout=#transcode{vcodec=h264,acodec=mpga,channels=1}:standard{dst="%ASF_FILENAME%"} vlc://quit

You will need to change IP, ID, PWD and ASF_BASEDIR.
  • IP is the IP number of your Foscam IP Camera.
  • ID is the user name for the camera that you use when you access Camera Web Interface.
  • PWD is the password for the user name on the Camera.
  • ASF_BASEDIR is where you want to store the video streaming data.

At this point, you should run this batch file and see if it does store the video data.
If you don't get any files saved, you will need check the id, password and VLC path.



Now here is my Fake FTP Server.
Save it as a batch file like "c:\batch\fake_ftp_server.bat"
@ECHO OFF
REM *** IP number of FTP server ***
SET ServerIP=xxx.xxx.xxx.xxx

REM *** Actual data transfer port number ***
SET ServerDataPort=2120

REM *** Case sensitive user name ***
SET ValidUserID=camera

REM *** Where is the batch file that triggers VLC to record video stream ***
REM *** Do no use quotation marks on the path
SET VLC_CAPTURE_BAT=C:\batch\record_foscam.bat


REM *** Do not change below ***
SET /a ServerPortHigh=%ServerDataPort% / 256
SET /a ServerPortLow=%ServerDataPort% - %ServerPortHigh% * 256

ECHO 220 Welcome message

SET /p user=
PAUSE > NUL
IF NOT "x%user%" == "xUSER %ValidUserID%" GOTO :ERROR_AUTH
ECHO 331 Please specify the password.

SET /p pass=
PAUSE > NUL
ECHO 230 Guest login ok, access restrictions apply.

SET /P typei=
PAUSE > NUL
ECHO 200 Type set to I

SET /P pasv=
ECHO 227 Entering passive mode (%ServerIP:.=,%,%ServerPortHigh%,%ServerPortLow%)
PAUSE > NUL

SET /P stor=
ECHO 150 FILE: no file will be stored but video recording will start...
START /B CMD /C CALL "%VLC_CAPTURE_BAT%" 2> NUL > NULECHO 226 Transfer complete.
PAUSE > NUL

SET /P next=
PAUSE > NUL
GOTO :END

:ERROR_AUTH
ECHO 530 Invalid login information...
GOTO :END

:END
Once you saved this batch file somewhere, you will need to modify two variables: ServerIP, ServerPort.

  • ServerIP is the IP number of the computer.
  • ServerDataPort needs little explanation. When FTP client access to a FTP server, it makes two connections. One is for the command communication and the other is for actual data transporting. They are traditionally port number 21 and 20 respectively. In this article, I am using 2121 and 2120 respectively. If you scroll up, you can see "FTP port" is set to be 2121 on the Foscam Camera and this fake_ftp_server is going to listen to the port number. Then this fake FTP server has to tell what port number is for the data transfer. So the ServerDataPort should be any port number that is not in use. In this article I will use 2120 for the data transfer.


Remember that this FTP server is a fake FTP server. It can check user name and password but for now it cares user name but not password.


The last step of this article is to have a network port listener.
NetCat will keep listening any network access from other machines.
Here is the batch file and save it as "c:\batch\fake_ftp_listener.bat"

@ECHO OFF
SET FTP_Port=2121
SET FTP_DataPort=2120

REM *** Do not use quotation marks on path
SET FTP_SERVER=C:\batch\fake_ftp_server.bat
SET NETCAT=C:\batch\nc111nt\nc.exe

ECHO [%DATE% %TIME%] Starting fake FTP server for IP camera...

:RESTART
START /B CMD /C "%NETCAT%" -l -p %FTP_DataPort% ^> NUL
"%NETCAT%" -l -p %FTP_Port% -e "%FTP_SERVER%"
"%NETCAT%" -z localhost %FTP_DataPort%
GOTO :RESTART
You will need to set the Path of batch file and NetCat; FTP_SERVER and NETCAT respectively.
Make sure the Port numbers are matching to other batch files; FTP_Port and FTP_DataPort.

Once you start this batch file, NetCat will start listening two network ports.
Now try to run the batch file.
To test whether it works or not, you can manually connect to the port while fake ftp listener is running.
telnet localhost 2121
If you see a welcome message, type in "USER camera".
If it asks password, then it is working.

As an optional step, you can start it as a hidden process when the windows start.
Save this line as a visual basic script file, "c:\batch\invisible.vbs"
CreateObject("Wscript.Shell").Run """" & WScript.Arguments(0) & """", 0, False
 Save this line as a batch file, "c:\batch\fake_ftp_listener_hidden_start.bat"
@"C:\Windows\System32\wscript.exe" "C:\batch\invisible.vbs" "C:\batch\fake_ftp_listener.bat"
 Now you can make a shortcut of the "fake_ftp_listener_hidden_start.bat" in start up folder of your windows.
It is usually C:\Users\[YourUserName]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup