Distributed Video Transcoding

[Midget]

Hi,

Today I'm exploring an idea that want to share with you.

I've recently bought my first TV card (Hauppauge PVR 150) and immediatly started massive TV recordings.
First problem was disk space with each TV recording (dvr-ms) measured in Gbytes.
I've reduced this problem after the discovery of the "MP plugin for Video Transcoding";.

Even with some problems this plugin works pretty well.

The second problem that arose was computing power: my PC with MP produce recordings faster than transcoding them.
I was thinking about buying another TV card (to avoid registration conflicts!) but feared that this problem will get bigger and bigger.

I've some others PC so I've had the idea of doing Distributed Computing to parallelize transcoding.

Here are some experiments:

I know that the video transcoding plugin is simply a wrapper for FFMPEG or MENCODER but without source code I've to discover the correct command line to execute transcoding.

With Process Explorer from SysInternals (now Microsoft) we can see the command line of a process:

"C:\Program Files\Team MediaPortal\MediaPortal\transcode\ffmpeg\ffmpeg.exe"
-i "F:\tv\TVProgramName_200710182300p103.dvr-ms"
-aspect 4:3 -vcodec xvid -vtag XVID -s 352x288 -b 1024k -r 25 -async 1 -mbd 2 -qmin 2 -bug autodetect -acodec mp3 -ab 128k 
"F:\tv\TVProgramName_200710182300p103.avi"

I want to execute that process on other PC in my home LAN. So I've modified the command line with UNC like this:

"\\MediaPortalPC\C$\Program Files\Team MediaPortal\MediaPortal\transcode\ffmpeg\ffmpeg.exe"
-i "\\MediaPortalPC\F$\tv\TVProgramName_200710182300p103.dvr-ms"
-aspect 4:3 -vcodec xvid -vtag XVID -s 352x288 -b 1024k -r 25 -async 1 -mbd 2 -qmin 2 -bug autodetect -acodec mp3 -ab 128k
"\\MediaPortalPC\F$\tv\TVProgramName_200710182300p103.avi"

Creating a CMD script with that command I can do the same thing done by video transcoding plugin from any PC in my home LAN (assuming the user launching that script has right access privileges). Obviosly I've to change file names in the script to transcode different files.

I can also start that script on others PC directly from my workstation using PSExec (again from SysInternals) with this command line:

psexec \\pc1 -u username -p password -c -f -d -belownormal transcode.cmd

Extremely useful is monitoring transcoding process on remote PC to know when is time to start a new transcoding process. We can do this from one workstation with PSList (again from SysInternals!) utility:

pslist \\pc1 ffmpeg

With these utilities I've managed distributed video transcoding on 4 PC simultaneously (one of them capable of 2 transcoding process with a dual core CPU).

This experiments needed some manual work (monitoring remote processes, writing correct file names, launching scripts) but resulted in a real 5X speed boost.

Maybe we could automate the manual work with some script magic (I've read about powerful things done with Windows Power Shell from Microsoft).

I hope to be helpful

Bye,
Midget

[mzemina]

I believe dukus has C# scripting working.

[GSteele]

You could modify something like this simple batch script I use to recusively trancode whole directories

for /R %%i in (*.avi) DO CALL :ENCODE "%%i%"

goto :eof

:ENCODE
echo "%~pnx1"
if NOT %~z1 GEQ 600000000 goto :eof
if exist "c:%~pnx1" goto :eof
if not exist "C:%~p1" md "c:%~p1"
rem "1st Pass started for %~n1 "
Time /T
"C:\program files\Team MediaPortal\MediaPortal\transcode\mencoder\mencode r.exe" -oac mp3lame -lameopts aq=0:cbr:br=96 -ofps 25 -ovc xvid -xvidencopts pass=1:trellis:me_quality=6:vhq=4:autoaspect:chrom a_opt:bitrate=700 -passlogfile "C:\program files\Team MediaPortal\MediaPortal\transcode\logs\%~n1.mlog" "%~f1" -o "C:%~pnx1"
rem "2nd Pass started for %~n1 "
Time /T
"C:\program files\Team MediaPortal\MediaPortal\transcode\mencoder\mencode r.exe" -oac mp3lame -lameopts aq=0:cbr:br=96 -ofps 25 -ovc xvid -xvidencopts pass=2:trellis:me_quality=6:vhq=4:autoaspect:chrom a_opt:bitrate=700 -passlogfile "C:\program files\Team MediaPortal\MediaPortal\transcode\logs\%~n1.mlog" "%~f1" -o "C:%~pnx1"
rem "2nd Pass finished for %~n1 "
Time /T



goto :eof



then I run this to copy back the contents once I'm happy it worked OK



for %%i in (*.avi) DO CALL :ENCODE "%%i%"

goto :eof

:ENCODE
echo "%~pnx1"
if %~z1 LEQ 100000000 goto :eof
rem if exist "c:%~pnx1" goto :eof
if not exist "c:%~p1" md "d:%~p1"
copy "%~1" "d:%~pnx1" /y
del "%~1"
fsutil file createnew "c:%~pnx1" 1024

goto :eof