Automatic commercial cutting & transcoding to xvid

[Nolgthorn]

I am planning on setting up a MythTV type of computer box for my TV, and found MediaPortal. It looks good but the only reason I'm actually going to use it is for this feature which you have provided in a well layed out medium.

I really have one question, is it possible to set the compression level of the encoding based on video length.

Such as, if it were a 30 minute video to compress it automatically to fit 1/4 or 1/6th the size of a CD exactly. 2 hours, to compress it to 700MB etc like that, so it's easy to store and archive?

[Nolgthorn]

Does the compressor support 1st pass/second pass encoding for more exact filesizes and higher visual quality or is it just single-pass? Is the audio compressed down to Mp3 or Ogg before being included in the XviD?

Seems to me that sinse the video is really what takes the most time to compress, a high quality audio technology should be included also.

[Noelix]

Originally Posted by Nolgthorn

I really have one question, is it possible to set the compression level of the encoding based on video length.

Such as, if it were a 30 minute video to compress it automatically to fit 1/4 or 1/6th the size of a CD exactly. 2 hours, to compress it to 700MB etc like that, so it's easy to store and archive?

You'd have to edit stattik's script to implement that kind of a feature. Currently it's a set bitrate but what you can do is make use of the different encoding profiles in stattik's batch script. If you look at it in notepad you can see that it can search for strings in a show and send them to different profiles. For example, let's say you want multiple profiles for shows that are different times. So you have a 30-minute show profile, a 1-hour show profile, etc. And the bitrate on each of those could be different (you'd have to use a bitrate calculator to figure it out exactly), so you'd have a bitrate that was twice as high for the 1/2 hour show as the 1 hour show. Currently the bitrate that is set is around 900kbps for all profiles. This roughly equates to a 1/2 hour show coming out to around 175MB, which is 1/4 of a CD. So in the end you could have the script look for let's say "Monk" in a series title, and since that is a 1-hour show you would send it to a 1-hour profile. And for all others they could be sent to the 1/2 hour profile. Anyways, these are just ideas. I'd be happy to help out further if you'd like, just email me or PM me sometime.

[Noelix]

Originally Posted by Nolgthorn

Does the compressor support 1st pass/second pass encoding for more exact filesizes and higher visual quality or is it just single-pass? Is the audio compressed down to Mp3 or Ogg before being included in the XviD?

Yes there's a two-pass profile in stattik's script so you can do that if you want. Just be aware that it takes twice as long to encode something. Yes the audio is compressed to mp3 simultaneously. I forget if its 128 cbr or vbr though...

[Green_Star]

I tried this method, here my problems.

AutoDVRconvert is trying to convert before the recording complets, and even i manually start AutoDVRconvert after recroding complets comskip and batch script is starting before AutoDVRconvert complets its conversion. any idea hot to avoid this?

[Noelix]

Originally Posted by Green_Star

AutoDVRconvert is trying to convert before the recording complets, and even i manually start AutoDVRconvert after recroding complets comskip and batch script is starting before AutoDVRconvert complets its conversion. any idea hot to avoid this?

There is an option in AutoDVRconvert (a checkbox) that says Convert while recording (on the fly) - I leave this unchecked for this very reason. Check to see if yours is checked or not. If it is not checked and you're still having this problem, then you should probably use a different method to convert your files from dvr-ms to mpeg. If this is the case let me know and I'll write up a little guide on it.

[EPGsetup]

Here are the instructions to commercial-cut/Xvid encode directly from DVR-MS to AVI. It took a little fishing to get the right parameters working, I am testing the automation portion.

This method basically lets you get rid of the extraneous 'copy' step (.DVR-MS to .MPG). Perhaps this can lead to the creation of an automatic plugin. You *do* need to have AutoDVRConvert. (see next post)

-------

Step buhleted because of Dir2mon "file in use" problems. Use the dummy filter workaround, it's in my next post.

- I recommend downloading the newest version of Mplayer/mencoder for Windows, which is 1.0pre8 as of 07/17/2006 from here. Unzip this to where you put mplayer before, overwrite all files.

- If you downloaded the latest version of comskip, you do not need to change anything here. Newer versions support the DVR-MS format, for the most part. Mine had no problems with the type of dvr-ms MediaPortal spews out.

- Edit the stattik batch file provided by Noelix. Search for phrase ":standard". It should be near the bottom of the file. This should take you to the GOTO section dealing with 90% of the files you can feed this from MediaPortal. All the bat file does is parse the filename to see which section it should go to.

-Add the argument "-demuxer 35" to the line. By adding this argument, you can open dvr-ms files in mplayer and mencoder. Also add -idx. That indexes the AVI, which was a problem I found later. The new line should read:

start /low /wait /min %mencoder_path% %name% -edl "%~dpn1.edl" -demuxer 35 -ovc xvid -xvidencopts bitrate= ........Cut out ffor space ....... -o "%~dpn1.avi" -idx

There is a 'prettier' way to do this by adding a new GOTO statement in the avi/mpg section, and I can post my version if someone needs it. This guide here is the quick and dirty version, however.

- Change the Dirmon2 program to look for a complete .dummy file instead of a mpg file in the Comskip job. Because of the .dummy bug fix, we actually wait for the .dummy file made by AutoDVRConvert.

-The settings that need to be changed are changing all 'targets' for both Encoding and Comskip jobs to "dvr-ms,DVR-MS" , and 'must exist' in the comskip job from "mpg OR MPG" to "dvr-ms OR DVR-MS". Also add ".log" to 'must NOT exist' to prevent the Comskip job from running multiple times.

-In the cleanup section, also remember to change the "mpg,MPG,mpeg,MPRG" part to "dvr-ms,DVR-MS" or the Comskip edl files will be deleted as soon as they are made off the DVR-MS file.

There are a couple other tweaks, so match settings to the pictures in my next post.

-----------

This general process is working for me, and it saves me from having to double up on free hard drive space just for the mpg copy. If there is enough demmand, I will post my other resources.

One more note, I prefer background encoding and analyzing of my shows, so that I can have it done every 20 min or so and not hammer the resources. That DVR-MS -> MPG copy step was the main culprit, but I have set the max 'Available Resources' on the others to 30%. The worst I have found so far is a little stutter when Comskip is running, and a bit elsewhere randomly.

Footnote: after some testing, the AVI files made by mencoder aren't indexed. I didn't notice it at first because VLC rebuilds the index when the file is played. It is probably trivial to have mencoder rebuild the index, but if someone could post the CLI, it would save time. I found the fix. It is now listed above.

After using this setup on a couple movies on FX, there are a couple bugs. First, because of the way MediaPortal writes DVR-MS files, Dirmon2 kicks in prematurely (even with wait-until-file-is-complete enabled). I have no idea how to fix this. Maybe use a dummy filter in AutoDVRConvert to make a blank .txt file, since that is the only program that correctly identifies an in-use DVR-MS file.

Second problem is the indexing is acting weird. It worked fine during some, but not others. Still working on this.

[Green_Star]

Originally Posted by EPGsetup

Here are the instructions to commercial-cut/Xvid encode directly from DVR-MS to AVI. It took a little fishing to get the right parameters working, I am testing ...............

Thanks man, i will follow your instructions and let you know if i see any other problems.

I usually record movies (around 3 hrs lenght recording with ads), I observed that once MP starts the recording the created file has extension dvr-ms(still it continues the recording by appending that file), hope dirmon will not kick the comskip (by seeing the drv-ms file) before MP completes its recording (in autodvrconverter convert on fly conversion box is unchecked but still it is starting before the recording stops, any way now i can take out this process) (i guess 'wait until target file is finished' check box in dirmon should handle this situation).

One other questions, i do not want to convert to avi format, drv-ms file is fine for me, because it takes more time to convert to avi than just leaving that as dvr-ms. How can i avoid that process?

I am using comskip79_48, is this okey for dvr-ms files or do i need to go for comskip79_29_dvr_beta2 ? (from list http://mk.kaashoek.com/gbpvr/)

[EPGsetup]

Ok, I have got it working 100% now. For simplicity's sake I am posting screenshots of the settings. Just change the settings to match what I have if you want direct DVR-MS -> Xvid encoding. Thanks to Noelix for the guide, there's no way I'd know about this apps otherwise.

It ends up you need AutoDVRConvert, just to detect when the file is done - Dirmon2's version of "detect when file completed" just doesn't cut it. It fires off the Comskip job while the file is still being written to.

I wish Dirmon2 was a bit smarter, then AutoDVR wouldn't be needed at all. But for now, it's the only application that actually can tell when a dvr-ms is 'finished'. I got around this by using a 'dummy' Graphedit filter (it outputs a blank, empty file with a .dummy extention) when the file is finished to signal Dirmon2 that it is ready to begin. Dirmon2 sees that, and picks up where the .mpg step would.

I also added one thing, Noelix's setup assumes comskip doesn't take too long. I ran into the issue Comskip would run multiple times since the conditions would be 'true' while comskip itself was running - no .EDL file, and a .DVR-MS exists. So I added .log in "must not exist", this prevents doubling-up.

Here is the 'dummy' XGR filter I used. Copy/paste it in a .txt file and rename it dummy.xgr, and load it into AutoDVRConvert. That's it.

<GRAPH version="1.0">
	<FILTER ID="File_writer" clsid="{8596E5F0-0DA5-11D0-BD21-00A0C911CE86}">
		<PARAM name="src" value="D:\mpStuff\mpRecordings\blah.test"/>
		<PARAM name="data" value="3000300030003000300030003000300030003000300020000800000000000000"/>
	</FILTER>
	<FILTER ID="DVR_Source_0" clsid="{C9F5FE02-F851-4EB5-99EE-AD602AF1E619}">
	</FILTER>
</GRAPH>

Here's links to the configurations:

Edit Shattik's batch file as highlighted (note WORDWRAP is ON. Don't add linebreaks.)
AutoDVR Settings (note the dummy.XGR and LACK of delete on completion)
Comskip job (that's "dummy" and "DUMMY" in the boxes)
Encoding job
Cleanup

[baloubic]

I've tried to follow the guide...

1. AutoDVRconvert creates a dummy file: OK
2. DirMon2 activates comskip, which outputs log, edl, txt, etc: OK
3. DirMon2 puts transcode script in que.
4. The transcode script never goes in the 'active jobs', or it's there so briefly that it doesn't show. Anyway, all that happens is that all files are deleted...


This kind of bugs me...

What could be wrong? All I've done with the transcode script is to enter the path to mencoder and the changes in the standard: section.

Why doesn't it convert anything before deleting everything? Anyone?