AutoCropper plugin (4 Viewers)

[ziphnor]

That would be really nice

Ive cleaned up a bit, and made the top down scan use a binary search style approach, which seems to consume very little CPU time.

I think im going to temporarily abandon the seperate scan for subtitles, because while its definetly doable, it makes things much more complex. Instead ill try to work towards a milestone version that provides the basic ability to automaticly crop black bars and logos using a simple binary search style scan from the bottom and top. Then i can add the code to maintain a stable bounding box.

When i got that working decently im going to look at integration into MP. At this point it should be usable by end users.

Only after that is working ill go back and offer support for finding subtitles and cropping them out or copying them into the active frame(the later being the most expensive as it must necesarily be done EVERY frame, while the other scans can be done at intervals ).
I just have a feeling that i could spend alot of time playing around with subtitle detection otherwise, and i think its better getting some more feedback on the basic detection algorithm first.

 

[jawbroken]

Yes, subtitles are the really tough part because of the similarity between subtitles and logos and the variance in size/etc.

I am looking forward to seeing this finished.

 

[knutinh]

Are you aiming for a "box of constant dimensions" encompassing the maximum area used for text, or cutting only the instantaneous area really used for each frame?

I envision that if a slowly changing bounding box defines the general text area, this will look better when shifted up into the movie frame, than text on a black background that is constantly changing.

Perhaps blending the black with video background will look better? Then we'll have white text on grey/blended background instead of white text on black background.

regards
Knut

 

[ziphnor]

I had hoped that subtitles would be very distinct in that they had very few brightness levels(ie VERY bright and VERY dark) but thats not the case, some subtitle lines have just as many distinct Y levels as image lines. I have managed to produce tests that always catch subtitles but need some auxiliary sanity tests to avoid messing with the image itself. It is definetly doable, but i think most software development theory says to develop incrementally, ie build something that works with some simple functionality first, *then* improve it

To keep the subtitles from bouncing up and down, it is best to maintain a constant(ish) box for them.

When copying the subtiles i only copy pixels with a brightness over a certain threshold, so any black/gray background for the subtitles is discarded. This also means that its okay to keep a larger bounding box for subtitles than is actually used, since a frame without any actual subtitles will not copy anything.

 

[tourettes]

Ziphor, remember that some subtitles are partially over the video frame, I think it will make it impossible to move those upwards as moving them would cause part of the video frame to be moved also. And I would guess it's almost impossible to make a difference between the subtitle and the video frame underneath it.

 

[ziphnor]

Thats a good point tourettes, but one i had already anticipated. What i have been doing is only moving subtitles if i can find a black bar seperation betwen them and the image area. This can fail to detect some subtitles if the subtitle check isnt foolproof, but it seems okay, however its one of the things that made me decide to hold back on the features and get something working instead.

Btw, tourettes, i know you been working with DS filters, so do you by any chance know how a DS filter placed just before the renderer can instruct the renderer to clipc/crop the image? Ive tried to play with rcSource and rcTarget, but it just doesnt work.

I got alot of help from microsoft.public.win32.programmer.directx.video in relation to my DS filter, but no one seems to know about clipping/cropping, at least not without stopping the graph and reconnecting:

google groups link

It would be so nice if i could perform the clipping directly in the filter graph , as i would then need almost no integration code in MP(only adding the filter if enabled and setting preferences based on config).

 

[tourettes]

Haven't looked in cropping in DS filter, but one way I guess would be to use different resolutions in input and output and then manually copy the needed image frame portion to the output.

Have you tried to use SetOutputRect from IVMRMixerControl9? Althou it doesn't seem to be advisable

http://msdn.microsoft.com/library/d...ctshow/htm/ivmrmixercontrol9setoutputrect.asp

"The IVMRMixerControl9 interface enables an application to manipulate the incoming video streams on the Video Mixing Renderer Filter 9 (VMR-9). This interface is intended for use by applications only; it should not be used by upstream filters."

 

[ziphnor]

Im not sure the first suggestion would work without stopping the graph. I can renegotiate the media type dynamicly using DynamicQueryAccept without changing the buffer size. But i guess i should check if VMR9 will accept a smaller buffer size, not sure how that would work though. I thought that one instead used rcSource to specify the part of the buffer that should be used(in fact the VMR9 does that sometimes, but maybe thats the only direction it works in). I guess its worth trying.

The second suggestion sounds more promising if it wasnt for that warning Im wondering if MP doesnt itself make a call to that method in connection with aspect ratio control etc? If so, i could cause a real mess

Thanks for your input, you have given me a few ideas, if you can provide more info please dont hesitate.

 

[ziphnor]

Regarding the DS cropping, i tried using DynamicQueryAccept to request a mediatype request a smaller buffer and VMR9 doesnt seem to mind, so that might be the way to do it, if i can figure out how to actually make the change

 

[ziphnor]

Status update:

I started working on the bounding box stabilizer. I had some problems deciding how to do it, but for now ive tried the following. Ive implemented a class to track the moving average of the bounds(ie top and bottom for now), each frame the averages are updated, but the current bounds only change if the moving average is 'significantly' different(which should probably be and adjustable parameter).

This does seem to work quite well with a 25 frame long moving average, but there is of course a slight delay in adapting to changes( that should be acceptable). I should probably adjust it to accept increases faster than decreases. I think that doing this and adding a few sanity checks to throw out bounds that are obviously to 'thin'(ie more widescreen than the widest video formats used, i think its 1:2.35 or something like that) as well as maybe finding away to ignore results from very dark frames might make this stable enough for everyday use.

Im still hoping to solve the cropping on the filter graph level, but we will have to see what happens.