Zoom : adding 'no aspect ratio change' in general settings (2 Viewers)

[miljbee]

of course EMKO, all in the config app would be the best solution, but starting with xml config file in next svn would be good start point, to see how to specify different zoom modes.

i mean with zoom modes, there is a lot of parameters to take in account.

% of the original picture size
% of the picture already resized to match the screen size
Aspect ratio based zoom modes like 14:9, 16:9, 4:3 and so

we could also imagine some sort of parameters like
% of picture lost in top and bottom of the screen
% of picture lost in left right of the screen

but also, aspect ratio modification like
y/x=1.1

and some other zoom modes based on the screen size.


I have one other idea about zoom modes. i don't know it this could be done. The previously proposed features would be great, but there is something which could be even more cool; i would call this "smart zoom".

smart zoom would be an adaptative zoom mode based on user preferences, original image size and screen size.

the user preferences would be simple questions :

would you accept some aspect ratio modifications to display a picture ?
A:Yes, B:No, C:Yes but limited
if C, enter the max aspect ratio error you would accept : C%
(if A, C=100%, if B, C=0%)

would you accept to lose (crop) a part of the picture ?
at the top and bottom of the screen ?
D: Yes, E: No, F:Yes but limited
if F, enter the % of the height you accept to lose : G %
(if D, G=100%, if E, G=0%)

at the left and right of the screen ?
H: Yes, I: No, J : Yes but limited
if I, enter the % of the width you accept to lose : K %
(if H, K=100%, if I, K=0%)

once we know C, G, K, The x and y size of the screen, the x and y size of the original video, it is possible to calculate the size of the zoomed video that correspond most to the user habits.

i hope i am clear, but perhaps it isn't ! don't hesitate to ask more info on this idea, i can try to be more precise and propose a sort of algorythm/calculation to obtain the size of the zoomed picture.

 

[miljbee]

Let's have an example to illustrate how smartZoom would be computed :

movie size : 400x300 (mx=400, my=300)
screen size : 1920x1080 (sx=1920, sy=1080)



would you accept some aspect ratio modifications to display a picture ?
A:Yes, B:No, C:Yes but limited
if C, enter the max aspect ratio error you would accept : C%
(if A, C=100%, if B, C=0%) C=5%

would you accept to lose (crop) a part of the picture ?
at the top and bottom of the screen ?
D: Yes, E: No, F:Yes but limited
if F, enter the % of the height you accept to lose : G %
(if D, G=100%, if E, G=0%) G=10%

at the left and right of the screen ?
H: Yes, I: No, J : Yes but limited
if I, enter the % of the width you accept to lose : K %
(if H, K=100%, if I, K=0%) K=0%
do you prefer aspect ratio modification or picture crop ?
AR : K, Crop : L L

Exemple: C=5%, G=10%, K=0%, L

if sx/sy<mx/my then
     yZoomFactor = sy/my '3.6
     xZoomFactor = yZoomFactor '3.6
     if mx*xZoomFactor<sx then
          if L then
               yZoomFactor = sy*(1+G)/my '3.96
               xZoomFactor = yZoomFactor '3.96
               if mx*xZoomFactor<sx then
                   xZoomFactor = xZoomFactor*(1+C) '4,158
               else
                    ...
               end if
          else
               ...
          end if
     else
       ...
     end if
else
     ...
end if

The result is
movie x Resized = 400 * 4,158 = 1663
movie y Resized = 300 * 3.96 = 1188
for information, aspect ratio ~= 1,4 ~= (4/3) * 1.05

The next step is to display the video centered on the screen ...


going further :

I know this sample code is too easy and not complete :
The method to recalculate xZoomFactor based on AR modification is wrong (the good one would have been a bit difficult to read)
This code ignores the AR Correction parametres defined in the screen calibration parameters
All the else are empty and some of them aren't easy to write

 

[knutinh]

I think this is a very good idea. Zoom is a seemingly "simple" functionality that never is done 100% right (probably because everyone has their own opinion, and it is used many times a day).

I Would like to add one parameter to your idea miljbee:
Original format detection threshold. Often, tv-channels will put a 16:9 signal inside a 4:3 envelope using black bars. The black bars may however contain a channel logo, text or noise that may or may not be wanted. By setting a threshold (% pixels that must be below rgb 5,5,5 for instance) and possibly a time-threshold (this condition must be true for 25 frames), one could allow MP to have a better idea of what is the "actual" image content.

Once it has a good, robust "mx" and "my", it can use user preferences to decide how to scale/stretch/crop those pictures inside the "sx", "sy" envelope.

For me, maximum upscaling without altering dimensions would probably be used the most, however the new 14:9 mode seems to be used a lot lately as well.

BTW, how do you think mixed content should be viewed? I see a lot of commercials and tv-shows use 16:9 letterbox as a creative tool, often only for a few seconds. Would you prefer a 16:9 display to catch this instantly, or do you prefer a 16:9 inside a 4:3 boundary inside a 16:9 display boundary?

regards
Knut

regards
Knut

 

[cheffe97]

Hi,
I like the idea of a smart Focus (suggested something similar a year ago )
For TV what about redoing the calculation every couple of secs b/c the AR changes from let's 16:9 (in a movie) to 4:3 for commercials.
Would be great if AR would adapt automatically within something like 2 secs.
Just my 2 cents
cheers
cheffe

 

[NLS]

I think we should keep things simple (no percentages etc.), yet, give the user the control IF s/he want it.

The situation has four sides.

On one side is the film real aspect ratio. The aspect ratios of films are standard and few.

The second side is what method the channel used to broadcast this film. Sometimes we have a letterbox (either with the channel logo in or out, we don't care), sometimes they cut the sides, sometimes they cut some and use some smaller letterbox etc.

For the first two, the best MP can do, is to make code smarter and understand both the film aspect ratio AND what the channel has done to it. Detecting the black stripes (even if they include channel logo) would be a nice start.

The third side of the problem, is what aspect the viewing equipment of the user is. MP has full knowledge of this (should be defined in the configuration), plus 99% of viewing equipment falls into the 4:3 and 16:9 categories. I say it should be defined in the config, because there are some stupid plasma panels (erm, in fact most plasma panels) that use 4:3 or 1:1 resolutions ON a 16:9 area (how stupid can they get?).

The fourth side of the problem is WHAT THE USER WANTS. Here MP should give control plus allow for some standard settings.

Here is how I look at it with an example with mr. NLS.

Film is 2:35:1.
Channel broadcasts this on a 4:3 frame, cutting some of the left/right edges (and using some pan-n-scan for the scenes that the action is on the side, but we don't care about that) AND giving some black stripes over and under, giving a 16:9 frame, although the logo is on the top stripe, the bottom is NOT used for subtitles (because if it is, it is a different situation).

MP user mr. NLS, has a nice 16:9 (1920x1080) panel, that has DEFINED not only the resolution but also told MP that it is in fact 16:9 (and not a stupid plasma with long pixels).

mr. NLS, wants to see this film as 16:9 and the work that the broacasters did, in fact HELPS.

What MP should do.

- Detect the black bars (ignoring the small static picture that probably is the channel logo).
- Detects that nothing messes with the black bars for a while (no subtitles) but remembers to check if something pops there.
- Detect the aspect ratio of the real viewing area. Finds it very close to 16:9 (maybe 16:9.2).
- Decide that this could be viewed to the real viewing device the user has as the user has declared that he uses a 16:9 display.
- Resize to the edges, cutting out the black bars (and logo).

...Now if something pops on the bottom bar, MP automaticaly changes the display to the original, with black stripes left and right (the original broadcast is 4:3 remember and the panel 16:9), so that the user can read if something popped. This functionality should be switchable though (btw, this functionality shouldn't affect the recording that records the full 4:3 frame in original resolution anyway).

Now if the user decides that what was down there isn't something that should bother him, he should go to the (future) aspect menu of MP and click "revert to previous setting and lock".

There should also be a setting "revert to previous setting" (i.e. no lock, i.e. it will happen again).

MP should keep track of what it did on the specific channel but RE-evaluate if the user switches back and forth the same channel (because the film may have ended and something else could be there using the whole 4:3 frame).

MP aspect ratio menu should have those entries:

Keep original broadcast frame
Autodetect real firm frame
Revert to previous setting and lock
Revert to previous setting
Pan and Scan (cut film sides) [only possible on 4:3 displays]
Letterbox (add black stripes up and down) [only possible on 4:3 displays]
Zoom to 16:9 frame (cut up and down areas) [only possible on 16:9 displays... like Pan & Scan for 16:9]
Sidebox (add black stripes left and right) [only possible on 16:9 displays... like Letterbox but for 16:9)
Expand [only possible on 16:9 displays, will resize 4:3 frame to 16:9]
Shrink [only possible on 4:3 displays, will resize 16:9 to 4:3... like cinemascope movie openings ]
Smart Expand [as expand but will expand sides more than the center]
Smart Shrink [as shrink but will shrink sides more than center]
Manual resize

(also the menu should be fully viewable and not scrollable, because you would need this to be done quick and easy... also remember that options not possible on a specific display, should not be in the menu... 16:9 options in 4:3 and the opposite)

Manual resize should take you to a process, where it shows the full broadcast frame and a crawling line (red/white) is drawn at the edges. Crowling line is important because else the moving features of the film, will not allow you to see it. Left-right cursor should widden or shrink the rectangle left and right (always centered). Up-down cursor should widden or shrink the rectangle up and down (again always centered).

NOTE that this should allow for the user to move BOTH inside the real broadcast frame AND outside. If someone moves outside, then as the rectangle for example grows left right, the broadcast should shrink and black stripes should grow on top and bottom. If someone moves INSIDE the broadcast frame (let say left-right again), then IF the other two sides (top-bottom in our case), are also shrunk, the broadcast frame should grow, up to the point that the up and down sides (the crowl not the film) touch the real screen edges. Now you understand that if the user keeps shrinking left-right past this, then the red/white crowl would start "moving" inside the broadcast frame's left and right edges. (and yes if then you try to shrink up-down, you in fact zoom in until the left-right crowls touch the real display edges)

Now for the smart expand and smart shrink. These are the functions people described again above in the thread. The smart functionality MOST 16:9 TV give nowadays (so why not MP), that expands (and as I said can work for shrink to 4:3 too) the sides more than the center and more and more as you go to the edge. This should use some sine curve function, not discrete areas (because at the point where discrete areas touch, the "step" would be easily noticed).

I was very detailed so that people understand, but in fact I describe a very simple functionality (ok smart expand/shrink not simple but the rest are). It should be easy to be done in 10 secs at the begining of a film and of course remembered for that channel (even if the user switches channel), unless the detected frames change aspect (ie. the film ended and something else is shown using the whole screen).

Of course for recorder video, this should be in the recorded info for the file (like the position is recorded now).

Now if all the above is done, we are talking about perfection and something not touched that much by any other software (well, I think - btw I know of software that does smart expand)...

 

[NLS]

(hope you spent the time to read all that)

 

[knutinh]

Perhaps a rule that says : "if scaling is within a factor of 1.1 or 1.05, then do no scaling". I think it is unnecessary to risk blurring the image to scale an image up 3 pixels to fill the screen when you can actually just center the image and enjoy a non-scaled image =)


This rapidly grows to a complex algo =)

-k

 

[knutinh]

1. Detect source film rectangle. It could be 4:3, 16:9, 2.35:1 or some other non-standard ratio. I dont know if MP receives such information with the DirectShow framework, and whether such meta info can be relied upon? For analog received tv I guess it receives only total pixel dimensions.

Some smart algo could be used to filter in space and time to provide the most likely true movie inside black bars.

2. Look-up user display. Could be 4:3, 16:9 but also 16:10, 5:4 rectangular or quadratic pixels.

3. Use preferences to decide what horisontal/vertical scaling/cropping will be needed.

%detect source format
for all lines
  if #black pixels > 0.9x#pixels_per_line
    this line = black line
    if this line (last frame) = black line
        if all lines so far have been black
          movie is below this line
        else
          movie is above this line
        end
    end
  end
end

-k

 

[Frodo]

Hi!

Great idea on the aspect ratio's
I think it will be simple to add support for this once we figured out the math. Since i'm no expert on aspect ratio stuff i would like to ask you guys to help me a bit with it;-)

To calculate a correct aspect ratio we need to look at the following parameters:

1. The screen display width/height.
The screen displaysize is the entire width/height on which we can present the video. In fullscreen this is simply the entire mediaportal window, but in preview mode its much smaller
Example when desktop is set to 1024x768 then in fullscreen mode the screen display size is also 1024x768

2. The video width/height
This is the width/height of the video itself.
Some examples
PAL = 720x576
NTSC = 704x480
HDTV 1080i = 1920x1080
HDTV 720p = 1280x720
Offcourse these are just examples, the video can be any size.

3. The video aspect ratio
The aspect ratio in which the video was encoded.
example For PAL this can be 4:3 or 16:9

4. The pixel aspect ratio
This one is a bit harder to understand.
On VGA screens a pixel has the same width/height so its aspect ratio is 1:1
However on a TV screen a pixel is however not 1:1, if i remember correctly its 768/767. On a plasma/lcd screen it maybe different as well

5. The desired aspect ratio
The aspect ratio the user desires. Normally this will be the same as 3. The video aspect ratio, but for example if you you choose stretch mode, then the desired aspect ratio is different then the video aspect ratio

Now if suppose we the 5 parameters
We then must come up with a formula which gives me 2 rectangles:

1) the rectangle which specifies which region of the video source should be displayed

2) the rectangle which specifies where on the screen the rectangle in 1) should be displayed

The video source rectangle specifies the left/top,width/height of a rectangle which should be copied.
Example:zoom mode. In this case you need to copy a region of the video source and not the entire video source
so if video is 720x576 then we might copy (10,10)-(700,557)

The rectangle onscreen tells us where exactly on screen we're gonna display the video region copied in step1
so if we again have the (10,10)-(700,557) then we might copy that to
(0,100)-(1024,758) which produces a 100pixel black bar on the top & bottom

So... who can give me the math??

Here are some tasks:
1)
screen display : 1024x768
video : 720x576
video AR : 4:3
pixel AR : 0.9 : 1
desired AR : 4:3
video source rect : ???
display rect : ???

2)
screen display : 800x600
video : 1920x1080
video AR : 16:9
pixel AR : 0.75 : 1
desired AR : 14:9
video source rect : ???
display rect : ???



Frodo

 

[knutinh]

What math is it that you are requesting actually, frodo?

Btw, can the available scaling algo take any practical integer input and scale it to any practical integer output?

The variables then are:
Video Vx[] and Vy[] where x1 and x2 are 0 and x2 and y2 are the input video width and height respectively

Video AR, a ratio giving the aspect ratio (not pixel ratio) of content in case it is non-quadratic pixel (does this ever happen?)

Video content VCx[] and VCy[] providing estimated positions for true video rectangle content.

Display Dx[] and Dy[] where x1 and x2 are 0 and x2 and y2 are the display video width and height respectively.

Display AR, a ratio giving the aspect ratio (not pixel ratio) of content in case it is non-quadratic pixel. Most Pc displays would simply have the same number as #ypixels/#xpixels, however, typically plasma displays have #ypixels/#xpixels = 4:3 while AR is 16:9

Desired video vectors Dvx[] and Dvy[] where x1 x2 y1 y2 are kind of abstract numbers pointing anywhere inside or outside the display device allowing scalers and croppers to do their thing

Desired video AR, a ratio giving the aspect ratio (not pixel ratio) of desired video .


Wow, getting dizzy here...

-k