Perhaps keeping a 25 frame moving average and only sampling every n frames would be a good idea? If it is just 25 frames in a row, that is only half a second at 50 frames per second, which seems like it could be fooled by dark/unsuitable frames.
so a renegotiation once per second to minute should be entirely invisible to the user.
And the renegotiation is only needed if the size of the frame has changed, right? So it will probably be even less than that.
Just wanted to let you know that your filter sounds awesome.
Thanks its nice to know someone cares