First off you have white thumbs (there is no white screem in recordings) then you have thumbs where it has not created enough pics. I have it set to 2x2 and some are created ok but some are created as a 2x1 grid and then this gets scaled in the recorded TV view so they get streteched and look wierd.
Sounds like a bug in mtn, unfortunately it is a 3rd party component, so getting any help might be hard.
Also should mtn.exe work with H.264 recordings?
Yes, at least on few test files I have it manages to create those.