1.11.0 No connection could be made because the target machine actively refused it (1 Viewer)

[Snoopy87]

Thanks for all the help!

I hope someone will have an idea. As said, I don't have any C++ skills, so at this point I can't help anymore, but of course I can test new versions of the above code, which hopefully fix the problem.

Also it is very annoying that it always takes 15-30 minutes in my current test environment until the exit happens. So any changes always take a lot of time to test.

 

[mm1352000]

If the problem is occurring in FramedSource::getNextFrame() where Sascha said, the error must be that fIsCurrentlyAwaitingData is true when it should be false.

How did that variable get set true? Well, fIsCurrentlyAwaitingData is only set true in one place:
https://github.com/MediaPortal/MediaPortal-1/blob/master/DirectShowFilters/LiveMedia555/liveMedia/FramedSource.cpp#L76

Note that is at the end of the same getNextFrame() function.


How is this getNextFrame() function called?
It's a long story that starts when the streaming server is asked to start streaming a timeshift file:
https://github.com/MediaPortal/Medi...wFilters/StreamingServer/Source/main.cpp#L214

StreamAddTimeShiftFile() creates a TsMPEG2TransportFileServerMediaSubsession, and adds it to the RTSP server.

We can see that the RTSP server is an instance of MPRTSPServer:
https://github.com/MediaPortal/Medi...wFilters/StreamingServer/Source/main.cpp#L164

MPRTSPServer is an extension of RTSPServer:
https://github.com/MediaPortal/Medi...s/LiveMedia555/MediaPortal/MPRTSPServer.h#L30

When a client tries to connect to an RTSPServer sub-session, it sends an RTSP SETUP command. That command is handled here:
https://github.com/MediaPortal/Medi...rs/LiveMedia555/liveMedia/RTSPServer.cpp#L606

As part of handleCmd_SETUP() we see that the sub-session's getStreamParameters() function is called:
https://github.com/MediaPortal/Medi...rs/LiveMedia555/liveMedia/RTSPServer.cpp#L726

Now, we have already seen that a timeshifting sub-session is an instance of TsMPEG2TransportFileServerMediaSubsession. A TsMPEG2TransportFileServerMediaSubsession is a FileServerMediaSubsession:
https://github.com/MediaPortal/Medi...sMPEG2TransportFileServerMediaSubsession.h#L9

...and a FileServerMediaSubsession is a OnDemandServerMediaSubsession:
https://github.com/MediaPortal/Medi...edia/include/FileServerMediaSubsession.hh#L29

Therefore, when getStreamParameters() is called, the base implementation in OnDemandServerMediaSubsession will be called:
https://github.com/MediaPortal/Medi...eMedia/OnDemandServerMediaSubsession.cpp#L144

Inside that function we see that createNewStreamSource() is called:
https://github.com/MediaPortal/Medi...eMedia/OnDemandServerMediaSubsession.cpp#L172

This function is overridden by TsMPEG2TransportFileServerMediaSubsession:
https://github.com/MediaPortal/Medi...EG2TransportFileServerMediaSubsession.cpp#L36

Within createNewStreamSource() we see that a new TsStreamFileSource is created, and wrapped in a TsMPEG2TransportStreamFramer:
https://github.com/MediaPortal/Medi...EG2TransportFileServerMediaSubsession.cpp#L42
https://github.com/MediaPortal/Medi...EG2TransportFileServerMediaSubsession.cpp#L47

When the server is streaming the timeshift file, doGetNextFrame() will be called repeatedly:
https://github.com/MediaPortal/Medi...r/Source/TsMPEG2TransportStreamFramer.cpp#L78

We already know that fInputSource is a TsStreamFileSource; a TsStreamFileSource is a FramedFileSource:
https://github.com/MediaPortal/Medi...reamingServer/Source/TsStreamFileSource.h#L33

...and a FramedFileSource is a FramedSource:
https://github.com/MediaPortal/Medi...555/liveMedia/include/FramedFileSource.hh#L28

So, when TsMPEG2TransportStreamFramer::doGetNextFrame() calls fInputSource->getNextFrame(), it will call the base FramedSource::getNextFrame() function.

Now finally (!!!) we start to get closer to the explanation of the error!


getNextFrame() calls the virtual doGetNextFrame() function. In the case of timeshifting, this will call the implementation in TsStreamFileSource:
https://github.com/MediaPortal/Medi...mingServer/Source/TsStreamFileSource.cpp#L149

At the end of TsStreamFileSource::doGetNextFrame(), we see that FramedSource::afterGetting() should be called:
https://github.com/MediaPortal/Medi...mingServer/Source/TsStreamFileSource.cpp#L202

afterGetting() is one of the three places where fIsCurrentlyAwaitingData is set false:
https://github.com/MediaPortal/Medi...s/LiveMedia555/liveMedia/FramedSource.cpp#L82

In other words: it seems to me that the code is designed to set fIsCurrentlyAwaitingData to true in getNextFrame(), then set it back to false in afterGetting(). If the value is true already at the start of getNextFrame(), my guess is that afterGetting() has not been executed yet because of a delay in doGetNextFrame().

Looking at the code in doGetNextFrame(), the most likely place for a delay seems to be when the code is trying to access the file system:
https://github.com/MediaPortal/Medi...mingServer/Source/TsStreamFileSource.cpp#L178

GetFileSize() is implemented here:
https://github.com/MediaPortal/Medi...reamingServer/Source/MultiFileReader.cpp#L130

As you can see, the implementation calls RefreshTSBufferFile():
https://github.com/MediaPortal/Medi...reamingServer/Source/MultiFileReader.cpp#L294

That function is relatively complex, and I suspect it could be slow in some rare cases. Especially:
1. When TsWriter is trying to update the buffer file.
2. When TV service is trying to clean up (delete) the buffer files.
3. If the timeshift folder is a shred folder.

@Snoopy87
In your plugin, what is the delay between stopping timeshifting... and starting timeshifting again?
If the delay is small, I wonder if TV service is still trying to delete the timeshift files... and that could be the reason for the problem.
I wonder what would happen if you try to increase (or reduce!) the delay.

@Owlsroost
Your thoughts about this would be most welcome.

 

[Snoopy87]

Awsome work!

Please keep in mind, that I always get the IOException, that the timeshift file could not be deleted due to the fact that a process is blocking the file. So maybe after all this could be also a possible reason for the delay?

At the moment I have a hardcore setup, to force the error, where between stopping and starting timeshift is only 1-2 secs. But the error also occured when I waited 10 secs. I have an SSD and timeshifting is only for max. 30 secs on a SD channel, so the timeshift file is very small.

 

[mm1352000]

Please keep in mind, that I always get the IOException, that the timeshift file could not be deleted due to the fact that a process is blocking the file. So maybe after all this could be also a possible reason for the delay?

Yes, it could be. I definitely feel a bit embarrassed about my previous answers now!

At the moment I have a hardcore setup, to force the error, where between stopping and starting timeshift is only 1-2 secs. But the error also occured when I waited 10 secs. I have an SSD and timeshifting is only for max. 30 secs on a SD channel, so the timeshift file is very small.

Hmmm, does the error happen more regularly now that stop/start delay is only 1-2 seconds (compared to 10 seconds)? ...or is it approximately the same?
Is the timeshift folder shared?
Could you try to put the timeshift folder on an HDD and see if it makes any difference? (...I'm wondering if maybe there is a slow TRIM command or something...)

 

[mm1352000]

I spent the last 4 hours looking into this, and now it's past 3 AM here in New Zealand. I need to go to bed.
One last thought: if the problem is related to the timeshift file cleanup, could you try to disable the TS_File_Cleanup thread and see if it makes any difference?

Have a good afternoon while I sleep!

 

[Owlsroost]

@Owlsroost
Your thoughts about this would be most welcome.

I haven't really been following the thread, but I did revise the MultiFileReader code in TsReader a while back, to make sure it handled errors from the lower-level file reading code properly (and passed them upstream if necessary).

 

[mm1352000]

@Snoopy87
Have you had a chance to check the questions/suggestions I posted?

 

[Snoopy87]

Sorry, no, I hope I can continue the work this week! I am very busy at the moment :-(

 

[Snoopy87]

New results:

- no difference whether CleanTimeshift Thread is running or not
- no difference where Timeshift folder is stored (HDD, SSD, Network Share, etc.)

It always happens randomly between 5-30 min. The above changes had no effect when it randomly happens.

 

[mm1352000]

I'm sorry, I have no further ideas.
In summary: we think we know where the error is occurring, but we don't know why. Without knowing the "why" part, we can't fix it.