[WiP] Timeshifting in a single looping .ts file (1 Viewer)

[FreakyJ]

AW: Timeshifting in a single looping .ts file

don't get me wrong, I'm always pro improvements, but wouldn't it better to just fix the streaming server/reader?!
You mentioned in the thread from miroslav that the streaming works with VLC without any problems, right? Due to my understanding there must be a bug in the stream reader, so the best and clean solution would be to modify the stream reader and not the timeshifting thing...

 

[disaster123]

AW: Timeshifting in a single looping .ts file

FreakyJ
dvdfreak mentioned that - not me. But yes as far as i know tsreader is buggy. I tried to fix it but sorry i wasn't capaple of doing this.

 

[dvdfreak]

So how does TsWriter know when the ts buffer is full? How does it track the read position of clients?

Remember there maybe many clients reading the same ts buffer and each one be located at a different position. TsWriter should not write past any client read position.

Normally there's only one client per timeshifting session (file). At least, in For The Record this is the case, but I think the same goes for TvServer, no?

In any case, extending the .tshift format wouldn't help much, since this file is permanently locked by the server (writing to both this file and the .ts file). But what would be possible it to introduce a .lock file which can be written by the client the moment the live stream is paused (and the contents of the file could be the playback position).

In that case the server could detect the presence of this file and stop writing once it reaches this position.

Should be fairly easy to implement... Unless I'm overlooking something this seems like a simple solution to the problem (I know the file won't grow, but you can't have everything )

By the way, nobody is proposing to get rid of the .tsbuffer format. This new format could simply be an option exactly for those people that have issues with .tsbuffer.

 

[morpheus_xx]

AW: Timeshifting in a single looping .ts file

I want to propose a new .tshift format, similar to the existing .tsbuffer format, but with one big difference: timeshifting takes place in a single .ts file instead of a list of (by default) four files.

The format of this .tshift file is the following:

1. First 8 bytes: 64-bit pointer to the current live-position (where the server is writing)
2. Next 4 bytes: a 32-bit integer counting the number of times the file has looped/wrapped (starting at 0)
3. Rest of the file: a Unicode filepath to the .ts file used (zero terminated, any path is ignored, file is in the same directory as the .tshift file)

As you can see: simple and straightforward.

The server writes to the .ts file and keeps the .tshift file up to date with the current position and wrapped-counter. When the server reaches the end of the file, the counter is incremented and the writing continues at the start of the same file.

What I'd like to upload first is the required patches to TsReader.ax and MediaPortal itself, so handle the client-side of this new format.

Note that the only reason I had to patch MP itself is to simply make sure .tshift files were recognized as live TV files and properly forwarded to tsreader. All the real logic is in tsreader itself.

Also note that there's still a small zapping issue, but we're look at that... Disaster123 is looking at the tswriter side of things, which he can add to this thread too I guess

Finally, what I uploaded does NOT work stand-alone yet. I have an internal For The Record build that writes out the new .tshift format, and disaster123 is working on tswriter...

One question: can you also maintain a list of channel changes and the real timestamps when writing to the buffer file?

I would need this information:
- minimum available timestamp in buffer
- maximum available timestamp in buffer
- list of timestamps for channel changes, maybe connected with the ServiceID or similar way to identify the channel

this would be very useful for my MP2 "project" ;-)

 

[dvdfreak]

Minumim/maximum timestamp in the buffer is already calculated by tsreader right now. But there's no "log" of channel changes yet.

I could fairly easily add this to our own Argus recorder/tuner, but how would this be useful? To extract the channel data later on perhaps, in case the user watched a full program?

 

[arion_p]

Re: AW: Timeshifting in a single looping .ts file

ah thanks arion - really didn't know that. So this won't work atm. Any ideas?

The only way I can think of is that each TsReader should place a shared lock at the current file position being read making sure that at no time it leaves the file without a lock in place (i.e. first place the new lock then release the previous). The lock can be 1 byte long. TsWriter OTOH should try to get an exclusive lock on the region of the file it is about to write before writing anything. The lock should span the entire region that will be modified (not just one byte). If the lock cannot be obtained (the call should be non-blocking) the data should be discarded - the buffer is full.

Use LockFileEx() to place shared and/or exclusive locks. I do not know how well this works over network.

 

[disaster123]

AW: Re: AW: Timeshifting in a single looping .ts file

Use LockFileEx() to place shared and/or exclusive locks. I do not know how well this works over network.

It works really bad over networks. You suggested the locking stuff also in miroslavs thread. I then implemented it for tswriter and tsreader but this results in even more stuttering as the locking on client side seems to be really slow.

 

[arion_p]

So how does TsWriter know when the ts buffer is full? How does it track the read position of clients?

Remember there maybe many clients reading the same ts buffer and each one be located at a different position. TsWriter should not write past any client read position.

Normally there's only one client per timeshifting session (file). At least, in For The Record this is the case, but I think the same goes for TvServer, no?

Definitely not true. When a second client tunes to a channel already tuned by another client, the second client will simply get access to the existing timeshift buffer. It will not create a new one.

In any case, extending the .tshift format wouldn't help much, since this file is permanently locked by the server (writing to both this file and the .ts file). But what would be possible it to introduce a .lock file which can be written by the client the moment the live stream is paused (and the contents of the file could be the playback position).

In that case the server could detect the presence of this file and stop writing once it reaches this position.

Should be fairly easy to implement... Unless I'm overlooking something this seems like a simple solution to the problem (I know the file won't grow, but you can't have everything )

I am not sure this would work. The operations need to be atomic otherwise there is always a chance that the server reads the lock file, sees it is ok to write, the reader writes to the lock file saying it is not ok to write, server overwrites the data that the reader requested to not be overwritten!

Also what would happen if the client skips back.

 

[dvdfreak]

Definitely not true. When a second client tunes to a channel already tuned by another client, the second client will simply get access to the existing timeshift buffer. It will not create a new one.

OK, this is not the case in For The Record. Here, every client gets his own file, much easier, especially when clients start to zap around. They own the file/rtsp, so they can zap as much they want.

In any case, extending the .tshift format wouldn't help much, since this file is permanently locked by the server (writing to both this file and the .ts file). But what would be possible it to introduce a .lock file which can be written by the client the moment the live stream is paused (and the contents of the file could be the playback position).

In that case the server could detect the presence of this file and stop writing once it reaches this position.

Should be fairly easy to implement... Unless I'm overlooking something this seems like a simple solution to the problem (I know the file won't grow, but you can't have everything )

I am not sure this would work. The operations need to be atomic otherwise there is always a chance that the server reads the lock file, sees it is ok to write, the reader writes to the lock file saying it is not ok to write, server overwrites the data that the reader requested to not be overwritten!

If the .lock file is written by the client the moment it pauses, I don't see the problem. In normal cases, the server has plenty of time to detect the file. Even if your live-point is really that hot on the heels of the client, it should be OK since there's a safety margin of a few seconds -- again plenty of time for the .lock file to be detected. When the client resumes playback it deletes the .lock file again.

Also what would happen if the client skips back.

The same comment goes for the current implementation, no? If a client is at the beginning of a file with .tsbuffer, and skips back? Same problem... With .tshift you could leave a buffer so you won't write exactly up to the pause-point, but stop -- say 5 minutes-- before you reach it (if possible).

 

[arion_p]

Definitely not true. When a second client tunes to a channel already tuned by another client, the second client will simply get access to the existing timeshift buffer. It will not create a new one.

OK, this is not the case in For The Record. Here, every client gets his own file, much easier, especially when clients start to zap around. They own the file/rtsp, so they can zap as much they want.

That defeats a great feature of TVServer: switching seats. In MP/TVE you can pause live TV in one client (living room) then go to the other client (bedroom) tune to the same channel, skip back to the position you where watching and pause. Then stop live TV in living room. Finally got to bedroom to resume your watching. With the new feature we are planning this is going to be a lot easier: Park your live TV session in living room and turn off htpc, then go to bedroom turn on htpc and resume your parked live TV session where you left it.

If the .lock file is written by the client the moment it pauses, I don't see the problem. In normal cases, the server has plenty of time to detect the file. Even if your live-point is really that hot on the heels of the client, it should be OK since there's a safety margin of a few seconds -- again plenty of time for the .lock file to be detected. When the client resumes playback it deletes the .lock file again.

If the client is far behind live position, it can be a problem. Imagine server write position is 16kb behind client's read position. When the client pauses it will write the lock file, but by the time it does server may have already written beyond client's read position. This can also happen even without the client pausing. If the client reads in largish blocks the read position may fall behind the server's write position. Also if client's clock is not accurate, it may play slower than real time and the 16kb leeway will soon vanish. When that happens it is better if server stops writing and live data is lost than to have the client suddenly seeing live data.