Multituner und EPG Grabber (2 Viewers)

[HTPCSourcer]

aber jemand sagte mir dass sein Rechner zu heiß (und zu laut?) wird wenn alle Tuner gleichzeitig Grabben.

Ich hoffe doch, dass nur EIN Tuner grabbt. Mehrere ist nicht sinnvoll.

Und wenn ein PC zu heiß wird, liegt ein Konstruktionsproblem vor. Der Tuner dürfte ziemlich konstante Leistung ziehen, da MP die PCIe-Schnittstelle im ON-Zustand belässt.

 

[chrissooo]

It is more about the storage speed than CPU. Writing to the DB at the same time as timeshifting causes the IO to be less sequential and more randomised. For people that don't have SSDs or RAM disks this can be an issue... or sometimes even if you do have those it can be an issue:
https://forum.team-mediaportal.com/...-lot-of-artifacts.117848/page-14#post-1027133


Sure, I'm not ruling it out. Of course you know the full situation...


Is something wrong or inferior with WebEPG?

Hier habe ich die Begründung wieder gefunden.

Momentan ist das tatsächlich so dass alle freien Tuner verwendet werden zum grabben. Ich habe zwei Tuner und wenn ich kein TV benutze, Grabben alle zwei Tuner. Hatte damit jetzt kein Problem. Ist halt doppelt so schnell. Gibt es aber z.B. 4 Tuner, dann könnte es tatsächlich problematisch werden könnte ich mir vorstellen. Vor allem im Zusammenhang mit Live tv.

Ich baue mal zusätzlich eine Option ein in der man einstellen kann wieviel Tuner maximal gleichzeitig graben können.

 

[HTPCSourcer]

Hi,

m. E. setzt du die Anmerkung von mm in einen nicht richtigen Zusammenhang.

Was mm meint, ist, dass der Grabber nach abgeschlossenem Durchlauf seinen Download in die Datenbank schreibt. Diese Schreibaktion erfordert CPU- und I/O-Ressourcen, die dann nicht mehr in vollem Umfang dem TV Service-Prozess zur Verfügung stehen, was auf schmalbrüstigen System zu Rucklern, etc. führen kann. Ob dabei mit einem oder drei Tunern gegrabbt wird, macht keinen grundliegenden Unterschied. Nichts anderes passiert, wenn Grabben bei TImeshifting aktiviert ist.

Fazit: Wenn GRabben parallel zu LIve TV ohnen Ruckler funktioniert, kann man auch mit der Maximalzahl an verfügbaren Tunern arbeiten.

 

[chrissooo]

Ob dabei mit einem oder drei Tunern gegrabbt wird, macht keinen grundliegenden Unterschied

Doch genau das macht den Unterschied. Würde man davon ausgehen dass es immer maximal 2 Tuner gibt, dann könnte man das so lassen. Nehmen wir aber das Beispiel 4 Tuner (hier im Thread hatte jemand 4 Tuner der gefragt hat). Jetzt schaut derjenige Live TV und die restlichen 3 Tuner machen EPG Grabbing. Wenn jetzt alle drei Tuner zufällig ungefähr gleichzeitig fertig werden mit sammeln und dann eben gleichzeitig in die DB schreiben, dann haben wir insgesamt 4 Prozesse die was auf der Platte machen.

Wenn man das ganze zusammenfasst, dann sehe ich unter anderem folgende Szenarien (Timeshiftgrabber könnte man weglassen)

1. User mit sehr schwachem System welches gleichzeitiges Grabben und Timeshift nicht packt
   • Grabber bei Timeshift aus
   • Idle Grabber wenn alle Tuner idle sind
2. User mit mittlerem System
   • Grabber bei Timeshift aus
   • Ein Idle Grabber auch während Timeshiftig
3. User mit stärkerem System
   • Grabber bei Timeshift ein
   • Ein Idle Grabber auch während Timeshifting
     ODER
   • Grabber bei Timeshift aus
   • Zwei Idle Grabber auch während Timeshifting
4. User mit sehr starkem System (SSD usw.)
   • Grabber bei Timeshift ein
   • Maximale Idle Grabber die verfügbar sind, zb. 3 bei 4 Tuner wenn Timeshift an

Deshalb denke ich das eine Einschränkung auf Maximale Graber Sinn machen würde. Schadet ja nicht.

 

[mm1352000]

@chrissooo
I hope you understand English enough to understand this response...

After my previous posts in this thread I learned more about the reasons for stuttering when the EPG grabber is active. Actually there are two common reasons for it:

1. HDD load during data storage (already described previously).
2. Time transferring EPG data from TsWriter to TV library.

Yes, people with "weak" systems (eg. slow/full/fragmented HDD etc.) may see stuttering caused by (1). However, even people with very "powerful" systems can (and do!!!) see stuttering caused by (2)... and even when only one tuner is grabbing EPG. It seems logical to me that the problem can only get worse (become more obvious; occur more frequently) if more than one tuner is grabbing simultaneously as you have proposed. Therefore, I think it would be necessary to solve problem (2) before such a feature could be added.

I also suggest that you think carefully about how to manage database updates. If the database is being updated by multiple simultaneous processes then it can lead to chaos - loss of data, overwriting of data, database corruption (especially with MySQL 5.1 and/or MyISAM).

Good luck.

 

[chrissooo]

@mm1352000

Thank your for the information. Becourse the database inserts, my hope was that nothing happen, maybe just some overridings of some entrys. Also i found sommthing like this in code "//remove following check to enable multi-card epg grabbing (still beta)" that make me hope that the db inserts from two threads makes no problem in mp.

But when here is a risk, then the only one solution is to prevent that more then one thread grap epg (idle and timeshift grabbing considered). I will change it in the patch. What do you thing about that?

In my case i have two tuners and dont use "grabbing while thimeshifting" and dont have an stuttering (use it since over 1 week) when one tuner make timeshifting and another tuner grab epg. I use Clickfinder EPG and this is really great feature for me becourse some channels that i dont tune so often and dont get data from Clickfinder have Data now.

 

[FreakyJ]

But when here is a risk, then the only one solution is to prevent that more then one thread grap epg (idle and timeshift grabbing considered). I will change it in the patch. What do you thing about that?

Maybe add an option and carefully test it

 

[CyberSimian]

Yes, people with "weak" systems (eg. slow/full/fragmented HDD etc.) may see...

...missing highlighting in the EPG; see this thread:

https://forum.team-mediaportal.com/...dings-not-always-highlighted-in-guide.128005/

Although my HTPC has a fast quad-core processor, I would classify my HTPC as a "weak" system because it uses a slow 5400rpm ex-laptop disk for the OS (the drive containing the SQL databases). My capture disk is a fast 7200rpm desktop disk, so I do not experience stuttering (either in live TV or recorded TV), and I have 8 tuners!

I recently tried a different set of EPG-grapping options, and this does seem to have cured the missing EPG highlights. The secret is to enable "Grab EPG only for channels on same transponder". The reason that I believe this works for the UK EPG is the way in which the data is transmitted. Instead of broadcasting the entire EPG for channel 1, followed by the entire EPG for channel 2, and so on, the data for different channels is interleaved. Consequently, if the data for channels in other MUXes (transponders) is ignored, it significantly reduces the rate at which SQL updates are generated.

If (say) it takes 5 minutes to transmit once the entire EPG for 6 MUXes, grabbing the EPG for all MUXes would capture the entire EPG in just 5 minutes, but produce a very high SQL transaction rate.

Ignoring the EPG for channels in other MUXes means that a channel in each MUX must be tuned in turn to capture the EPG for that MUX, so the entire EPG would be captured in (5 minutes) x (6 MUXes) = 30 minutes, but the SQL transaction rate would be reduced by a factor of 6.

But the free lunches are not being distributed just yet. The UK has quite a dynamic EPG, meaning that the major broadcasters try to update it with last-minute schedule changes. And indeed, in the four months that I have been using MP, there have been three occasions when MP made two simultaneous recordings of the same programme, but displaced by 5 minutes. These programmes followed the main evening news, which had been extended by 5 minutes due to the amount/severity of the news that day. The broadcasters had updated the EPG, and MP had by coincidence performed an EPG grab before the programmes started, and so recorded each programme twice (at the original scheduled time, and at the revised scheduled time). Live sports programmes also often overrun, affecting subsequent programmes. However, the problem with ignoring the EPG for channels in other MUXes is that if an EPG grab is performed whilst recording/watching live TV, these last-minute EPG changes may go unnoticed.

Complicated stuff, this EPG grabbing!

-- from CyberSimian in the UK

 

[chrissooo]

Sooo,

nochmal eine neue Version die sicherstellt dass es maximal nur einen Grabber gibt der zur gleichen Zeit seine Arbeit verrichtet. Vielleicht hat die Variante mal eine Chance übernommen zu werden ... immerhin verschlechtert es sogesehen nicht die Situation da durch den in MP vorhandenen Timeshiftgrabber eigentlich fast die gleiche Situation entsteht wie wenn man den Timeshiftgrabber aus hat und dafür den Idlegrabber auf einem anderem tuner laufen lässt.

Habe die Optionen ein wenig umgestaltet:

Wenn man den Timeshift Grabber nutzt, dann wird das ganze dektiviert:

Dateien sind wie immer auf der ersten Seite.

@mm1352000
What do you thing about this now? This patch does not make it worse then the original 1.10. The new feature make nearly the same state like with the timeshiftgrabber (except that there are two timeshifts, but on many tuners and recording we have it also). But finally it is ensured that only one EPG Grabber thread runs.

Maybe a chance to get it in future Versions of MP?

 

[snowball]

Hi jumping into the thread. Please do not make free tuner epg grapping the default. Iam running a multiseat with up to 4 clients parallel. Idle grapping is the only working solution all others just cause perf issues. And ill not throw hw at it just for epg grapping. I disabled timeshift grapping by default. So if people want to update epg more regular they can use the new function optionaly. Or let people like me turn the newb function optionaly off. What i see much more relevnt is the grap wait time. Long time there was already a patch proposal to switch to seconds which was not implemented. I am running that patch with 10 second waittime per channel and now epg loads much faster. Iam waking up the system to grap epg at times where it is not used.