Slow TV channel scan (1 Viewer)

[Vasilich]

I just wonder why MP is so much slower than other software when it comes to TV channel scan?
I tested it with several programs (MyTheater, DVBViewer, DVBPortal, AltDVB) - all of them need 15 to 20 minutes to scan all transponders on Astra19 and Hotbird13 (around 3500 channels), but scan in MP takes almost 2 hours for same two satellites!
I am talking only about DVB-S/S2 tunes.

 

[Vasilich]

i tried to delete all channels and did full rescan of both satellites. I have seen that at the beginning of scan the speed of adding channels was quite good - it took to add 50 channels for one transponder in less than 2 seconds, but after some minutes it took again 2 channels per second. So it seems that channel accessing in DB is not quite optimal - see e.g. https://forum.team-mediaportal.com/...er-config-gui-slow-channel-enumeration-79475/ as i believe both of these issues have the same reason.

 

[mm1352000]

Hi Vasilich

I suspect this is a database configuration issue, but in any case please provide a *full* set of TV Server logs (the log in the other thread is useless). If you can find them then it would also be helpful to see the MySQL logs (especially the slow query log).

Also, how hard are your HDD and CPU working when channel enumeration is slow?

mm

 

[Vasilich]

OK, will post TVServer logs tonight.
But in MySQL logging is deactivated. Should i activate it (assuming that things go even more slow ) ?

 

[mm1352000]

Hi again

Well, let me put it this way: TV Server could be at fault, but it would be the first time I have ever heard of such an issue. If we can't see any faults on the TV Server side then the MySQL logs would be required to know if something is going wrong at the database level...

mm

 

[Vasilich]

ok mm, here are my logs from TV Server and MySQL when accessing TV channel list.
How to reproduce:
MySQL slow log set to log all queries that took longer that 1 second to execute.
cleaned all logs from TVService and MySQL
01:38:00 - Started MySQL service
01:39:00 - Started TVService
01:40:00 - Started TVServer config
01:40:35 - TVServer config window appeared
01:41:00 - selected "TV Channels" in TVServer config
01:41:45 - all channels appeared in the list
01:42:00 - pressed OK on TVServer config window
01:43:00 - stopped TVService
01:44:00 - stopped MySQL service

I hope there is enough logs and you can clear who is doing something terribly slow.
Also note that i have around 3600 channels at all.

Igor.

 

[mm1352000]

Hi Vasilich

After only a quick look at your logs it is very clear that something is wrong.

First, in the TV Server logs you are getting Powerscheduler polling very often - approximately 1 time per second. That just seems too often to me.

Second, you have a channelgroup missing from the DB:

2011-06-26 01:41:27.981000 [SetupTv(1)]: channelgroup for channel id=3736 with channelgroup id=1 does not exist

Third: if you look at your MySQL log, the level of activity there seems way over the top. By my count, around 17000 queries in 4 minutes.

Finally, in your MySQL slow query log there are a some slow queries registered.

-------------------------------------------------------------------------------------
Okay, so what do we do?

I wouldn't pay any attention to the MySQL slow query log. In this case it seems that the queries aren't that slow (although the 160ms one is a bit unusual). I think they are registered simply because the DB is working relatively hard...

And why is the DB working so hard? I would say the Powerscheduler. The first thing I would do is disable that plugin and see if the problem goes away.

It might be a good idea to delete the mappings for the channel group that no longer exists (DELETE FROM groupmap WHERE idGroup = 1). Having said that, channel group 1 should be the "all channels" group, and that should never be deleted. Do you have an "all channels" group?

mm

 

[Vasilich]

Hi MM,
i fixed channelgroup yesterday, but this didn't make things faster. Sure i leave it fixed as at least logs are more clear now.
next thing i'm gonna check is disabling powerscheduler. In his options i set "check interval in seconds" to 60, so no idea why it polls every second.

P.S. tested it without powerscheduler - not better. ok, there is no entries in log about powerscheduler anymore, but getting the list of channel is still slow:

2011-06-26 09:24:03.730400 [(5)]: Filter listview for 
2011-06-26 09:24:40.858400 [(5)]: Finished filtering 2429 items for

so retrieving the list of channels still takes 37 seconds.
Any idea what can be wrong then, if not powerscheduler?
new logs attached

BTW. how many channels do you have in your DB? i still believe noone else had tested MP with so many channels scanned.

 

[Vasilich]

other threads approving my belief:
https://forum.team-mediaportal.com/1-0-rc3-svn-builds-288/rc3-slow-epg-46316/
https://forum.team-mediaportal.com/1-0-rc4-svn-builds-298/very-slow-database-performance-49880/

 

[mm1352000]

Hi again

Well it is certainly clear that the listview is taking a long time there. I'm wondering whether it is the listview filtering mechanism or the database interface (Gentle) that is slow/inefficient...

mm