Dish/Bev EPG grabber released in SVN. - Page 2

CommGuru · 2007-04-04, 01:11

I tested it with the latest SVN as of this post and had very mixed results. Scanning a 2 day EPG alone cause the SQL service to rise to 255MB of RAM usage while the TVService.exe used about 100MB of RAM. The scan and sql part of it took about 20 minutes which is somewhat better than what I got a few days ago. To clear the RAM usage required a restart of the TVE3 service. As for the missing channels and programs, I still had quite a few missing and possibly more missing than I did from a couple day old build.

Gemx is the man for all this work into this and it's appreciated. I look forward to continue testing it.

CG

CodeMonkey · 2007-04-04, 14:47

CommGuru could you please post a list of the channels that were missed? Maybe we can see something in common between them.

And/or maybe gemx could give you test code that would do some extra logging if that would be helpful to him.

CommGuru · 2007-04-04, 15:36

It seems the missing channels were ramdon and different each time I killed MP and restarted it and I don't believe the EPG had time to update itself between those restarts.

I'd be happy to do whatever gemx thinks might help.

CG

CodeMonkey · 2007-04-05, 13:54

So in theory if it ran for a while all the information would eventually be in the guide?

CommGuru · 2007-04-05, 14:12

No, I don't believe so. I let it run over night once and still had the missing channel issues the next morning.

ASiDiE · 2007-04-05, 15:37

I also thought the same thing when I was working with GEMX... When the epg starts grabbing it keeps grabbing until it hasn't received any more data after 1 min. I thought this was too quick so we changed it to 2 minutes. I thought for whatever reason, it would pause for over a min.. but still had a little bit of data to send. Because all of the data comes in random I thought that these were the missing shows. This does not seem to be the case.

If you use Tsreader, it has a EPG guide you can display. It seems that the data is random but once it finds a channel it downloads ALL of the data for that channel. You might have skipped channels while it is downloading but never skipped shows.

CodeMonkey · 2007-04-05, 18:40

And if you run the EPG again, the same channels/shows are missing after the second pass as well? Or will the same ones still be missing?

ASiDiE · 2007-04-05, 19:22

Totally random.. you can do a scan.. and have some shows come up missing.. delete the database.. and do the same scan... different shows.

CodeMonkey · 2007-04-05, 19:38

Right but what I was getting at is, if you keep the same database and different shows/channels are missing the second time, do you have them all after 2 passes? Or does the database get wiped automatically by the tvserver between EPG runs?

My point being if you have everything in the database after 2 passes maybe it isn't a huge problem.

gemx · 2007-04-06, 13:40

It won't fill up the missing shows as it reads the latest endtime of a show in the db and imports only newer shows but i found the problem for the skipped shows.
There is a problem in the SectionDecoder class in TsWriter which requires a specific handling of decoding ts packets for DISH.
At the moment i have some difficulties fixing it / finding the correct position where it happens.

2007-04-04, 14:47	#12 (permalink)
CodeMonkey Portal Member Join Date: Dec 2004 Location: North America Posts: 265 Thanks: 1 Thanked 0 Times in 0 Posts Country:	CommGuru could you please post a list of the channels that were missed? Maybe we can see something in common between them. And/or maybe gemx could give you test code that would do some extra logging if that would be helpful to him. __________________ -CodeMonkey P4 2.8gHz, 512MB memory, nVidia FX5200, 40GB HD, 200GB HD OS: Windows XP Home SP2 TVcard(s): Hauppauge PVR500MCE, Hauppauge PVR 250MCE(Installed in development machine) Remote: MCE 2005 (US Model) ________________

2007-04-05, 13:54	#14 (permalink)
CodeMonkey Portal Member Join Date: Dec 2004 Location: North America Posts: 265 Thanks: 1 Thanked 0 Times in 0 Posts Country:	So in theory if it ran for a while all the information would eventually be in the guide? __________________ -CodeMonkey P4 2.8gHz, 512MB memory, nVidia FX5200, 40GB HD, 200GB HD OS: Windows XP Home SP2 TVcard(s): Hauppauge PVR500MCE, Hauppauge PVR 250MCE(Installed in development machine) Remote: MCE 2005 (US Model) ________________

2007-04-05, 15:37	#16 (permalink)
ASiDiE Portal Tester Join Date: Jan 2005 Location: USA Age: 32 Posts: 888 Thanks: 6 Thanked 2 Times in 2 Posts My System	I also thought the same thing when I was working with GEMX... When the epg starts grabbing it keeps grabbing until it hasn't received any more data after 1 min. I thought this was too quick so we changed it to 2 minutes. I thought for whatever reason, it would pause for over a min.. but still had a little bit of data to send. Because all of the data comes in random I thought that these were the missing shows. This does not seem to be the case. If you use Tsreader, it has a EPG guide you can display. It seems that the data is random but once it finds a channel it downloads ALL of the data for that channel. You might have skipped channels while it is downloading but never skipped shows. __________________ I SEE WHAT YOU DID THERE!

2007-04-05, 18:40	#17 (permalink)
CodeMonkey Portal Member Join Date: Dec 2004 Location: North America Posts: 265 Thanks: 1 Thanked 0 Times in 0 Posts Country:	And if you run the EPG again, the same channels/shows are missing after the second pass as well? Or will the same ones still be missing? __________________ -CodeMonkey P4 2.8gHz, 512MB memory, nVidia FX5200, 40GB HD, 200GB HD OS: Windows XP Home SP2 TVcard(s): Hauppauge PVR500MCE, Hauppauge PVR 250MCE(Installed in development machine) Remote: MCE 2005 (US Model) ________________

2007-04-05, 19:22	#18 (permalink)
ASiDiE Portal Tester Join Date: Jan 2005 Location: USA Age: 32 Posts: 888 Thanks: 6 Thanked 2 Times in 2 Posts My System	Totally random.. you can do a scan.. and have some shows come up missing.. delete the database.. and do the same scan... different shows. __________________ I SEE WHAT YOU DID THERE!

2007-04-04, 01:11	#11 (permalink)
CommGuru Portal Member Join Date: Oct 2006 Age: 31 Posts: 50 Thanks: 1 Thanked 0 Times in 0 Posts	I tested it with the latest SVN as of this post and had very mixed results. Scanning a 2 day EPG alone cause the SQL service to rise to 255MB of RAM usage while the TVService.exe used about 100MB of RAM. The scan and sql part of it took about 20 minutes which is somewhat better than what I got a few days ago. To clear the RAM usage required a restart of the TVE3 service. As for the missing channels and programs, I still had quite a few missing and possibly more missing than I did from a couple day old build. Gemx is the man for all this work into this and it's appreciated. I look forward to continue testing it. CG

2007-04-04, 15:36	#13 (permalink)
CommGuru Portal Member Join Date: Oct 2006 Age: 31 Posts: 50 Thanks: 1 Thanked 0 Times in 0 Posts	It seems the missing channels were ramdon and different each time I killed MP and restarted it and I don't believe the EPG had time to update itself between those restarts. I'd be happy to do whatever gemx thinks might help. CG

2007-04-05, 14:12	#15 (permalink)
CommGuru Portal Member Join Date: Oct 2006 Age: 31 Posts: 50 Thanks: 1 Thanked 0 Times in 0 Posts	No, I don't believe so. I let it run over night once and still had the missing channel issues the next morning.

2007-04-05, 19:38	#19 (permalink)
CodeMonkey Portal Member Join Date: Dec 2004 Location: North America Posts: 265 Thanks: 1 Thanked 0 Times in 0 Posts Country:	Right but what I was getting at is, if you keep the same database and different shows/channels are missing the second time, do you have them all after 2 passes? Or does the database get wiped automatically by the tvserver between EPG runs? My point being if you have everything in the database after 2 passes maybe it isn't a huge problem. __________________ -CodeMonkey P4 2.8gHz, 512MB memory, nVidia FX5200, 40GB HD, 200GB HD OS: Windows XP Home SP2 TVcard(s): Hauppauge PVR500MCE, Hauppauge PVR 250MCE(Installed in development machine) Remote: MCE 2005 (US Model) ________________

2007-04-06, 13:40	#20 (permalink)
gemx Portal Developer Join Date: Oct 2006 Age: 32 Posts: 1,227 Thanks: 22 Thanked 183 Times in 96 Posts Country: My System	It won't fill up the missing shows as it reads the latest endtime of a show in the db and imports only newer shows but i found the problem for the skipped shows. There is a problem in the SectionDecoder class in TsWriter which requires a specific handling of decoding ts packets for DISH. At the moment i have some difficulties fixing it / finding the correct position where it happens. __________________ ACCENT HT-100 \| 2x16 VFD \| ADM Athlon 64 3500+ \| Win XP SP2 \| SkyStar 2 DVB-S PCI \| TT connect S2-3600

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Skin release] SimpleMP Beta 4 (MP 0.2.3.0)	mikael	Simple MP	241	2008-09-04 19:39
(original thread) US/Canadian/Mexico EPG Zap2it Plugin Released	bradsjm	zap2it (discontinued)	513	2007-02-21 14:07
how to get released source from SVN?	garry	General Development (no feature request here!)	2	2006-11-16 06:48
DVB-EPG Grabber	chris-at	WebEPG	1	2006-08-25 06:33