Reply to thread

Message

<blockquote data-quote="Alphathon" data-source="post: 924106" data-attributes="member: 105814">I imagine this scenario isn't all that common so I understand if no-one has an answer to this. Regardless, here goes.&nbsp;As good as WebEPG is, it is a tad limited, and doesn't seem to handle multiple variations within a listings page very well. As such, I thought I'd try to pre-process listings using a php script. This also has the advantage that I can reformat parts of the listing to my preference, combine parts from multiple parts of the listing into one field (e.g. combining information on cast, video format etc into the description field where available. So far I have successfully created one such script, as much as a test/experiments as anything else, which takes information from zdf.de and formats it in nice clean HTML tables as shown below:&nbsp;Basic format:[HTML]&lt;table border=&quot;1&quot;&gt;&lt;tr&gt;&lt;th&gt;#TITLE&lt;/th&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;#SUBTITLE&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;#EPISODE&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;#START&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;#DESCRIPTION&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;#GENRE&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;[/HTML]Example:[HTML]&lt;table border=&quot;1&quot;&gt;&lt;tr&gt;&lt;th&gt;Zwei Männer am Herd&lt;/th&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Wie Pfeffer und Salz&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;06:15&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Die Liebeswelt hat ihn endlich wieder. Walter hat sich entschlossen, Giulietta zu erobern, bewaffnet mit einem Italienischsprachkurs, einem Riesenblumenstrauß und einer nächtlich romantischen Einladung zum Hamburger Dom. Von seiner guten Laune profitiert auch Hilmar, der Walter um Wiederaufnahme ins Conrad's bittet und von ihm zu Moritz' Verwunderung prompt eine Zusage erhält.&nbsp;Nicht weniger erstaunt ist Moritz, als in der folgenden Nacht Walter bei ihm mit wenigen Habseligkeiten vor der Wohnung steht. Sein Hausboot ist aus unerklärlichen Gründen auf Grund gelaufen. Ob Conrads Rachefeldzug dahintersteckt? Auf jeden Fall ist ihm die miese Kritik im örtlichen Restaurantführer zuzuschreiben.&nbsp;Land: DeutschlandJahr: 2000Bildformat: 16:9&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Serie&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;[/HTML]&nbsp;So far so good, the php script works; now for the grabber. I have tried out the script in WebEPG designer and it finds all the listings using the following text along with the &quot;T&quot; tag:&nbsp;[CODE]&amp;lt;table border=&quot;1&quot;&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;th&amp;gt;&amp;lt;#TITLE&amp;gt;&amp;lt;/th&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#SUBTITLE&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#EPISODE&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#START&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#DESCRIPTION&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#GENRE&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;/table&amp;gt;[/CODE]&nbsp;The address used to test it in this case was -ttp://localhost/public_HTML/ZDFlistings.php?id=ZDF.kultur&amp;y=2012&amp;m=10&amp;d=16 where ZDF.kultur is the channel and the date is 2012-10-16 (and -ttp = http so as to avoid making it a link). Again, so far so good.&nbsp;So, then I make up a grabber file, which is shown below:&nbsp;[HTML]&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;&lt;Grabber&gt;&nbsp; &nbsp; &lt;Info language=&quot;de&quot; availableDays=&quot;14&quot; timezone=&quot;W. Europe Standard Time&quot; version=&quot;2.0&quot; /&gt;&nbsp; &nbsp; &lt;Channels&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;ZDF&quot; siteId=&quot;ZDF&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;ZDF_neo&quot; siteId=&quot;ZDFneo&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;ZDFinfo&quot; siteId=&quot;ZDFinfo&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;ZDF.kultur&quot; siteId=&quot;ZDF.kultur&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;3sat&quot; siteId=&quot;3sat&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;Phoenix&quot; siteId=&quot;phoenix&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;arte&quot; siteId=&quot;arte&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Channel id=&quot;KiKA&quot; siteId=&quot;KI.KA&quot;/&gt;&nbsp; &nbsp; &lt;/Channels&gt;&nbsp; &nbsp; &lt;Listing type=&quot;Html&quot;&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Site url=&quot;http://localhost/public_HTML/ZDFlistings.php?id=[ID]&amp;amp;y=[YYYY]&amp;amp;m=[MM]&amp;amp;d=[DD]&quot; external=&quot;false&quot; encoding=&quot;utf-8&quot;/&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;Html&gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &lt;Template name=&quot;default&quot;&gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &lt;SectionTemplate tags=&quot;T&quot;&gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &lt;TemplateText&gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;table border=&quot;1&quot;&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;tr&amp;gt;&amp;lt;th&amp;gt;&amp;lt;#TITLE&amp;gt;&amp;lt;/th&amp;gt;&amp;lt;/tr&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#SUBTITLE&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#EPISODE&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#START&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#DESCRIPTION&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;lt;#GENRE&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &amp;lt;/table&amp;gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &lt;/TemplateText&gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &lt;/SectionTemplate&gt;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &lt;/Template&gt;&nbsp; &nbsp; &nbsp; &nbsp; &lt;/Html&gt;&nbsp; &nbsp; &lt;/Listing&gt;&lt;/Grabber&gt;[/HTML]&nbsp;Unfortunately, when WebEPG tries to grab the data, it finds no listings, with the following recorded in tv.log (in this case when grabbing info for ZDFinfo for 2012-10-16):&nbsp;[CODE]2012-10-16 17:13:00.981506 [WebEPGImporter(9)]: WebEPG: Getting Channel ID: ZDFinfo2012-10-16 17:13:00.985507 [WebEPGImporter(9)]:&nbsp; &nbsp; &nbsp; &nbsp; [10 of 46]2012-10-16 17:13:00.988507 [WebEPGImporter(9)]: WebEPG: ChannelId: ZDFinfo2012-10-16 17:13:00.991507 [WebEPGImporter(9)]: WebEPG: Grab Start 17:13 16/10/20122012-10-16 17:13:01.114514 [WebEPGImporter(9)]: WebEPG: Reading http://localhost/public_HTML/ZDFlistings.php?id=ZDFinfo&amp;y=2012&amp;m=10&amp;d=16 POST:&nbsp; - Delay: 500ms2012-10-16 17:13:56.703694 [WebEPGImporter(9)]: HTMLPage: GetInternal encoding: Forced: utf-82012-10-16 17:13:56.707694 [WebEPGImporter(9)]: WebEPG: No Listings Found2012-10-16 17:13:56.730695 [WebEPGImporter(9)]: WebEPG: Writing Channel: ZDFinfo[/CODE]&nbsp;Have I made some stupid error with the grabber, or does this have something to do with it being hosted on localhost? Could it even be some kind of timeout issue (each page can take quite a while to load)?</blockquote>

[QUOTE="Alphathon, post: 924106, member: 105814"] I imagine this scenario isn't all that common so I understand if no-one has an answer to this. Regardless, here goes. As good as WebEPG is, it is a tad limited, and doesn't seem to handle multiple variations within a listings page very well. As such, I thought I'd try to pre-process listings using a php script. This also has the advantage that I can reformat parts of the listing to my preference, combine parts from multiple parts of the listing into one field (e.g. combining information on cast, video format etc into the description field where available. So far I have successfully created one such script, as much as a test/experiments as anything else, which takes information from zdf.de and formats it in nice clean HTML tables as shown below: [B]Basic format:[/B] [HTML]<table border="1"> <tr><th>#TITLE</th></tr> <tr><td>#SUBTITLE</td></tr> <tr><td>#EPISODE</td></tr> <tr><td>#START</td></tr> <tr><td>#DESCRIPTION</td></tr> <tr><td>#GENRE</td></tr> </table>[/HTML] [B]Example:[/B] [HTML]<table border="1"> <tr><th>Zwei Männer am Herd</th></tr> <tr><td>Wie Pfeffer und Salz</td></tr> <tr><td></td></tr> <tr><td>06:15</td></tr> <tr><td>Die Liebeswelt hat ihn endlich wieder. Walter hat sich entschlossen, Giulietta zu erobern, bewaffnet mit einem Italienischsprachkurs, einem Riesenblumenstrauß und einer nächtlich romantischen Einladung zum Hamburger Dom. Von seiner guten Laune profitiert auch Hilmar, der Walter um Wiederaufnahme ins Conrad's bittet und von ihm zu Moritz' Verwunderung prompt eine Zusage erhält. Nicht weniger erstaunt ist Moritz, als in der folgenden Nacht Walter bei ihm mit wenigen Habseligkeiten vor der Wohnung steht. Sein Hausboot ist aus unerklärlichen Gründen auf Grund gelaufen. Ob Conrads Rachefeldzug dahintersteckt? Auf jeden Fall ist ihm die miese Kritik im örtlichen Restaurantführer zuzuschreiben. Land: Deutschland Jahr: 2000 Bildformat: 16:9</td></tr> <tr><td>Serie</td></tr> </table>[/HTML] So far so good, the php script works; now for the grabber. I have tried out the script in WebEPG designer and it finds all the listings using the following text along with the "T" tag: [CODE]<table border="1"> <tr><th><#TITLE></th></tr> <tr><td><#SUBTITLE></td></tr> <tr><td><#EPISODE></td></tr> <tr><td><#START></td></tr> <tr><td><#DESCRIPTION></td></tr> <tr><td><#GENRE></td></tr> </table>[/CODE] The address used to test it in this case was [I]-ttp://localhost/public_HTML/ZDFlistings.php?id=ZDF.kultur&y=2012&m=10&d=16[/I] where ZDF.kultur is the channel and the date is 2012-10-16 (and -ttp = http so as to avoid making it a link). Again, so far so good. So, then I make up a grabber file, which is shown below: [HTML]<?xml version="1.0" encoding="utf-8"?> <Grabber> <Info language="de" availableDays="14" timezone="W. Europe Standard Time" version="2.0" /> <Channels> <Channel id="ZDF" siteId="ZDF"/> <Channel id="ZDF_neo" siteId="ZDFneo"/> <Channel id="ZDFinfo" siteId="ZDFinfo"/> <Channel id="ZDF.kultur" siteId="ZDF.kultur"/> <Channel id="3sat" siteId="3sat"/> <Channel id="Phoenix" siteId="phoenix"/> <Channel id="arte" siteId="arte"/> <Channel id="KiKA" siteId="KI.KA"/> </Channels> <Listing type="Html"> <Site url="http://localhost/public_HTML/ZDFlistings.php?id=[ID]&y=[YYYY]&m=[MM]&d=[DD]" external="false" encoding="utf-8"/> <Html> <Template name="default"> <SectionTemplate tags="T"> <TemplateText> <table border="1"> <tr><th><#TITLE></th></tr> <tr><td><#SUBTITLE></td></tr> <tr><td><#EPISODE></td></tr> <tr><td><#START></td></tr> <tr><td><#DESCRIPTION></td></tr> <tr><td><#GENRE></td></tr> </table> </TemplateText> </SectionTemplate> </Template> </Html> </Listing> </Grabber>[/HTML] Unfortunately, when WebEPG tries to grab the data, it finds no listings, with the following recorded in tv.log (in this case when grabbing info for ZDFinfo for 2012-10-16): [CODE]2012-10-16 17:13:00.981506 [WebEPGImporter(9)]: WebEPG: Getting Channel ID: ZDFinfo 2012-10-16 17:13:00.985507 [WebEPGImporter(9)]: [10 of 46] 2012-10-16 17:13:00.988507 [WebEPGImporter(9)]: WebEPG: ChannelId: ZDFinfo 2012-10-16 17:13:00.991507 [WebEPGImporter(9)]: WebEPG: Grab Start 17:13 16/10/2012 2012-10-16 17:13:01.114514 [WebEPGImporter(9)]: WebEPG: Reading http://localhost/public_HTML/ZDFlistings.php?id=ZDFinfo&y=2012&m=10&d=16 POST: - Delay: 500ms 2012-10-16 17:13:56.703694 [WebEPGImporter(9)]: HTMLPage: GetInternal encoding: Forced: utf-8 2012-10-16 17:13:56.707694 [WebEPGImporter(9)]: WebEPG: No Listings Found 2012-10-16 17:13:56.730695 [WebEPGImporter(9)]: WebEPG: Writing Channel: ZDFinfo[/CODE] Have I made some stupid error with the grabber, or does this have something to do with it being hosted on localhost? Could it even be some kind of timeout issue (each page can take quite a while to load)? [/QUOTE]

Verification