Reply to thread

Message

<blockquote data-quote="zeflash" data-source="post: 80354" data-attributes="member: 13641">I digged a bit more into the HTML parsing. It uses the sgmlreader dll from microsoft and some complex code (sgmlparser) in order to match the template file to the actual html code.After some debugging, I now understand why Fifth gear isn't added properly;The template *needs* a matching description, an IMDB url and a tv.com url.Trouble is for fifth gear there isn't a tv.com url. So even though the other fields are there, it fails to parse properly.Furthermore, the way the parser works means that it will always end up the description on the first BR tag.On Fifth gear, the plot outline of the series uses &lt;BR /&gt; to indent the description =&gt; I'm only getting the first line.So, I ask you this question (since I'm new to this project):What is the need for sgmlparser, when one could use regular expressions to properly parse pieces of the page?For instance, here is the description parsing using sgml:[CODE]&lt;body&gt;Plot Outline:&lt;br&gt;{show_description}&lt;br&gt;[/CODE]that only extracts the text up to the first BR tag.Using Regex, it looks like this one works better:[CODE]Plot Outline:.*?&lt;br&gt;(?&lt;description&gt;.*?)&lt;br&gt;&lt;br&gt;[/CODE]Doesn't look more difficult to understand to me - and it gets the whole thing.And with a little change to the way the template is read by the plugin, some values could just be optional (tv.com or IMDB) so it wouldn't break the whole import process.In the end, my belief is that the SGMLParser.cs file adds rigidity and more complexity to the process for the only upside of being able to write more natural (in a sense) templates.Given that those templates are rarely modified and that regexp, once understood, are pretty easy to write too, it looks like an unnecessary layer to me.For those still reading this thread, what do you think? Do you know of any parsing case that would not work with regexp and would need the SGML parser to work?Edit: Well, or course, it makes it easier to list multiple episodes. Hmm. I'll keep thinking about this. It would really help to know what this sgmlparse class is able to do!I can retrieve the episodelisting info with this:[code]&lt;tr bgcolor=&quot;.*?td.*?&gt;(?&lt;production_number&gt;.*?)&lt;/td&gt;.*?&lt;a href=&quot;/episode/(?&lt;episode_id&gt;.*?)&quot;&gt;(?&lt;episode_season_and_number&gt;.*?)&lt;/td&gt;.*?&lt;a.*?&gt;(?&lt;episode_title&gt;.*?)&lt;/a&gt;.*?&lt;td.*?&gt;(?&lt;production_id&gt;.*?)&lt;/td&gt;.*?&lt;/span&gt;.*?&lt;a.*?td nowrap.*?&gt;(?&lt;broadcast_date&gt;.*?)&lt;/td&gt;[/code]And then cycling through all the different regex matches. That's an alternative to the episodelist[] search tag in the SGMLParser template.I'll try to have an equivalent regexp for all the templates. If I'm able to do that, then I just might change the whole parsing to use those instead of sgmlreader.</blockquote>

[QUOTE="zeflash, post: 80354, member: 13641"] I digged a bit more into the HTML parsing. It uses the sgmlreader dll from microsoft and some complex code (sgmlparser) in order to match the template file to the actual html code. After some debugging, I now understand why Fifth gear isn't added properly; The template *needs* a matching description, an IMDB url and a tv.com url. Trouble is for fifth gear there isn't a tv.com url. So even though the other fields are there, it fails to parse properly. Furthermore, the way the parser works means that it will always end up the description on the first BR tag. On Fifth gear, the plot outline of the series uses to indent the description => I'm only getting the first line. So, I ask you this question (since I'm new to this project): What is the need for sgmlparser, when one could use regular expressions to properly parse pieces of the page? For instance, here is the description parsing using sgml: [CODE]<body>Plot Outline: {show_description} [/CODE] that only extracts the text up to the first BR tag. Using Regex, it looks like this one works better: [CODE]Plot Outline:.*? (?<description>.*?) [/CODE] Doesn't look more difficult to understand to me - and it gets the whole thing. And with a little change to the way the template is read by the plugin, some values could just be optional (tv.com or IMDB) so it wouldn't break the whole import process. In the end, my belief is that the SGMLParser.cs file adds rigidity and more complexity to the process for the only upside of being able to write more natural (in a sense) templates. Given that those templates are rarely modified and that regexp, once understood, are pretty easy to write too, it looks like an unnecessary layer to me. For those still reading this thread, what do you think? Do you know of any parsing case that would not work with regexp and would need the SGML parser to work? Edit: Well, or course, it makes it easier to list multiple episodes. Hmm. I'll keep thinking about this. It would really help to know what this sgmlparse class is able to do! I can retrieve the episodelisting info with this: [code] <tr bgcolor=".*?td.*?>(?<production_number>.*?)</td>.*?<a href="/episode/(?<episode_id>.*?)">(?<episode_season_and_number>.*?)</td>.*?<a.*?>(?<episode_title>.*?)</a>.*?<td.*?>(?<production_id>.*?)</td>.*?.*?<a.*?td nowrap.*?>(?<broadcast_date>.*?)</td> [/code] And then cycling through all the different regex matches. That's an alternative to the episodelist[] search tag in the SGMLParser template. I'll try to have an equivalent regexp for all the templates. If I'm able to do that, then I just might change the whole parsing to use those instead of sgmlreader. [/QUOTE]

Verification