Reply to thread

Message

<blockquote data-quote="RoChess" data-source="post: 1096485" data-attributes="member: 18896">movingpictures.zip:&nbsp;[code]18-Aug-2014 10:45:06 Debug [&nbsp; ScraperNode]: executing parse: &lt;parse name=&quot;writers&quot; input=&quot;${details[0].drehbuch}&quot; xpath=&quot;//name&quot; /&gt;18-Aug-2014 10:45:06 Debug [&nbsp; ScraperNode]: Assigned variable: writers.count = 118-Aug-2014 10:45:06 Debug [&nbsp; ScraperNode]: Assigned variable: writers[0] = Christopher Miller18-Aug-2014 10:45:06 Debug [&nbsp; ScraperNode]: Assigned variable: movie.writers =18-Aug-2014 10:45:06 Debug [&nbsp; ScraperNode]: Assigned variable: writer = Christopher Miller18-Aug-2014 10:45:06 Debug [&nbsp; ScraperNode]: Assigned variable: count = 018-Aug-2014 10:45:06 Debug [&nbsp; ScraperNode]: Assigned variable: movie.writers = |Christopher Miller[/code]&nbsp;So that should result in &quot;Christopher Miller&quot;, and does not demonstrate the failure you mentioned.&nbsp;movingpictures2.zip gives me:&nbsp;[code]18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: rx_writers_block = (?&lt;=Writing\scredits)(?&lt;WritersBlock&gt;.*?)(?=&lt;/table&gt;)18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: rx_writers = (?:&gt;)([^&lt;]+?)(?:&lt;/)18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: rx_cmnt = ($\s*WGA\s*$)|($in alphabetical order$)|(\sand\s)|(&amp;)18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: executing parse: &lt;parse name=&quot;writers_block&quot; input=&quot;${cast_page:htmldecode}&quot; regex=&quot;${rx_writers_block}&quot; /&gt;18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: name: writers_block ||| pattern: (?&lt;=Writing\scredits)(?&lt;WritersBlock&gt;.*?)(?=&lt;/table&gt;) ||| input: [not logged due to size]18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: writers_block.count = 118-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: movie.writers =18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: executing replace: &lt;replace name=&quot;writers&quot; input=&quot;${writers_block}&quot; pattern=&quot;${rx_tag}&quot; with=&quot; &quot; /&gt;18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: writers =&nbsp; Erica Rivinoja&nbsp; ...&nbsp; (screenplay) &amp;&nbsp; John Francis Daley&nbsp; ...&nbsp; (screenplay) &amp;&nbsp; Jonathan M. Goldstein&nbsp; ...&nbsp; (screenplay) (as Jonathan Goldstein)&nbsp; Phil Lord&nbsp; ...&nbsp; (story) &amp;&nbsp; Christopher Miller&nbsp; ...&nbsp; (story) &amp;&nbsp; Erica Rivinoja&nbsp; ...&nbsp; (story)&nbsp; Judi Barrett&nbsp; ...&nbsp; (characters) &amp;&nbsp; Ron Barrett&nbsp; ...&nbsp; (characters)18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: executing replace: &lt;replace name=&quot;writers&quot; input=&quot;${writers}&quot; pattern=&quot;${rx_cmnt}&quot; with=&quot; &quot; /&gt;18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: writers =&nbsp; Erica Rivinoja&nbsp; ...&nbsp; (screenplay)&nbsp; John Francis Daley&nbsp; ...&nbsp; (screenplay)&nbsp; Jonathan M. Goldstein&nbsp; ...&nbsp; (screenplay) (as Jonathan Goldstein)&nbsp; Phil Lord&nbsp; ...&nbsp; (story)&nbsp; Christopher Miller&nbsp; ...&nbsp; (story)&nbsp; Erica Rivinoja&nbsp; ...&nbsp; (story)&nbsp; Judi Barrett&nbsp; ...&nbsp; (characters)&nbsp; Ron Barrett&nbsp; ...&nbsp; (characters)18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: executing replace: &lt;replace name=&quot;writers&quot; input=&quot;${writers}&quot; pattern=&quot;\s+&quot; with=&quot; &quot; /&gt;18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: writers =&nbsp; Erica Rivinoja ... (screenplay) John Francis Daley ... (screenplay) Jonathan M. Goldstein ... (screenplay) (as Jonathan Goldstein) Phil Lord ... (story) Christopher Miller ... (story) Erica Rivinoja ... (story) Judi Barrett ... (characters) Ron Barrett ... (characters)18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: executing replace: &lt;replace name=&quot;writers&quot; input=&quot;${writers}&quot; pattern=&quot;\)&quot; with=&quot;)|&quot; /&gt;18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: writers =&nbsp; Erica Rivinoja ... (screenplay)| John Francis Daley ... (screenplay)| Jonathan M. Goldstein ... (screenplay)| (as Jonathan Goldstein)| Phil Lord ... (story)| Christopher Miller ... (story)| Erica Rivinoja ... (story)| Judi Barrett ... (characters)| Ron Barrett ... (characters)|18-Aug-2014 11:35:08 Debug [&nbsp; ScraperNode]: Assigned variable: movie.writers =&nbsp; Erica Rivinoja ... (screenplay)| John Francis Daley ... (screenplay)| Jonathan M. Goldstein ... (screenplay)| (as Jonathan Goldstein)| Phil Lord ... (story)| Christopher Miller ... (story)| Erica Rivinoja ... (story)| Judi Barrett ... (characters)| Ron Barrett ... (characters)|[/code]&nbsp;So we got a winner there.&nbsp;The Regular Expression &quot;rx_cmnt&quot; is what is used to clean up the results (weird way though), and current value is &quot;($\s*WGA\s*$)|($in alphabetical order$)|(\sand\s)|(&amp;)&quot;&nbsp;No idea why Merlyn made the stuff between parenthasis static, as I can not think of any reason that should be used.&nbsp;So change &quot;rx_cmnt&quot; into:&nbsp;&nbsp; (?:$as[^)]+$)|(?:$[^)]+)|(?:\s*\.{2,}\s*)|(?:&amp;)&nbsp;And &quot;&lt;replace name=&quot;writers&quot; input=&quot;${writers}&quot; pattern=&quot;$&quot; with=&quot;)|&quot; /&gt;&quot; into:&nbsp; &lt;replace name=&quot;writers&quot; input=&quot;${writers}&quot; pattern=&quot;\)&quot; with=&quot;|&quot; /&gt;&nbsp;And you will be good to go.&nbsp;Really weird way that was done on multiple lines, as it can all be done in a single Regular Expression, but that requires more work from me to explain, so you'll have to settle for this easy 'ugly' fix.&nbsp;PS: the bold stuff is the only stuff you end up adjusting into.</blockquote>

[QUOTE="RoChess, post: 1096485, member: 18896"] movingpictures.zip: [code] 18-Aug-2014 10:45:06 Debug [ ScraperNode]: executing parse: <parse name="writers" input="${details[0].drehbuch}" xpath="//name" /> 18-Aug-2014 10:45:06 Debug [ ScraperNode]: Assigned variable: writers.count = 1 18-Aug-2014 10:45:06 Debug [ ScraperNode]: Assigned variable: writers[0] = Christopher Miller 18-Aug-2014 10:45:06 Debug [ ScraperNode]: Assigned variable: movie.writers = 18-Aug-2014 10:45:06 Debug [ ScraperNode]: Assigned variable: writer = Christopher Miller 18-Aug-2014 10:45:06 Debug [ ScraperNode]: Assigned variable: count = 0 18-Aug-2014 10:45:06 Debug [ ScraperNode]: Assigned variable: movie.writers = |Christopher Miller [/code] So that should result in "Christopher Miller", and does not demonstrate the failure you mentioned. movingpictures2.zip gives me: [code] 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: rx_writers_block = (?<=Writing\scredits)(?<WritersBlock>.*?)(?=</table>) 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: rx_writers = (?:>)([^<]+?)(?:</) 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: rx_cmnt = ($\s*WGA\s*$)|($in alphabetical order$)|(\sand\s)|(&) 18-Aug-2014 11:35:08 Debug [ ScraperNode]: executing parse: <parse name="writers_block" input="${cast_page:htmldecode}" regex="${rx_writers_block}" /> 18-Aug-2014 11:35:08 Debug [ ScraperNode]: name: writers_block ||| pattern: (?<=Writing\scredits)(?<WritersBlock>.*?)(?=</table>) ||| input: [not logged due to size] 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: writers_block.count = 1 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: movie.writers = 18-Aug-2014 11:35:08 Debug [ ScraperNode]: executing replace: <replace name="writers" input="${writers_block}" pattern="${rx_tag}" with=" " /> 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: writers = Erica Rivinoja ... (screenplay) & John Francis Daley ... (screenplay) & Jonathan M. Goldstein ... (screenplay) (as Jonathan Goldstein) Phil Lord ... (story) & Christopher Miller ... (story) & Erica Rivinoja ... (story) Judi Barrett ... (characters) & Ron Barrett ... (characters) 18-Aug-2014 11:35:08 Debug [ ScraperNode]: executing replace: <replace name="writers" input="${writers}" pattern="${rx_cmnt}" with=" " /> 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: writers = Erica Rivinoja ... (screenplay) John Francis Daley ... (screenplay) Jonathan M. Goldstein ... (screenplay) (as Jonathan Goldstein) Phil Lord ... (story) Christopher Miller ... (story) Erica Rivinoja ... (story) Judi Barrett ... (characters) Ron Barrett ... (characters) 18-Aug-2014 11:35:08 Debug [ ScraperNode]: executing replace: <replace name="writers" input="${writers}" pattern="\s+" with=" " /> 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: writers = Erica Rivinoja ... (screenplay) John Francis Daley ... (screenplay) Jonathan M. Goldstein ... (screenplay) (as Jonathan Goldstein) Phil Lord ... (story) Christopher Miller ... (story) Erica Rivinoja ... (story) Judi Barrett ... (characters) Ron Barrett ... (characters) 18-Aug-2014 11:35:08 Debug [ ScraperNode]: executing replace: <replace name="writers" input="${writers}" pattern="\)" with=")|" /> 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: writers = Erica Rivinoja ... (screenplay)| John Francis Daley ... (screenplay)| Jonathan M. Goldstein ... (screenplay)| (as Jonathan Goldstein)| Phil Lord ... (story)| Christopher Miller ... (story)| Erica Rivinoja ... (story)| Judi Barrett ... (characters)| Ron Barrett ... (characters)| 18-Aug-2014 11:35:08 Debug [ ScraperNode]: Assigned variable: movie.writers = Erica Rivinoja ... (screenplay)| John Francis Daley ... (screenplay)| Jonathan M. Goldstein ... (screenplay)| (as Jonathan Goldstein)| Phil Lord ... (story)| Christopher Miller ... (story)| Erica Rivinoja ... (story)| Judi Barrett ... (characters)| Ron Barrett ... (characters)| [/code] So we got a winner there. The Regular Expression "rx_cmnt" is what is used to clean up the results (weird way though), and current value is "($\s*WGA\s*$)|($in alphabetical order$)|(\sand\s)|(&)" No idea why Merlyn made the stuff between parenthasis static, as I can not think of any reason that should be used. So change "rx_cmnt" into: [B](?:$as[^)]+$)|(?:$[^)]+)|(?:\s*\.{2,}\s*)|(?:&)[/B] And "<replace name="writers" input="${writers}" pattern="$" with=")|" />" into: [B]<replace name="writers" input="${writers}" pattern="\)" with="|" />[/B] And you will be good to go. Really weird way that was done on multiple lines, as it can all be done in a single Regular Expression, but that requires more work from me to explain, so you'll have to settle for this easy 'ugly' fix. PS: the [B]bold[/B] stuff is the only stuff you end up adjusting into. [/QUOTE]

Verification