Reply to thread

Message

<blockquote data-quote="RoChess" data-source="post: 1271531" data-attributes="member: 18896">What you could do is this:[code]&lt;retrieve&gt;&lt;store what you need in a variable, or process first page completely&gt;&lt;if first-page-contains-a-reference-to-next-page-via-regex-match == true&gt;&nbsp; &nbsp; &lt;retrieve second page&gt;&nbsp; &nbsp; &lt;add what you need to the same variable, or process second page completely&gt;&lt;/if&gt;&lt;if second-page-contains-a-reference-to-third-page-via-regex-match == true&gt;&nbsp; &nbsp; &lt;retrieve third page&gt;&nbsp; &nbsp; &lt;add what you need to the same variable, or process third page completely&gt;&lt;/if&gt;&lt;process variable if you did not process each page already&gt;[/code]Processing 3-pages should be enough i would think, based on your example of a movie's proper result not found until page-3.You can also re-investigate if you can tweak your search query, and maybe bring in the year to narrow down the results, so that instead of scraping 3-pages you adjust your search Kung-Fu to ensure the proper movie shows on first page, so you never have to scrape more than one page <img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite11" alt=":rolleyes:" title="Roll Eyes&nbsp; &nbsp; :rolleyes:" loading="lazy" data-shortname=":rolleyes:" /></blockquote>

[QUOTE="RoChess, post: 1271531, member: 18896"] What you could do is this: [code] <retrieve> <store what you need in a variable, or process first page completely> <if first-page-contains-a-reference-to-next-page-via-regex-match == true> <retrieve second page> <add what you need to the same variable, or process second page completely> </if> <if second-page-contains-a-reference-to-third-page-via-regex-match == true> <retrieve third page> <add what you need to the same variable, or process third page completely> </if> <process variable if you did not process each page already> [/code] Processing 3-pages should be enough i would think, based on your example of a movie's proper result not found until page-3. You can also re-investigate if you can tweak your search query, and maybe bring in the year to narrow down the results, so that instead of scraping 3-pages you adjust your search Kung-Fu to ensure the proper movie shows on first page, so you never have to scrape more than one page :rolleyes: [/QUOTE]

Verification