home
products
contribute
download
documentation
forum
Home
Forums
New posts
Search forums
What's new
New posts
All posts
Latest activity
Members
Registered members
Current visitors
Donate
Log in
Register
What's new
Search
Search
Search titles only
By:
New posts
Search forums
Search titles only
By:
Menu
Log in
Register
Navigation
Install the app
Install
More options
Contact us
Close Menu
Forums
MediaPortal 1
MediaPortal 1 Plugins
Popular Plugins
Moving Pictures
FilmInfo+ - A german movie details scraper with auto grouping
Contact us
RSS
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Message
<blockquote data-quote="RoChess" data-source="post: 944012" data-attributes="member: 18896"><p>@<a href="https://forum.team-mediaportal.com/members/merlyn.114248/" target="_blank">Merlyn</a>, one solution I thought off for the replication of the load-balancer was to scrape the mediaportal.log file itself (the folder location is now a variable in MovPic). RegExp the timestamp and find a random value of 1, 2 or 3. This will then determine what mirror you hit first, out of an array you can build inside your scraper-script.</p><p> </p><p>Something like:</p><p> </p><p>[code]</p><p> <set name="ofdb_mirrors_array"></p><p> <![CDATA[</p><p> #[URL='http://ofdbgw.home-of-root.de/']http://ofdbgw.home-of-root.de[/URL]/#</p><p> #[URL]http://ofdbgw.lirzg.net/[/URL]#</p><p> #[URL]http://ofdbgw.scheeper.net/[/URL]#</p><p> #[URL='http://ofdbgw.home-of-root.de/']http://ofdbgw.home-of-root.de[/URL]/#</p><p> #[URL]http://ofdbgw.lirzg.net/[/URL]#</p><p> ]]></p><p> </set></p><p><parse name="ofdb_mirrors" input="${ofdb_mirrors_array}" regex="#([^#]+)#" /></p><p>[/code]</p><p> </p><p>and then you can use "${ofdb_mirrors[0]}movie/" and "${ofdb_mirrors[0]}imdb2ofdb/" as before to use "<a href="http://ofdbgw.home-of-root.de/" target="_blank">http://ofdbgw.home-of-root.de/</a>" site, but you can also use [1], and [2]. By then replacing the 0, 1, or 2 value (arrays start at '0', so keep that in mind) with the random value obtained from mediaportal.log file you can do ${ofdb_mirrors[${random_value}]} and do random_value + 1 on failure and random_value + 1 once more on another failure.</p><p> </p><p>That is why I repeated the 1st and 2nd mirror in the array. Of course you can also add smarter logic to the script that when random_value + 1 > 2 to make it '0', but you get the idea.</p><p> </p><p>Of course you can also stick with what you got if it is no problem to the OFDb mirrors, but the fact they have a load balancer in place makes me think they want to see an even spread of users hitting it. Since the data is not reliable on each mirror you have to hit them all 3 in order, but the above code should help to make that random.</p><p> </p><p>You will have to figure out a way to get 0/1/2 value out of mediaportal.log file though, my guess is you simply look for (?:0|1|2) at the seconds+miliseconds part of the timestamp, cuz 2012 year will otherwise ruin the random part. Whatever match hits first is your random value to go with then.</p><p> </p><p>PS: Glad you are back, cuz I didn't see myself having time to dig into the details-node to fix the summary problem.</p></blockquote><p></p>
[QUOTE="RoChess, post: 944012, member: 18896"] @[URL='https://forum.team-mediaportal.com/members/merlyn.114248/']Merlyn[/URL], one solution I thought off for the replication of the load-balancer was to scrape the mediaportal.log file itself (the folder location is now a variable in MovPic). RegExp the timestamp and find a random value of 1, 2 or 3. This will then determine what mirror you hit first, out of an array you can build inside your scraper-script. Something like: [code] <set name="ofdb_mirrors_array"> <![CDATA[ #[URL='http://ofdbgw.home-of-root.de/']http://ofdbgw.home-of-root.de[/URL]/# #[URL]http://ofdbgw.lirzg.net/[/URL]# #[URL]http://ofdbgw.scheeper.net/[/URL]# #[URL='http://ofdbgw.home-of-root.de/']http://ofdbgw.home-of-root.de[/URL]/# #[URL]http://ofdbgw.lirzg.net/[/URL]# ]]> </set> <parse name="ofdb_mirrors" input="${ofdb_mirrors_array}" regex="#([^#]+)#" /> [/code] and then you can use "${ofdb_mirrors[0]}movie/" and "${ofdb_mirrors[0]}imdb2ofdb/" as before to use "[url]http://ofdbgw.home-of-root.de/[/url]" site, but you can also use [1], and [2]. By then replacing the 0, 1, or 2 value (arrays start at '0', so keep that in mind) with the random value obtained from mediaportal.log file you can do ${ofdb_mirrors[${random_value}]} and do random_value + 1 on failure and random_value + 1 once more on another failure. That is why I repeated the 1st and 2nd mirror in the array. Of course you can also add smarter logic to the script that when random_value + 1 > 2 to make it '0', but you get the idea. Of course you can also stick with what you got if it is no problem to the OFDb mirrors, but the fact they have a load balancer in place makes me think they want to see an even spread of users hitting it. Since the data is not reliable on each mirror you have to hit them all 3 in order, but the above code should help to make that random. You will have to figure out a way to get 0/1/2 value out of mediaportal.log file though, my guess is you simply look for (?:0|1|2) at the seconds+miliseconds part of the timestamp, cuz 2012 year will otherwise ruin the random part. Whatever match hits first is your random value to go with then. PS: Glad you are back, cuz I didn't see myself having time to dig into the details-node to fix the summary problem. [/QUOTE]
Insert quotes…
Verification
Post reply
Forums
MediaPortal 1
MediaPortal 1 Plugins
Popular Plugins
Moving Pictures
FilmInfo+ - A german movie details scraper with auto grouping
Contact us
RSS
Top
Bottom