My Anime plugin (2 Viewers)

[FdL]

@Bert-r: Where is that "web service" option? I want to try that, at least to see how this works with the few series I have in my HD.

lwerndly: I have to say that I access said anime files through the LAN, they are not local files. Maybe that's why it's so slow. But in any case, with them already having AniDB hashes in the file name, I don't think it's right for the program to re-scan them for identification...

Yes, I too see the naming conventions being a major hurdle for this project. You can't just use the ones in MP-TVSeries, or you'd have to tweak the heck out of them. Also the seasons problem.

With DVDs the thing is lke this. I burn compilations with the full series I download. Having over 70 DVDs, I have yet to make an index to make it easier for searches. Now, with plugins like MP-TVSeries it records the volume name of the DVD, so I can instantly know which disc to look for. Or I could even use the same database from an external program, to search for my stuff or pirnt lists. I like this kind of thing.

OOT: The whole Blue3 name thing in the context of anime inevitably reminds me of the Azumanga Daioh skit

 

[bert_r]

You can enable the 'web service' in the plugin itself (main screen: options -> under miscellaneous 'use Web Service'), or in the configuration dialog (main tab - Network: Submit/retrieve data from plugin web service). I believe it will try to match the filename, and return the anidb data.

Regarding the matching of the file: Most of the anime files (or all of them) contain the CRC32 hash. The AniDB API requires ed2k hashes (+the file size). For details see the AniDB API specification.

lwerndly: I had a look at the hashing algorithm, and I couldn't do much there. It seems as if the slowest part is not the file access on disk (+/- 10%) when reading from a local HDD, but the actual calculation time. I have tried several things, and I could only managed to improve the hashing time with +/- 5%, which is nearly nothing.

 

[lwerndly]

Yes, I too see the naming conventions being a major hurdle for this project. You can't just use the ones in MP-TVSeries, or you'd have to tweak the heck out of them. Also the seasons problem.

With DVDs the thing is lke this. I burn compilations with the full series I download. Having over 70 DVDs, I have yet to make an index to make it easier for searches. Now, with plugins like MP-TVSeries it records the volume name of the DVD, so I can instantly know which disc to look for. Or I could even use the same database from an external program, to search for my stuff or print lists. I like this kind of thing.

Again as bert_r said the AniDB API does not allow the use of the CRC32 hash that is commonly used in anime file names. However enabling the web service may help you a little bit, if you have not renamed the files yourself, or kept the CRC32 in the file name.
The web service tries a match in the following way
1. Files you have previously hashed and submitted yourself
2. Files other people have submitted with the same name and file size.
3. Files other people have submitted with the same CRC32 in the name and the same file size.

Obviously the web service only has a small subset of AniDB though (about 60,000 unique files at the moment)

I share your frustration with the DVD issue.
I have over 400 DVD's now plus large external drives.
I will do something decent with this hopefully.

@bert_r: OK, when i did my testing I always thought that the disk read was the bottleneck. I'll take a look again, as it might be sped up by having multiple threads reading and hashing asynchronously.

 

[FdL]

That's great, lwerndly. I appreciate your work.
It's actually better to have this working apart from MP-TVSeries or any other plugin, because anime collecting has its idiosyncrasies.

Thanks for the good work and good luck on the future.

 

[Ommina_AniDB]

The API requires that you log-off when you have no more queries to make.

Feel free to request any new features or report bugs.

OK. I'm sorry for opening with a quote from a 14 month old message, but this is most assuredly not the case, and the plugin's habit of doing a full login / query / logout cycle for each query is making me cranky. If the client is likely to send another request within ~20 minutes, the client SHOULD remain logged in. In short, as long as you hashing thread is occupied, you shouldn't be logging off. Logoff only after all hashes are done. In the future, clients that exhibit this behaviour will likely be autobanned.

Also, and I want to put this in the strongest possible terms, do not scrape the AniDB HTML pages (and I note that there is still code that does so). If there is some bit of data that you desperately need, by all means, submit a feature request for it. I'm not adverse to adding additional commands for projects such as this, so long as they are feasible (no, sorry, I can't give you "lookup by CRC"). Better I add new commands than clients scraping pages.

Thank you.

 

[lwerndly]

OK. I'm sorry for opening with a quote from a 14 month old message, but this is most assuredly not the case, and the plugin's habit of doing a full login / query / logout cycle for each query is making me cranky. If the client is likely to send another request within ~20 minutes, the client SHOULD remain logged in. In short, as long as you hashing thread is occupied, you shouldn't be logging off. Logoff only after all hashes are done. In the future, clients that exhibit this behaviour will likely be autobanned.

Obviously I didn't read the wiki very well, but I believed it would be a problem to stay logged in without sending further commands. I'll fix this as a priority for version 2, and patch it back into version 1.

Also, and I want to put this in the strongest possible terms, do not scrape the AniDB HTML pages (and I note that there is still code that does so). If there is some bit of data that you desperately need, by all means, submit a feature request for it. I'm not adverse to adding additional commands for projects such as this, so long as they are feasible (no, sorry, I can't give you "lookup by CRC"). Better I add new commands than clients scraping pages.

I have already implemented the ANIMEDESC command for v2, and will patch back into v1 as well.

Apologies for abusing the API, no way I want to upset the great work you guys are doing at AniDB

 

[Ommina_AniDB]

Obviously I didn't read the wiki very well, but I believed it would be a problem to stay logged in without sending further commands. I'll fix this as a priority for version 2, and patch it back into version 1.

<snip>

I have already implemented the ANIMEDESC command for v2, and will patch back into v1 as well.

Apologies for abusing the API, no way I want to upset the great work you guys are doing at AniDB

Thank you very much on both counts. I took the opportunity earlier in the week to ensure the API specifications were more explicit on the matter. While it is certainly the case that remaining logged in consumes some resources, it isn't nearly the resources required to process the logins. All those login/logouts were keeping me awake at night.

Thanks again.

 

[lwerndly]

Quick Fix 0.33

• FIXED - Getting anime description is now done via the API rather than html scraping (much faster)
• FIXED - Will no longer LOGOUT when queue is empty, will wait for 15 minutes of no activity instead
• FIXED - if you have a file in your view which cannot be found on AniDB, it will no longer get stuck in an infinite loop

 

[bert_r]

RELEASE 0.3.4:

• FIXED: Under certain circumstances, an exception could occur while processing new files
• FIXED: When an exception occurs while processing an AniDB command, the command could be executed infinitely
• FIXED: MYLISTADD could be executed multiple times
• FIXED: Anime completed status and unwatched count isn't updated after processing new files (status was only correct after restart program)

I have also noticed that with this version, while processing new files, the episode list could be empty for a few seconds ... I have not yet found the cause, but because it's a small issue, and the fixed are rather important I have decided to release this version immediately.

I strongly advise everybody to upgrade, because the multiple AniDB commands are probably not appreciated by the AniDB team (and who could blame them!). Things might also appear to be faster because fewer AniDB commands are executed.

lwerndly: Could you update the web service and your website with this release?



Unrelated to this release, if somebody has problems locating the MediaPortal logfiles when using Vista, you have to watch out for the 'Virtual Store' feature (this prevents unauthorized access to system directories).
In my case, the logfiles where written to C:\Users\HTPC\AppData\Local\VirtualStore\ProgramData\Team MediaPortal\MediaPortal\log instead of C:\ProgramData\Team MediaPortal\MediaPortal\log

 

[lwerndly]

@lwerndly: Could you update the web service and your website with this release?

done