MPAnywhere - a proof of concept for an MP2 webservice and webinterface (3 Viewers)

[FreakyJ]

one question: why do you use jason at all? Isn't xml much better for this? As far as I know you could just use DOM to parse your elements and get everything like you could do it with HTML code

 

[MJGraf]

Well, good question.
First of all, I want the WebAPI to support both, XML and Json so that every developer who makes use of the WebAPI can choose what he likes more for his specific client app.
The next thing is that we would run into the same problems I have with Json now when we use XML. We need to parse XML and make objects out of it and currently we would have MediaItem objects which are different in the internal structure for every Media Item ==> exactly the problem we have now with Json.
Finally, and this is probably the main reason for this project: MP2 internally (i.e. MP2 Server and MP2 Client) communicate via XML. This means that we cannot change the structure of the XML serialization at all. It is just like it is and we have to deal with it on the client side. For Json we are still free to decide how our Media Items should look like. That makes it a lot easier for the client side development. And since I feel much more comfortable on the server side with C# than on the client side with JS and sencha (be it extjs or sencha touch), I prefer much more adapting the server side in a way that makes it as easy as possible on the client side to make use of our MediaItems.
When I find a bit more time, I'll explain a bit about what I read on sencha's MVC model, which IMO is great and very mature. In the end you could probably use every form of Json or XML objects. But that means that you have to do all the work on the client side. I don't like that (in particular because the client with JS is much slower than the server with C#) and I therefore want to do most of the work on the server side. As a result, we should be able to use the components provided by sencha on the client side (grids, listviews, comboboxes, etc.) with very little additional code on the client side ==> easier for me and better performance.

 

[FreakyJ]

I agree
Also on the server side you have to do the work once on the client side you have to do the work for every client, so I totally agree I hit the spot I missed

 

[Lehmden]

Hi.

How should our webfrontend look like?

For me this Frontend should be as clean as possible. On Chefkochs gfx there is too much information at once imho. Especially if this should be usable on Tablets/Smartphones as well... So I would prefer a Wizard type of UI. As normally no one will change lots of mixed media items at once. First only list available items in Chefs "Main menu" section together with the filter options (in gfx it's 1 and 2). Then, if the user selected one (filtered) menu item, results are listed on the whole page This would be part 3 of gfx but on next page. There a multi select should, could be done with the result displayed on next page. To easily switch between those pages a "breadcrumb" on top (similar to MP2 top menu) could be implemented...
With this wizard syle display the lists (grids) can be dynamically generated based on really existing MIA...
Also I don't know a lot about ASP (I'm used to PHP instead as it's open source) there it is a pretty simple task to make a DB query to get the MIA for this filter. At least it's easy in PHP (a single line of code) but should be no big deal in ASP also...

 

[MJGraf]

Especially if this should be usable on Tablets/Smartphones as well...

To be hones, I don't think this EditMediaItems plugin makes much sense on a tablet. This was the reason why I included the option to have some apps of our frontent only for specific client types. Having grids with maybe 20k or 50k items to edit them on a touch device is probably not really what you want to do. In any case we would need a considerably different UI for such type of function on a touch device than on a computer with mouse and keyboard...

For me this Frontend should be as clean as possible

But anyway I completely agree with this one. Nevertheless, in sencha's frameworks it is no problem to change e.g. area 3 of chef's sketch from being part of the main page to being a separate overlay. Area 3 would just be a panel with some components on it and you can put this panel including all of its components wherever you want: dock it on the right side, make it a "movable window", display it on a separate page or just open it as modal overlay as soon as someone double clicks on a MediaItem in the main page. So this can easily be changed as we like it - as soon as we have the area3-panel as such working...

To easily switch between those pages a "breadcrumb" on top (similar to MP2 top menu) could be implemented...

I don't think there is a breadcrumb component in sencha. But what we can do is just open another tab in the main-tab-panel. Have a look at the sencha docs (just google for "sencha docs"). They are made using sencha extjs and quite similar to what I am trying here...

it is a pretty simple task to make a DB query

And there we hit the problem: we don't do DB queries. We query objects in order to make our frontend independent from the database used. So while MediaItemAspects on the database level are relationships of the MediaItem table to other tables, in object form they are nested objects. This makes it a little bit more complicated, but IMO it is worth the effort because I don't want to have to change our frontend everytime someone changes from SQLCE to SQLite or MySQL as database backend on the server. Furthermore it would be bad design if we just circumvent the provided API (i.e. the MediaLibrary) to access the database directly, because we may decide to change the Medialibrary in the future and completely change the database layout without changing the MediaLibrary's API. If we do so, a frontend using the MediaLirbary's API would still work as expected, while a frontend directly accessing the database would just be broken.
But I'm sure we will sort that out

 

[Lehmden]

Hi.

To be hones, I don't think this EditMediaItems plugin makes much sense on a tablet.

IMHO it makes a lot of sense especially as long as we can't correct DB errors from within the MP2 GUI itself. I'm sitting on my sofa using MP2 on big screen, browsing through my series collection and suddenly I see a cyrillic Series poster (in fact there are lots of them on my collection). Now I grab my phone and correct this without the need to leave MP2 at all...

we don't do DB queries.

Maybe I don't fully understand but the API delivers Data to the WebFrontend, right? Then this is in principle the same functionality as a direct DB query in my understanding. That's what I meant.... The frontend queries Data from the API and from that the List/Grid could be dynamically generated. So the API "simply" delivers all Data, leave the rest to the frontend. No need to access DB directly for that...

Have a look at the sencha docs

I've just started this. It will take a while until I can do something with Sencha but I think it's possible to me...

 

[Oxan]

I will extend the ASPNETWebStackPlugin so that it checks whether the requested port for a server is already used and if so, try the next following ports until it finds an available port. The server will then start on the next free port, which is returned to the caller.

It might be useful to add an option to disable this. (For example MPExtended shouldn't run on a different port without the user explicitely instructing to do so, since (s)he then needs to change the client configuration.)

But all the Json parsers I found so far (including the one in extjs) expect every object in an array of objects to have the same internal structure, i.e. the same properties just with different values.

Curious. While it's certainly unusual and a bit harder to use, JSON parsers shouldn't have a problem with it.

one question: why do you use jason at all? Isn't xml much better for this? As far as I know you could just use DOM to parse your elements and get everything like you could do it with HTML code

Along other reasons, XML easily uses 30% more bandwidth to transfer the same data than JSON.

 

[MJGraf]

I'm sitting on my sofa using MP2 on big screen, browsing through my series collection and suddenly I see a cyrillic Series poster (in fact there are lots of them on my collection). Now I grab my phone and correct this without the need to leave MP2 at all...

ok, convinced - but I have now clue how you want to store a poster locally on your phone to upload to MPAnywhere. But doesn't matter - we'll figure that out...

Maybe I don't fully understand but the API delivers Data to the WebFrontend, right? Then this is in principle the same functionality as a direct DB query in my understanding. That's what I meant.... The frontend queries Data from the API and from that the List/Grid could be dynamically generated. So the API "simply" delivers all Data, leave the rest to the frontend. No need to access DB directly for that...

That's in general correct. The difference is that when you query a db, the result is a table with just rows and columns - exactly the structure you need to display in a grid view, list view, etc. Every row has exactly the same fields (field = intersections between a row and a column). When we directly query our MediaItems here, we get an array (i.e. a list) of MediaItems. One MediaItem is like a row in a table. But you don't have simple columns. The MediaItems are quite complex objects with nested maps, etc. I already simplified the objects a lot before we get the Json structure above. But it seems, we have to simplify further. I still have to think about it, but I will certainly share my thought here...

I've just started this. It will take a while until I can do something with Sencha but I think it's possible to me...

This would really be great! As mentioned, when I find more time, I'll post everything I learned so far about sencha here, which is relevant for this project.

It might be useful to add an option to disable this. (For example MPExtended shouldn't run on a different port without the user explicitely instructing to do so, since (s)he then needs to change the client configuration.)

Noted - will be included. Thanks! I'm just wondering whether it is better to not start a service at all instead of starting it on a different port.

urious. While it's certainly unusual and a bit harder to use, JSON parsers shouldn't have a problem with it.

Sorry, I wasn't precise here. The Json parser of course is perfectly fine with out objects. However, all the components of sencha touch and extjs are based on the MVC pattern. And if you make use of the included model objects, you can easily hook them up to the respective view components such as grids, list views, etc. These model objects (and store objects, proxy-objects, reader-objects) do all the dirty stuff, it looks really easy. The only problem is that they expect "normalized objects", which we don't have, yet. I can easily point the Json-reader to an array of objects in another object, hook the reader to a proxy, the proxy to a store and the store to a grid. Just a few lines of code and it looks as if it took years to develop it - amazing. But only, if the array only contains objects of the same type.
I can also model 1:n relations easily in sencha. But although it looks like "one MediaItem has many MediaItemAspects" is a classical 1:n relation, it is not. Because every MIA has a different structure. Furthermore what we call ID of a MIA is not really the ID of the MIA object. It is the ID of the MIA class.

I'm currently thinking in two directions:

• Maybe it makes sense to directly treat the attributes of the "MediaItemAspect" as attributes of the MediaItem. We can safely assume that every MediaItem has a MediaItemAspect.
• For all the other aspects I'm thinking of an array of objects directly contained in the MediaItem with the following structure:

[{AspectID: "123...321", Name: "Title", Value: "Harry Potter"},
{AspectID: "456...654", Name: "Actors", Value: "Emma Watson|Daniel Radcliff|...}]
That way we split up the different MIAs into their attributes. I know, we get redundancy in particular regarding the AspectID, because if a MIA has 8 attributes, we transfer the MIA-ID 8 times. But maybe it's worth the additional overhead.

That way at least, our MediaItems would always look the same - only difference is the number of attributes in the array of attributes. But this is ok, since that's what arrays are for...

Any comments?
Michael

 

[MJGraf]

Well, thank god I called this "proof of concept" I hardly mentioned that the ASPNETWebStack plugin is more or less feature complete, I just realized that we probably have to completely rewrite this plugin and call it "OwinPlugin". But let me start from the beginning...

One of our forum members contacted me in a convo since he has heard that I am doing some web app development for MP2. He mentioned that maybe SignalR would be a useful framework that would allow for many good features in the future. For those of you who don't know: SignalR is part of asp.net and makes it possible to have a real time communication in web apps. A typical example for using SignalR is a real time chat (for details see here: http://signalr.net/). Well, I actually read about SignalR while I was reading about ASP.Net and also thought that you could build cool features with it (SignalR has e.g. a possibility to send real time messages from the web server to its clients such as browsers. MP2 already has a messaging system and my first thought was that we could have a simple "bridge" to send all those MP2 messages to web clients as well. Imagine a website displaying in real time what is played via the MP2 server without having to click "refresh" - ok, completely useless but absolutely cool Another thing that sounded pretty cool to me is the possibility to invoke JS script code in a client browser from a call on the server-side). I responded to him that using SignalR later is no problem since I already use asp.net and SignalR is part of it. To be on the safe side, I had a quick read on how to integrate SignalR into httpselfhostserver, which I currently use. And there my journey began....

I found out that SignalR currently cannot be used in the httpselfhostserver. You have to use Owin/Katana to self host SignalR. Well, I also read about Owin/Katana before, but to be honest, I never really understood what the benefits of them are and just ignored this. But now I know better...

To cut a (very) long story short: Owin is an abstraction layer between the hosting environment of a webserver and the webserver itself. But owin is only a standard. Katana is an implementation of the Owin standard (made by Microsoft, but open source and freely available). Owin/Katana have two advantages over httpselfhostserver (if not more, which I haven't seen, yet):

• Applications using Owin/Katana are completely decoupled from the hosting environment. I.e., you can start your application using IIS, IIS express, using Owin.exe as host or (most importantly) self hosting the server in your application such as we do with MP2 without changing anything in the implementation. This reminded me of your idea, Oxan, to self host WebMP instead of using IIS express. If you could make WebMP use Owin as an intermediate layer, it would still run with IIS express, but you could self host it afterward without any changes in your application code.
• Owin/Katana are designed to combine an endless number of web technologies in one host such as a WebApi, a web server for static files, SignalR and many others. This is basically what I tried to do "manually" in my code now and I think it is a much better idea to make use of a standard instead of re-inventing the wheel.

It is hard to explain all the advantages of Owin/Katana here, but for those of you interested in this topic, I can recommend the following very interesting blogs, where you can also read more on the technologies, which can be integrated into an Owin/Katana server.
http://www.asp.net/aspnet/overview/owin-and-katana/an-overview-of-project-katana
http://chimera.labs.oreilly.com/books/1234000001708/ch09.html
http://weblogs.asp.net/pglavich/archive/2013/04/05/owin-katana-and-getting-started.aspx

Well, what shall I say. I think I have to rewrite the ASPNETWebStack plugin - but hey, that was the purpose of this thread: share ideas and make sure that we make the right decisions in the beginning. So if someone of you has any experience with Owin/Katana, please share your thoughts!

And as a final note: This conversation showed me that chefkoch was absolutely right and at least now it is time to make this thread read-only public. We need the community and their thoughts as well and maybe we can even recruit some more interested developers. I'm currently quite busy with real life work, but I'll try to find the time to (a) amend the first post a bit to make it ready for publication and (b) at least include some documentation and our copyright headers in the source so that we can have it in a public thread.

Cheers,
Michael

 

[morpheus_xx]

@MJGraf: I'm really amazed how many new technologies I learn about when reading your post! So many things exists that I didn't notice before. So keep up your investigations, I really appreciate your work!