MPAnywhere - a proof of concept for an MP2 webservice and webinterface (1 Viewer)

Status
Not open for further replies.

MJGraf

Retired Team Member
  • Premium Supporter
  • January 13, 2006
    2,478
    1,385
    Thanks Morph! It's basically the same for me. Sometimes I feel that I have missed at least 20 years of progress in web technology, but then I realize that e.g. the Katana project wasn't started too long ago. I think that this is mainly because MS finally realized two things:
    • To keep track with new web technologies in time MS itself is too big and too slow. That seems to be the reason why they e.g. decoupled the development of WebAPI from the development of .net. The release cycle of .net is just too slow and providing NuGet packages instead makes it possible to deliver updates with new features much quicker.
    • The open source community has so many ideas that MS can take advantage of it. That seems to be the reason why they opensourced the asp.net webstack and just make sure that the code quality is high enough by reserving write access to the repo to MS employees while letting the community make pull request for improvements and new features.
    These two developments seem to have really boosted the development of new web technologies, which is really great. I wonder sometimes why we still have "regular websites" instead of every website opening a fully fledged application in your browser :D But maybe the technologies are still too new... And what I also realized is that our MP(2) documentation is better organized than the MS documentation in that respect. To really get an overview on all the possible features of a technology, you very often have to read the code due to a lack of documentation :p

    As a reminder to myself - Here we have an example how to implement my "virtual file system" (i.e. extendable by MP2 plugins and taking care of different client types) directly in Owin/Katana:
    http://www.choudeshell.com/blog/extending-katana-for-fun-and-profit/
    https://gist.github.com/choudeshell/5174495
    This btw can also be used to expose our MP2 Resource Provider system to a http server and later on replace our existing http server. I haven't had a look into serving UPnP with it, but I'm sure it's not too hard to do...
    They really implemented a lot of possibilities in Owin/Katana already and it is really fun to see how easy it is to use and extend it. If you need an additional technology, it's just a single line of code to add it to your web app. You need a simple static file web server? app.UseStaticFiles() - that's it. You need authentication support via a login form? app.UseFormsAuthentication() - same goes for compression and even complete frameworks like ASP.Net MVC, ASP.Net WebAPI, SignalR etc. If you need it, just one line of code to add it.
    The only downside is that it takes a lot of reading to understand the possibilities and as mentioned, the documentation is not always good.

    But it's fun, so I will definitely continue :)
     

    Spragleknas

    Moderator
  • Team MediaPortal
  • December 21, 2005
    9,474
    1,822
    Located
    Home Country
    Norway Norway
    [Part 1 - the tech stuff]
    I like it!!!! I don't understand anythinkg, but I still like it!!!! :D
    [And as a final note: riggnix' conversation showed me that chefkoch was absolutely right and at least now it is time to make this thread read-only public. We need the community and their thoughts as well and maybe we can even recruit some more interested developers. I'm currently quite busy with real life work, but I'll try to find the time to (a) amend the first post a bit to make it ready for publication and (b) at least include some documentation and our copyright headers in the source so that we can have it in a public thread. To those of you, who made "promises" they do not wish to be public to avoid nagging in terms of "can you tell me when it's ready", please feel free to edit your posts accordingly. I will certainly not make this thread public before the next Sunday - probably even later, so there is still some time...]

    Cheers,
    Michael
    I'm, willing to help with whatever forum-related stuff you'd need me for. No guarantee regarding what workload I can carry, but the will(y :D) is big!

    When you'r set to "set free" the information,, I'll help out with FB posting - even G+ if I bother to get access. That being said, we do have a PR-group.
     

    MJGraf

    Retired Team Member
  • Premium Supporter
  • January 13, 2006
    2,478
    1,385
    Thanks a lot, Spragleknas, for your offer. I will definitely get back to you, once we are ready to publish. But it takes longer than I thought (did I mention that this is a long term project :D ).
    First of all, I wanted to begin with making the code a bit prettier and adding some comments to the code, but then I realized that it doesn't really make sense to add tons of comments to ASPNETWebStack just to publish it and afterwards just deleting this plugin and replacing it with an Owin/Katana-Plugin. So I decided to dig a bit deeper into Owin/Katana before going public.

    The most important thing first: Right after this post, I'll update the first post with a new version of sources and binaries. ASPNETWebStackPlugin and its httpselfhostserver is dead (and removed) and we are hosting everything with Owin/Katana now. The new plugin is called MPOwinPlugin. But the way there was quite cumbersome...

    All the tutorials and examples I found in the internet state that you just have to add a nuget package called "Microsoft.ASPNet.WebApi.OwinSelfhost" - current version is 5.0.0-beta2. I tried to do so, but the result was
    Code:
    Attempting to resolve dependency 'Microsoft.AspNet.WebApi.Core (≥ 5.0.0-beta2)'.
    Attempting to resolve dependency 'Microsoft.AspNet.WebApi.Client (≥ 5.0.0-beta2)'.
    Attempting to resolve dependency 'Microsoft.Owin (≥ 1.1.0-beta1)'.
    Attempting to resolve dependency 'Owin (≥ 1.0)'.
    Successfully installed 'Microsoft.AspNet.WebApi.Client 5.0.0-beta2'.
    Successfully installed 'Microsoft.AspNet.WebApi.Core 5.0.0-beta2'.
    Successfully installed 'Microsoft.AspNet.WebApi.Owin 5.0.0-beta2'.
    Successfully uninstalled 'Microsoft.AspNet.WebApi.Client 5.0.0-beta2'.
    Install failed. Rolling back...
    Could not install package 'Microsoft.AspNet.WebApi.Client 5.0.0-beta2'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.0', but the package does not contain any assembly references that are compatible with that framework. For more information, contact the package author.
    Apparently, this package only works with .Net 4.5 - and MP2 still uses .Net 4.0.
    After a lot of reading I found this, which basically says that it "should" still work with .Net 4.0, since we are not using OwinHost.exe (we are self-hosting in our MP2Server service process). But as it seems, this isnt't quite true (or at least there is something wrong with the dependencies in the nuget-package...)
    After even more reading, I found out that Microsoft.ASPNet.WebApi.OwinSelfhost mainly consists of three other packages:
    • Microsoft.Owin.Hosting,
    • Microsoft.Owin.Host.HttpListener and
    • Microsoft.AspNet.WebApi.Owin
    So I tried to install these three packages instead of Microsoft.ASPNet.WebApi.OwinSelfhost. Microsoft.Owin.Hosting (version 1.1.0-beta2) worked as well as Microsoft.Owin.Host.HttpListener (version 1.1.0-beta2). But I got the same error as above for Microsoft.AspNet.WebApi.Owin. However, while there were no older versions available for Microsoft.ASPNet.WebApi.OwinSelfhost (besides 5.0.0-beta1, which gave me the same error), there was an older version of Microsoft.AspNet.WebApi.Owin: version 0.21.0-pre. To install this version, I had to enter the following at the nuget console:
    Code:
    Install-Package -Id Microsoft.AspNet.WebApi.Owin -ProjectName MPOwinService -IncludePrerelease -version 0.21.0-pre

    I read some further blogs, news etc. on .Net4.5 and Owin/Katana and found out that Microsoft is really pushing towards .Net 4.5. Although I haven't implemented it yet, we can now e.g. use SignalR, because we use Owin/Katana for the hosting. The thing is: we have to take an old version of SignalR (1.2), because the current version (2.0) requires .Net 4.5 as well. Same goes for some other technologies. I don't know why MS wants to move to .Net 4.5 so quickly, but I suppose one of the reasons is the nice async/await-pattern, which is used a lot in many examples. I have to admit that I like it too - because it makes multithreading at least look easy.
    The remaining question is: Morph, Chef, OffByOne I know someone of you was already talking about moving MP2 to .Net 4.5. Are there still plans? Since I got this working for now, it is not time critical for this project. But sooner or later there will be a lot of new technologies in the webapp area, that we can only use when we are on .Net 4.5.

    Ok, now I had the necessary packages. But how would IASPNETWebStackService look like for Owin? Well, not so much different than before:
    Code:
      public interface IOwinService
      {
    bool RegisterServer(int port, Action<IAppBuilder> startup);
    bool UnregisterServer(int port);
    void Shutdown();
      }

    The only difference is that Owin does not use something like a httpselfhostconfiguration object, which is passed to the httpselfhostserver to start it. Owin heavily works with delegates. And our delegate here is Action<IAppBuilder> startup. The OwinService takes this delegate and passes it to the "WebApp.Start" method of Owin, which in turn starts our webserver. While starting the webserver, Owin makes a callback to our delegate and passes an IAppBuilder instance, which can then be filled by our delegate method.

    Just as a sidenote:
    The WebApp.Start method of Owin is heavily overloaded and you have a lot of choices what parameters you want to pass when you start your webserver. In all the examples I have found so far, they use this overload: WebApplication.Start<Startup>(). This somehow looks strange, because they use a template to pass a type - in this case "Startup". Startup is a class, which you have to define e.g. as follows:
    Code:
    public class Startup
    {
    public void Configuration(IAppBuilder app)
    {
    var config = new HttpConfiguration();
    config.Routes.MapHttpRoute("default", "{controller}");
    app.UseWebApi(config);
    }
    }
    The downside of this WebApp.Start overload is in my opinion, that in our MPOwinService we only have a classtype - not an object. The object is then apparently instatiated somewhere inside Owin/Katana. But this means that our MPOwinService cannot access any kind of configuration object. Why would we need to do so? Well, maybe we want MPOwinService to add a trace logger to every webserver that is started. And now we don't have access to the IAppBuilder object in MPOwinService, which means, we cannot change anything. We only have the type of our Startup class - but we cannot change the type...
    That's why I used the Action<IAppBuilder> version of WebApp.Start. By doing so, our MPOwinService gets the Action<IAppBuilder> delegate. And what we can do now (not implemented, yet, but I thought it would be a good idea for future enhancements) is to make this delegate a multicast delegate. I.e., MPOwinService can add another delegate with += and Owin will automatically call both delegates: The one, which was passed by the calling plugin (in this case MPAnywherePlugin) and the one, which was added by MPOwinService.
    Maybe I have not enough C# knowledge and what I want to do is also possible with WebApplication.Start<Startup>() - but I just don't know how :p

    Another thing to mention is that in MPOwinService, I had to add
    Code:
    options.ServerFactory = "Microsoft.Owin.Host.HttpListener";
    This shouldn't be the case, since Owin should find this assembly on its own. But it didn't - I always got an exception. This thread, however, suggested to do as described and voilà, it worked.
    I don't know the reason for this, but maybe this is caused by not using the recommended Microsoft.ASPNet.WebApi.OwinSelfhost nuget package. As described above, I added the Microsoft.Owin.Host.HttpListener nuget package manually and I suppose that this could be the reason, why we need this line.
    [Edit: seems to have been a bug in Owin/Katana. By updating the references I could remove this line]

    Finally, the MPAnywherePlugin was amended to make use of the new MPOwinService. For the WebApi part, this was easy and it will remain how it is now.
    For the Web File Server (which serves our html, js and css files) I also just moved the old server into the new Owin host. But this is now more or less a workaround, since we are (ab)using a WebApi server to server static files. Thanks to owin, we don't need this anymore: There is a Microsoft.Owin.StaticFiles nuget package, which gives us a fully fledged static files webserver. I only didn't have the time, yet, to implement this in a way that is takes into account my idea of ClientTypes and the possibility for other plugins to register additional (virtual) root paths. But this will follow - for now I just ported the old solution to the Owin host, because that worked with just a few lines of code.

    After I had all this done, everything worked. There was only one strange thing: It took for ages to open the MPAnywhere website. The reason was easy to find: The extjs main file has more than 6MB, which have to be served by our web file server to the client - and this was dead slow. There was no change in this part of the code - as mentioned above. It was just the switch from httpselfhostserver to Owin/Katana. And it took quite some time to solve the problem. I still don't know why that was, but at least I know now how to solve it: Our VirtualFileHandler assings a StreamContent object to the content of the HTTPResponseMessage. The StreamContent has a bufferSize property, which defaults to 4096 bytes. By setting this property to 8192 bytes instead, loading the MPAnywhere website takes less than 2 seconds, instead of more than 2 minutes. Do not ask me why - it works now and since I want to replace the virtual file handler anyway with the static file server of Owin, I will not dig deeper into this. Let's just call it a "feature" of the StreamContent class... ;)

    Well, that's it so far. But what has changed now from the user perspective? That is the downside of taking much time for finding the right technology: nothing has changed from that perspective.
    But internally the whole thing works completely different now. First of all, we can now use whatever host we consider appropriate: selfhosting (as we do now), IIS and IIS express and I even read about the possibility to use linux hosts with mono (doesn't make sense, yet, since the rest of MP2 is not linux ready - nevertheless, everyone who wants to port MP2Server to linux will not have a problem with MPAnywhere...). And secondly - and most importantly - we can now also host a lot of other web technologies in our MPOwinService: We can use compression, authetication, SignalR and all the other goodies that you can put into an Owin container.
    But again, many of these technologies will only be available with .Net 4.5 Those of you, who are interested, can read a bit here. From this it seems clear to me that MS will - with VS 2013 - make many web technologies only available to .Net 4.5. And when you read carefully, you will notice that Owin/Katana will be an integral part of VS 2013. When you start a selfhosted WebApi project in VS 2013, it will automatically be hosted by Owin/Katana - not httpselhostserver anymore. So Owin/Katana is definitely the right way to go, as MS sees it as standard in VS 2013.

    And now that we have this: Back to square one. I'll beautify the code, add comments, etc. and we will first make this thread readonly public before I add new functionality...

    cheers,
    Michael
     
    Last edited:

    Spragleknas

    Moderator
  • Team MediaPortal
  • December 21, 2005
    9,474
    1,822
    Located
    Home Country
    Norway Norway
    Remember: It doesn't need to be finished to be newsworthy!
    We want attention/ideas/involvement more than we need user. Only great thing about users it that they bring attention/ideas/involvement :)
     

    Oxan

    Retired Team Member
  • Premium Supporter
  • August 29, 2009
    1,730
    1,124
    Home Country
    Netherlands Netherlands
    Applications using Owin/Katana are completely decoupled from the hosting environment. I.e., you can start your application using IIS, IIS express, using Owin.exe as host or (most importantly) self hosting the server in your application such as we do with MP2 without changing anything in the implementation. This reminded me of your idea, Oxan, to self host WebMP instead of using IIS express. If you could make WebMP use Owin as an intermediate layer, it would still run with IIS express, but you could self host it afterward without any changes in your application code.
    Unfortunately Owin/Katana doesn't support ASP.NET MVC yet, and since MVC is quite deeply tied into System.Web I doubt that will happen soon.
     

    morpheus_xx

    Retired Team Member
  • Team MediaPortal
  • March 24, 2007
    12,073
    7,459
    Home Country
    Germany Germany

    MJGraf

    Retired Team Member
  • Premium Supporter
  • January 13, 2006
    2,478
    1,385
    Unfortunately Owin/Katana doesn't support ASP.NET MVC yet, and since MVC is quite deeply tied into System.Web I doubt that will happen soon.
    Yep, Oxan, it seems like at the moment you are still stuck with IIS (express) if you have to use ASP.Net MVC. But I wouldn't be too pessimistic. They have already achieved that you can use Owin/Katana as "middleware" for your ASP.Net MVC application - but you are then (still) bound to IIS as host. Nevertheless by doing so, you can use the Owin/Katana inbuilt authentication and compression mechanisms for your ASP.Net MVC application. I don't know whether this makes things easier for you, since you mentioned that authentication is not so easy taking into account the many devices and their respective OSses and browsers. But I guess the main culprit there is also moving to .Net 4.5 (from one of the posts I mentioned above you can see that all the authentication mechanisms in Owin/Katana such as Google, Facebook, Twitter, etc. only work with .Net 4.5 in their latest versions).
    It is already possible to use NancyFX as a replacement for ASP.Net MVC and Nancy can be hostet in Owin/Katana using any hosting method (IIS, IISe, self-host, etc.) but I don't know how much work it is to switch to NancyFX - never used it before...

    Morpheus, I completely agree. Latest gossip about VS 2013 release date seems to be this. That means chances are good that they will release it together with Win 8.1 in October. Not too long anymore...
     

    MJGraf

    Retired Team Member
  • Premium Supporter
  • January 13, 2006
    2,478
    1,385
    Finally added v0.04 to the first post. For changes see second post ;)

    Most importantly the code is finally in a condition to be published. I would propose to wait for Alpha-3 release and then make this thread public more or less at the same time. Maybe we can add a short link at the end of the Alpha-3 news for the brave among the MP2 users :p

    There's only a single functional change in v0.04: For testing purposes I have added Windows Authentication to the webinterface (Thanks to Owin only 2 lines of code in MPAnywhereService). If you try to access it you are prompted to enter username and password. You can enter any windows username / password combination which is valid on your MP2 Server computer.

    Besides that, I have read A LOT about Owin/Katana, WebAPI and SignalR. To really get the beauty of Owin we need to move to .NET 4.5 (or 4.5.1). But I already have a lot of ideas so stay tuned ;)

    Sorry that this thread is cooling down a bit - A lot to do in real live. But as I told you: I won't give up and will develop this further. It may just take a little while ;)

    Michael
     

    MJGraf

    Retired Team Member
  • Premium Supporter
  • January 13, 2006
    2,478
    1,385
    Just edited the first post to make it ready for publication.

    @Spragleknas: As soon as MP2 Alpha 3 is published, please feel free to move this thread to the MediaPortal 2 / Plugin Development Forum and make it read-only public. Thanks in advance for helping!

    As soon as it is public, I will open a new "MPAnywhere Public Discussion" thread to keep this one clean.

    If anyone feels like the text in the first post could be improved, please feel free to do so!

    Michael
     

    Spragleknas

    Moderator
  • Team MediaPortal
  • December 21, 2005
    9,474
    1,822
    Located
    Home Country
    Norway Norway
    clear.png
    OK.
    No need to QA before moving?
     
    Status
    Not open for further replies.

    Users who are viewing this thread

    Top Bottom