Extension Management powered by NuGet engine (1 Viewer)

chefkoch

Retired Team Member
  • Premium Supporter
  • October 5, 2004
    3,129
    1,634
    Dresden / Munich / Maastricht
    Home Country
    Germany Germany
    Warning: This thread targets developers more than users. Keep this in mind when reading and maybe replying.

    There have been several concepts for managing plugins and skin or let's say extensions during the last years and again the last months.

    While the first trials to implement an extension installer (MPEI) have been done with spending a lot of efforts and are even better than what there was before (nothing or zips, or exe installer, ...), it still has a lot of architectural design issues, which results problems maintaining packages for the plugin authors and even for our admins managing the list of available plugins etc.

    Many of the concepts' that have been raised since first implementation of MPEI, mostly focused on the graphical user interface first, then the usability and last but not least "some" versioning and dependency concepts.
    There are several major architectural drawbacks imo:
    List is incomplete, went to bed before I finished it ;)
    • Definition of a package (creating it) is too complicated: There is no need for a GUI to create plugin / extension package if the xml syntax is well documented and extremely powerful.
    • The concepts were always focused on MP1's architecture with different kind of extensions' like
      • MP1 vs. TVServer
      • For MP: Windows, process, .....
      • and skin of course
    • These kind of plugins do no exist anymore within MP2. Here everything is a plugin no matter if it is code extension (classic plugin) or a skin
    • The most "important" within these concepts mostly was the GUI to browse and manage plugins for the end user.

    In my opinion the priority of things to focus is the other way around. It's more the way MP2 has been built during the last years.
    • First build a powerful, flexible and easy to maintain architecture. This can be ugly from the user pov ;)
    • Then later you can build the fancy parts around.

    The core
    The most important for such a extension management is the it's core, which will be reused in several other components.
    The core component knows how to pack (build/compile), install, uninstall, update, reinstall an extension.
    It's code that might might be available within a dll and so can be reused or even better a powerful command line application that is being reused by the other components.

    The "fancy" components
    The following list just gives examples:
    • Online Repository: The online repository / gallery also is built around the core. There is no need to enter all the plugin information again, no need to list the dependencies again or manage various download locations for different versions. All these things are managed automatically by the online repository and metadata is read from the uploaded extension packages.
    • GUI plugin: A plugin for MP2-Client to browse installed plugins on the current client, the server and any other client within the network, which is attached to the same server.
    • Managing you client's and server's extensions from within the browser? Imagine we could have a webinterface for configuring MediaPortal 2 through the browser. Why not also installing and updating plugins from there. For some talk and discussion about this see https://forum.team-mediaportal.com/...ure-mp2-browse-edit-play-medialibrary.119978/
    • A separated extension installer application: see https://forum.team-mediaportal.com/forums/general.656/

    Why am I posting this now? here?
    Why yet another thread about it?
    Why aren't there any requirements or specifications?
    Simple answer: I won't be to work on this, because of the lack of spare time (busy with real life) within the next weeks/months, but I know this is a very important component for MP2.

    If done correctly it can already improve the plugin installation experience for MP2 a lot. There is not need to have a fancy app for browsing extensions in GUI. It would already be great having
    • an online repository just as a simple listing of available plugins
    • an local application listing installed plugins and showing available updates
    Everything else improving on it's "own" as soon as more and more other people realize how amazing it is.

    So what it is all about: The NuGet-approach of managing extensions, versions and dependencies.
    During the last weeks I worked a lot with NuGet. I removed almost all binaries from the MP2 source code (https://issues.team-mediaportal.com/browse/MP2-292) and integrated them via packages from the official NuGet feed or via our own packages from our own nuget feed.

    Maybe this is a great architecture that can be used for managing extensions in MediaPortal 2 as well. I don't know if it is the solution, but I imagine it could be a major part of it. That's why I am writing this post. Spreading the word about it and what others already do with nuget in their own projects / applications.

    So if you think this an interesting topic or you know alternative solutions, please share your thoughts.
    If you might be interested in working on such a task definitely get in contact with us.


    Further information about NuGet:
    If you think the NuGet gallery sucks (no screenshots, comments, ratings, etc..), stop. Didn't I say it's not about the fancy stuff yet? So with NuGet you got known a bit to the the versioning and dependency management.
    There is also another project:
    The NuGetGallery is similar to the orchardgallery, but kept much simpler since it targets developers only.
    We would like to target end users again, so maybe it is possible to reuse many things from the OrchardGallery and merge it into our own custom nugetgallery (both are based on the GalleryServer http://galleryserver.codeplex.com/):

    NuGet + OrchardGallery + MediaPortal extensions = The ultimate extension experience


    Others who are already (will be) reusing Nuget for their own applications:

    Like I already said I really would like to read and work myself deeper into that topic and evaluate the possibilities, but that's won't be possible because real life these days. Nevertheless I will keep an eye on other NuGet developments. forks and projects that are going to use it and will post them here. :)

    Please excuse any mistakes and parts that make no sense. It's almost sunrise here ;) Feel free to edit my post :p
     
    Last edited:

    Valk

    Portal Pro
    February 25, 2006
    302
    108
    Home Country
    Australia Australia
    If people have plugins via nuget we could probably automate the rebuild on a major release. I suspect by the time MP2 hits 1.0 the core will be stable enough that 90% of the time that'll be all that's required (if it's even required) to get plugins working with newer builds.

    Really liking this concept and can see it evolving (with support from all the community).
     

    chefkoch

    Retired Team Member
  • Premium Supporter
  • October 5, 2004
    3,129
    1,634
    Dresden / Munich / Maastricht
    Home Country
    Germany Germany

    chefkoch

    Retired Team Member
  • Premium Supporter
  • October 5, 2004
    3,129
    1,634
    Dresden / Munich / Maastricht
    Home Country
    Germany Germany
    Here are some information how JetBrains designed their package structure for ReSharper plugins: http://confluence.jetbrains.com/display/NETCOM/1.9 Packaging (R8)

    Package Structure

    In order to locate plug-ins, ReSharper searches a number of predefined locations inside the package. These locations are searched recursively, so you can create nested folders for keeping your plug-ins. Several plugins for different versions of ReSharper can be deployed in one package.
    In terms of versioning, it’s possible to constrain the deployment to a specific version of ReSharper (the recommended option), or to specific versions of both ReSharper and Visual Studio (for example, if you need to take advantage of specific Visual Studio interfaces):
    The general package structure should be
    \{Product}\{Version}\[VsVersion]\["plugins"|"settings"|"annotations"]
    Where:
    • {Product} is the name of a product, e.g. "ReSharper", "dotTrace", "dotCover" or "dotPeek"
    • {Version} is the version of the target product in form of "v8.0", "v.8.1.1", "v8.2.3" or "vAny". This allows the package to contain mulitple files that are specific to different versions of a product. For example, a package can contain the plugin files for ReSharper 8.0 and 8.1. The special value "vAny" means the files are applicable to all versions of the product.
    • [VsVersion] is an optional folder to restrict your extension to a specific version of Visual Studio. This is in the form "vsX.X", e.g. "vs10.0", "vs11.0", etc. If your plugin only works on a specific version of Visual Studio (for example, enables ReSharper to use Visual Studio 2012's preview tab), then it should be packaged in the appropriate Visual Studio specific folder (VS2012 == vs11.0)
    • ["plugins"] is an optional subfolder where the extension manager will look to load any plugin dlls. This folder is not required if there are no plugins
    • ["settings"] is an optional subfolder for settings to deploy in the form of a standard ReSharper .DotSettings format. These settings should not be default values for a plugin's settings - useIHaveDefaultSettingsStream for this. The settings can contain Live Templates or Structural Search and Replace patterns, or override other, existing settings. The package can contain only settings, and does not require a plugin.
    • ["annotations"] is an optional subfolder for external annotations that you may wish to deploy. Once again, there's no requirement for these annotations to be related to the plug-in, and the package can contain only the external annotations, without any binary files. Note that annotations files must have a ".xml" file extension, and must be named after the assembly they're for, or live in a directory with that assembly's name. E.g. annotations\myAssembly.xml andannotations\myAssembly\annotations.xml are both valid.annotations\myAssembly.ExternalAnnotations.xml is not.
    Package must have dependency on virtual package corresponding necessary product to appear in Extentsion Manager.
    For example, package for ReSharper 8 must have dependency on package ReSharper, version 8.0 and higher.
     
    Last edited:

    chefkoch

    Retired Team Member
  • Premium Supporter
  • October 5, 2004
    3,129
    1,634
    Dresden / Munich / Maastricht
    Home Country
    Germany Germany
    Okay, I just made a few things within 20 minutes. Keep in mind that I did not write one single line of code to achieve the following. To make actually use of this and integrate it into MP2 some developments would be required, which is the reason I started this thread. I don't have enough development knowledge to write such a "complex" / important component. (some dev-related ideas are listed at the end of the post)

    So what have I done?
    I took 2 different "extensions"
    • CECRemote plugin, which has been released recently by -Manfred-
    • and the MP2-ClientLauncher, which is not a plugin, but more a tool to start MP2 by clicking the green button on the MCE remote
    For both plugins I created a nuspec file containing some metadata:
    20130728_003501.png20130728_003509.png

    And placed the related file next to the package exactly like the should be located in the installation directory. Keeping in mind that the installation dir and also the plugins directory are customizable, the structure in the package is more a definition of placeholders and the package installation code should of course extract the files to the correct folders in case these are modified.
    20130728_003337.png20130728_003349.png 20130728_003358.png


    When opening the nuspec file now with the nuget package explorer, the metadata is read as well as the files from the file system next to the nuspec file. So there is no manually, complicated definition of files and their target folders required. Just place the files correctly and prepare the filesystem. But of course you also can explicitly define the files within the nuget specification. See nuget documentation for more details about explicitly including, excluding and defining the target folder/filename.
    20130728_003451.png20130728_003527.png

    Pressing "Publish" builds the package (packaging) and uploads it to the online repository: The nuget feed.
    20130728_011147.png


    Nuget feed ?? Which nuget feed?
    Yeah, sorry before publishing it I create a feed at myget.org which took me a few seconds:
    https://www.myget.org/feed/Packages/mp-plugins
    https://www.myget.org/feed/mp-plugins/package/CECRemote
    https://www.myget.org/feed/mp-plugins/package/MP2-ClientLauncher
    https://www.myget.org/F/mp-plugins/api/v2
    https://www.myget.org/F/mp-plugins/
    20130728_012306.png

    I've attached the content of the nuget packages, the nuspec and the packages itself to this post. If you are interested into the package structure/content you can rename it to *.zip and extract it. ;)

    In case you are interested in this topic, task and technology feel free to get in contact with us :)



    The first task would be make use of the NuGet.Core (http://www.nuget.org/packages/Nuget.Core | http://channel9.msdn.com/coding4fun/blog/Think-NuGet-is-only-for-Visual-Studio-or-IDE-Think-again, highly recommended) and be able to read the nuget feed, the already installed plugins and being able to to extract the content where it belongs to.

    Having a rock stable core which is able to fulfill these tasks already would be great, even if it is launched manually, asks for uac and needs to be controlled by mouse.
    If this core is designed properly, it should be able to reuse it within the following components without the need to throw any core code away or change it again. (means even starting with a simple non-fashion windows application is no waste of resources):
    • have a windows service to be able to unpack the packages and extract the files into the MP2 installation directory. The windows service would be required to be able to install / update plugins without approving the UAC request.
    • a GUI plugin for MP2-Client to
      • consume the nuget feed, display the content nicely to the user and allow installing, updating and uninstalling through the remote
      • communicate with the windows service running on the client's machine to forward these tasks to be actually executed
      • communicate with the windows service running on the server's machins (or the server itself) to do the same with
    maybe other things.....
    and maybe these
    • For decreasing the redundant information about the plugin metadata, version, dependencies etc, it might be useful to combine the nuget specification (*.nuspec) and MediaPortal 2's plugin descriptor (plugin.xml). This would improve the usability for developers, but depending on the changes it might required changes to MediaPortal 2 plugin management and/or the NuGetPackage Explorer.
      Maybe this also it not required, since duplicating these few infos is not a big deal, but it also needs to be possible to detect the plugins that are actually already installed (plugin.xml) and matched to the available nuget packages (nuget feed > nuspec metadata)
    • Creating a own gallery based on the NuGetGallery ( http://www.nuget.org/packages | https://github.com/NuGet/NuGetGallery ) similar to the ReSharperGallery ( https://resharper-plugins.jetbrains.com/packages | https://github.com/JetBrains/ReSharperGallery ). If additional features might be required, like rating commenting, screenshots, maybe also extending this forked "MediaPortalGallery" would be possible by adding components from OrchardGallery ( http://gallery.orchardproject.net/ )
     

    Attachments

    • Packages.7z
      2 MB
    Last edited:

    BigGranu

    Retired Team Member
  • Premium Supporter
  • February 7, 2013
    240
    202
    53
    Home Country
    Germany Germany
    Today, I read all the Informations :eek: and make any Tests.
    Create a Package, Publish it und host it on MyGet.
    Then I create a little Tool and read the Packages and Informations from the MyGetfeed.
    This all OK.
    But, I cant find Informations of Sourcepath in the Package.
    So I do not know where to put it. :(
     

    chefkoch

    Retired Team Member
  • Premium Supporter
  • October 5, 2004
    3,129
    1,634
    Dresden / Munich / Maastricht
    Home Country
    Germany Germany
    Sorry, but what are you trying to achieve?

    Are you trying to setup your repository / project to use nuget packges, i.e. the Bass.NET dll?
    This I already have prepared in my own fork of your plugin, but I am planning to improve the plugin development a bit more. Especially the references to MP2 core projects/ binaries like MediaPortal.Common , MediaPortal.Utilities etc...
    I will also make sure that it my solution can be use for various plugin repos like Webradio and OnlineVideos, so it also would be easier to build those plugins by continuous integration tools.
    Nevertheless, this is not related to this thread.

    In case you are trying to pack your plugin into a nuget package, what this thread is about, I would say, don't do it and simply release it as a zip file.
    Why?
    Because we don't have any extension management in MP2 yet. NuGet can be the container for the extensions, but there needs to be a extension management developed first. I.e.
    • a service which is able to apply changes to the ProgramFiles directory without the need for a uac window
    • a specification (directory structure) how files are organized inside a nuget / mp package
    • a gui (most likely an in-GUI plugin) which is able to
      • collect infos of installed extensions and available extension
      • receive user input
      • download packages and
      • advice the service (point 1) to apply it
    There are things that needs to be thought about like
    • an extension cache (to revert to previous installed versions),
    • how the service/gui handles
      • server and client specific plugins,
      • locally and on remote machines in the MP2 network

    Unless you are targetting to think about (some of) these points and read even more :p and work and try out concepts etc...., I suggest not to pack your plugin into nuget packages, yet.
     

    Users who are viewing this thread

    Top Bottom