Comedy network (1 Viewer)

[offbyone]

I guess on the first connection, the app should only be "ondemand". For faster figuring of theses values you can use rtmpdump to find the correct parameters.

 

[corporate_gadfly]

Re: Support for a site request thread

That rtmpe link looks good, what does MP log say when sending it to be played? It might be that the playpath and app need special care and can't be easily guessed.

Edit: Moved all related posts here

More progress. For now, I have given up on Comedy Network and concentrating on CTV.

I'm able to stream URLs of type:

rtmpe://cp45924.edgefcs.net/ondemand/s_!ctv/shows/2011/08/22/CWD-308-EP-CLIP01.mp4?auth=dbEaWc2cOdkdacvdxcKbQbvawamdqcDaLaz-bow9ix-eS-iYG-wwI3noDDn&aifp=v001&slist=/s_!ctv/shows/2011/08/22/

by transforming them within my CTVUtil to something like:

rtmpurl=rtmp://cp45924.edgefcs.net/ondemand?auth=dbEaWc2cOdkdacvdxcKbQbvawamdqcDaLaz-bow9ix-eS-iYG-wwI3noDDn&aifp=v001&slist=/s_!ctv/shows/2011/08/22/
playpath=mp4:s_!ctv/shows/2011/08/22/CWD-308-EP-CLIP01.mp4
swfVfy=http://watch.ctv.ca/Flash/player.swf?themeURL=http://watch.ctv.ca/themes/CTV/player/theme.aspx

Basically, the following parameters are supplied to rtmplib:

• take the filename out (in magenta) make it the playpath parameter
• add swfVfy

For future coders, I had to provide overridden implementations of getMultipleVideoUrls and getPlaylistItemUrl.

I will test CTV on my home rig this evening and then post the code. rtmpdump on the Mac OS X command-line produced an output.mp4 without any problems. My bare-bones VMWare MePo install for some reason has never streamed anything successfully using OnlineVideos (because of what? lack of SAF codecs? DirectX or something else). Don't worry about the VMWare part. All it means is that my testing will get delayed till I get home.

For now, I have given up on Comedy Network and concentrating on CTV.

Any one knowledgeable in the mood to debug output of rtmpdump for comedy network?

I do see a Connection succeeded, but then it fails at the end with Failed to play.

Following command-line was used on Mac OS X:

rtmpdump -z -r 'rtmpe://ctvcomedy.fcod.llnwd.net/a4945/d1/2011/08/12/DAILY-J16106-EP-CLIP01.mp4?h=8523c8e918c191c86fea740524f33a6a' \
    -s 'http://watch.thecomedynetwork.ca/Flash/player.swf?themeURL=http://watch.thecomedynetwork.ca/themes/Comedy/player/theme.aspx' \
    -o out.mp4

 

[corporate_gadfly]

Re: Support for a site request thread

My bare-bones VMWare MePo install for some reason has never streamed anything successfully using OnlineVideos (because of what? lack of SAF codecs? DirectX or something else).

Installed SAF 5.02 and all is well with OnlineVideos in VMWare Fusion.

I will test CTV on my home rig this evening and then post the code.

CTV works . Code is as follows:

Spoiler

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;

namespace OnlineVideos.Sites
{
    public class CTVUtil : SiteUtilBase
    {
        private string baseUrl = @"http://watch.ctv.ca";
        private Regex mainCategoriesRegex = new Regex(@"<li[^>]*>\s*<a\sid=""(?<id>[^""]*)""\sonclick=""[^""]*""\shref=""(?<url>[^""]*)""\stitle=""[^""]*"">\s*(?<title>[^<]*)<span></span>\s*</a>\s*</li>",
            RegexOptions.Compiled);
        private Regex subcategoriesRegex = new Regex(@"<li[^>]*>\s*<a\sid=""(?<id>[^""]*)""\sonclick=""return\sInterface\.GetChildPanel\('Season'[^""]*""\shref=""(?<url>[^""]*)""\stitle=""[^""]*"">\s*(?<title>[^<]*)<span></span>\s*</a>\s*</li>",
            RegexOptions.Compiled);
        private Regex episodeListRegex = new Regex(@"<dd(\sid=""[^""]*"")?\sclass=""Thumbnail""><a\shref=""javascript:Interface\.PlayEpisode\((?<episode>[^,]*),\strue\s\)""\stitle=""(?<title>[^""]*)""><img\ssrc=""(?<thumb>[^""]*)""\s/><span></span></a></dd>.*?<dd class=""Description"">(?<description>[^<]*)</dd>",
            RegexOptions.Compiled | RegexOptions.Singleline);
        private Regex clipListRegex = new Regex(@"<dt><a\shref=""[^#]*#clip(?<clip>[^""]*)""\sonclick=""return\sPlaylist\.GetInstance.*?<img.*?/></a></dt>",
            RegexOptions.Compiled);
        private Regex clipUrlRegex = new Regex(@"Video\.Load\({url:'(?<url>[^']*)'.*?",
            RegexOptions.Compiled);
        private Regex rtmpUrlRegex = new Regex(@"rtmpe://(?<host>[^/]*)/ondemand/(?<file>[^?]*)\?(?<params>.*)",
            RegexOptions.Compiled);

        public override int DiscoverDynamicCategories()
        {
            Settings.Categories.Clear();

            string webData = GetWebData(baseUrl + @"/AJAX/VideoLibraryWithFrame.aspx");

            if (!string.IsNullOrEmpty(webData))
            {
                foreach (Match m in mainCategoriesRegex.Matches(webData))
                {
                    RssLink cat = new RssLink();

                    cat.Name = HttpUtility.HtmlDecode(m.Groups["title"].Value);
                    cat.Url = HttpUtility.HtmlDecode(m.Groups["url"].Value);
                    cat.HasSubCategories = true;

                    Settings.Categories.Add(cat);
                }
            }

            Settings.DynamicCategoriesDiscovered = true;
            return Settings.Categories.Count;
        }

        public override int DiscoverSubCategories(Category parentCategory)
        {
            parentCategory.SubCategories = new List<Category>();

            RssLink parentRssLink = (RssLink)parentCategory;
            string webData = GetWebData((parentRssLink).Url);

            if (!string.IsNullOrEmpty(webData))
            {
                foreach (Match m in subcategoriesRegex.Matches(webData))
                {
                    RssLink cat = new RssLink();

                    cat.ParentCategory = parentCategory;
                    cat.Name = HttpUtility.HtmlDecode(m.Groups["title"].Value);
                    cat.Url = HttpUtility.HtmlDecode(m.Groups["url"].Value);
                    // this id will be used later on, so store it in the .Other property
                    cat.Other = HttpUtility.HtmlDecode(m.Groups["id"].Value);
                    cat.HasSubCategories = false;

                    parentCategory.SubCategories.Add(cat);
                }
            }

            parentCategory.SubCategoriesDiscovered = true;
            return parentCategory.SubCategories.Count;
        }

        public override List<VideoInfo> getVideoList(Category category)
        {
            List<VideoInfo> result = new List<VideoInfo>();

            string parentId = (string) ((RssLink) category).Other;

            // Level 3 shows episodes
            string webData = GetWebData(baseUrl
                + @"/AJAX/VideoLibraryContents.aspx?GetChildOnly=true&PanelID=3&SeasonID="
                + parentId);

            if (!string.IsNullOrEmpty(webData))
            {
                foreach (Match m in episodeListRegex.Matches(webData))
                {
                    VideoInfo info = new VideoInfo();
                    info.Title = HttpUtility.HtmlDecode(m.Groups["title"].Value);
                    info.ImageUrl = HttpUtility.HtmlDecode(m.Groups["thumb"].Value);
                    info.Description = HttpUtility.HtmlDecode(m.Groups["description"].Value);
                    // this episode ID will be used later on, so store it in the .Other property
                    info.Other = HttpUtility.HtmlDecode(m.Groups["episode"].Value);

                    result.Add(info);
                }
            }

            return result;
        }

        public override List<string> getMultipleVideoUrls(VideoInfo video, bool inPlaylist = false)
        {
            List<string> result = new List<string>();

            // Level 4 shows clips
            string webData = GetWebData(baseUrl
                + @"/AJAX/VideoLibraryContents.aspx?GetChildOnly=true&PanelID=4&EpisodeID="
                + video.Other);

            if (!string.IsNullOrEmpty(webData))
            {
                foreach (Match m in clipListRegex.Matches(webData))
                {
                    string clipId = HttpUtility.HtmlDecode(m.Groups["clip"].Value);
                    // this is the URL which will eventually reveal the rtmpe locations
                    result.Add(String.Format("http://cls.ctvdigital.net/cliplookup.aspx?id={0}", clipId));
                }
            }

            return result;
        }

        public override string getPlaylistItemUrl(VideoInfo clonedVideoInfo, string chosenPlaybackOption, bool inPlaylist = false)
        {
            Log.Debug(@"Video URL (before): {0}", clonedVideoInfo.VideoUrl);

            string result = clonedVideoInfo.VideoUrl;

            // must specify referer (or we will get 403 Forbidden from cls.ctvdigital.net)
            string webData = GetWebData(clonedVideoInfo.VideoUrl, null, baseUrl);

            if (!string.IsNullOrEmpty(webData))
            {
                Match urlMatch = clipUrlRegex.Match(webData);
                if (urlMatch.Success)
                {
                    string rtmp = HttpUtility.HtmlDecode(urlMatch.Groups["url"].Value);

                    Log.Debug("RTMP URL found: {0}", rtmp);

                    Match m = rtmpUrlRegex.Match(rtmp);
                    if (m.Success)
                    {
                        string rtmpUrl = String.Format("rtmp://{0}/ondemand?{1}", m.Groups["host"], m.Groups["params"]);
                        string playPath = String.Format("mp4:{0}", m.Groups["file"]);

                        string url = String.Format("http://127.0.0.1/stream.flv?rtmpurl={0}&playpath={1}&swfVfy={2}",
                            HttpUtility.UrlEncode(rtmpUrl),
                            HttpUtility.UrlEncode(playPath),
                            HttpUtility.UrlEncode(@"http://watch.ctv.ca/Flash/player.swf?themeURL=http://watch.ctv.ca/themes/CTV/player/theme.aspx"));

                        Log.Debug(@"Video URL (after): {0}", url);
                        result = ReverseProxy.GetProxyUri(RTMP_LIB.RTMPRequestHandler.Instance, url);
                    }
                }
            }
            return result;
        }
    }
}

Does anyone want to take it for a "test ride"? Sorry, no descriptions at the moment and minimal thumbnails (easy to add, I guess)

Unless anyone else has a better idea, I'm planning on creating a BaseCTVUtil and try to get all the other stations working (as per my original post) except Comedy Network. I'm pretty sure CTV, TSN (sports), CTV News, Discovery, Space, Bravo, BNN and Fashion can all be run with the same infrastructure but differing base URLs.

And again, I apologize for my C# (it is my first project ever).

Cheers.

 

[offbyone]

Great progress! Once you feel you're in a user fiendly state with you c# you can add this to the OnlineVideos google code project if you want. I can give you commit access.

 

[corporate_gadfly]

Great progress! Once you feel you're in a user fiendly state with you c# you can add this to the OnlineVideos google code project if you want. I can give you commit access.

Thanks for the kind words. I am learning a lot by looking at the existing code. It's still early days. I will let you know once I have a base util for CTV ready.

I have a question regarding skins. In the default skin, I can see the descriptions on the episode list. On StreamedMP, I only see titles (and no descriptions). I do see a little space in StreamedMP for the length/airdate, etc. Any ideas how to handle that consistently? Perhaps, I'll add screenshots once my kid is done watching his movie.

Cheers.

 

[offbyone]

It's up to the skin designer what and where to display any info. The code basically only provides the info to the skin.

 

[corporate_gadfly]

Great progress! Once you feel you're in a user fiendly state with you c# you can add this to the OnlineVideos google code project if you want. I can give you commit access.

Good news and bad news!

Good news is that I got CTV, Space, Fashion, Bravo!, Bravo! FACT, CTV Local News working.
Bad news is that Comedy Network, TSN and other channels use the "difficult" style of RTMP URL which I have not been able to do anything about (will try to tackle later, if at all possible - looks impossible at the moment).

If you would prefere, I can send you a diff, or you can give me commit access to google code project.

Cheers,

 

[doskabouter]

Great progress! Once you feel you're in a user fiendly state with you c# you can add this to the OnlineVideos google code project if you want. I can give you commit access.

Good news and bad news!

Good news is that I got CTV, Space, Fashion, Bravo!, Bravo! FACT, CTV Local News working.
Bad news is that Comedy Network, TSN and other channels use the "difficult" style of RTMP URL which I have not been able to do anything about (will try to tackle later, if at all possible - looks impossible at the moment).

If you would prefere, I can send you a diff, or you can give me commit access to google code project.

Cheers,

What do you mean by "difficult" style urls?

You could try fiddling with the "auth=" part i.c.w. -u parameter of rtmpdump?

If that fails, could you provide me with a full rtmp-url, so that I can take a crack at it?
Or better, post or PM me your code, so I can get the rtmp-url, because it could be that the auth-part is only usable for a short time.

 

[corporate_gadfly]

What do you mean by "difficult" style urls?

You could try fiddling with the "auth=" part i.c.w. -u parameter of rtmpdump?

If that fails, could you provide me with a full rtmp-url, so that I can take a crack at it?
Or better, post or PM me your code, so I can get the rtmp-url, because it could be that the auth-part is only usable for a short time.

Hi Doskabouter,

Here is an example of a good URL:

rtmpe://cp45924.edgefcs.net/ondemand/s_!ctv/shows/2011/08/22/CWD-308-EP-CLIP01.mp4?auth=dbEducabOb7cXbDdLcLaoaYdHaVbwagdAag-bozOsG-eS-iYG-owE2nmMCw&aifp=v001&slist=/s_!ctv/shows/2011/08/22/

obtained by running the following command on Mac OS X command-line (if you don't provide referer, you get denied):

curl --referer "http://watch.ctv.ca" "http://cls.ctvdigital.net/cliplookup.aspx?id=522506"

Following is an example of a bad "difficult" URL (notice no parameters except h):

rtmpe://tsn.fcod.llnwd.net/a5504/d1/2011/09/06/burke4_090611.mp4?h=7da678f65cb678b5fc261a4dce425bd8

which was obtained by the following command-line (different clip ID):

curl --referer "http://watch.ctv.ca" "http://cls.ctvdigital.net/cliplookup.aspx?id=527515"

The wireshark dump for the "difficult" URL looks unlike anything that I am used to at least.

Cheers.

 

[doskabouter]

grr. can't make heads or tails from it, and searching the internet/peeking at xbmc didn't help either.

Edit: maybe some lucky shot combination of playpath/app/auth/etc param of rtmpdump...