AI-based search in (local) MediaItems (1 Viewer)

[ge2301]

Hi,

I'm planning to write a plugin for MediaPortal 2, that supports users to find the right media items with help of AI frameworks.

Following is used:

• OpenAI Framework
• ChatGPT3.5 Client
• Model: text-davinci-003
• OpenAI API

It makes mostly sense in combination of speech control which is not existing/working for MP2 as complex questions are uncomfortable to enter with remote control.

1. In first step I want to only limit this to existing local media items in the MP2-DB
2. In second step (only if first one works well) it could be extended to all media items, also not in MP2-DB existent ones

Target state for Step 1:

• You can perform easy filter actions for year, gerne, actors, ... (E.g.: "Show me all movies from the genre Thriller", ...)
• You can perform complex filter actions (E.g.: "Show me all movies, that contain cowboys", "Show me movies with blue-skinned aliens", ...)

I wrote the program at first independently from MP2 as this is easier especially for testing purposes. Once it works content-wise it can be integrated as plugin.

What does the program do yet:

• The C# WPF application reads movie titles from MP2 SQLite database table (M_MOVIEITEM) and displays the complete list of movie titles on the right side of the window (I first concentrate only on the movie title)
• It then uses the OpenAI GPT-3 API to find the best movie match based on a search question entered by the user (Restiction: Learning date are from 12/2021, all titles after that can currently not be considered)
• The result is now displayed in a ListBox named resultListBox, which allows for multiple matches to be shown. The GetBestMovieMatches method returns a list of best matches, and the resultListBox.ItemsSource is set to this list for display.

MainWindow.xaml

<Window x:Class="MovieSearchApp.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="Movie Search" Height="350" Width="500">
  <Grid>
    <Grid.ColumnDefinitions>
      <ColumnDefinition Width="*" />
      <ColumnDefinition Width="*" />
    </Grid.ColumnDefinitions>

    <!-- Left Column: Movie List -->
    <ListBox x:Name="movieListBox" Grid.Column="0" Margin="10"/>

    <!-- Right Column: Search and Result -->
    <StackPanel Grid.Column="1" Margin="10">
      <Label Content="Enter search query:"/>
      <TextBox x:Name="searchTextBox" Width="200" Margin="0,0,0,10"/>
      <Button Content="OK" Click="SearchButton_Click"/>
      <Label Content="Best Movie Matches:" Margin="0,20,0,0"/>
      <ListBox x:Name="resultListBox" Width="200" Height="60"/>
    </StackPanel>
  </Grid>
</Window>

MainWindow.xaml.cs

using System;
using System.Collections.Generic;
using System.Data.SQLite;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;

namespace MovieSearchApp
{
    public partial class MainWindow : Window
    {
        private const string ConnectionString = "Data Source=C:\\ProgramData\\Team MediaPortal\\MP2-Server\\Database\\Datastore.s3db;Version=3;";
        private const string OpenAIApiKey = "your_openai_api_key";
        private const string OpenAIEndpoint = "https://api.openai.com/v1/engines/davinci-codex/completions";

        public MainWindow()
        {
            InitializeComponent();
            PopulateMovieList();
        }

        private void PopulateMovieList()
        {
            using (SQLiteConnection connection = new SQLiteConnection(ConnectionString))
            {
                connection.Open();

                string sql = "SELECT MOVIENAME FROM M_MOVIEITEM;";
                using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                using (SQLiteDataReader reader = command.ExecuteReader())
                {
                    while (reader.Read())
                    {
                        string movieTitle = reader["MOVIENAME"].ToString();
                        movieListBox.Items.Add(movieTitle);
                    }
                }
            }
        }

        private async void SearchButton_Click(object sender, RoutedEventArgs e)
        {
            try
            {
                string searchQuery = searchTextBox.Text.Trim();

                if (string.IsNullOrEmpty(searchQuery))
                {
                    MessageBox.Show("Please enter a search query.");
                    return;
                }

                // Use ChatGPT to generate responses based on the search question
                List<string> bestMatches = await GetBestMovieMatches(searchQuery);

                // Display the results
                resultListBox.ItemsSource = bestMatches;
            }
            catch (Exception ex)
            {
                MessageBox.Show($"An error occurred: {ex.Message}");
            }
        }

        private async Task<List<string>> GetBestMovieMatches(string searchQuery)
        {
            List<string> bestMatches = new List<string>();

            foreach (string movieTitle in movieListBox.Items)
            {
                // Use ChatGPT to determine the match
                string generatedText = await GenerateResponseWithGPT(searchQuery, movieTitle);

                // You can add your matching logic here based on the generated text
                // For simplicity, we're using the whole generated text as the match
                if (!string.IsNullOrEmpty(generatedText))
                {
                    bestMatches.Add(generatedText);
                }
            }

            return bestMatches;
        }

        private async Task<string> GenerateResponseWithGPT(string searchQuery, string movieTitle)
        {
            using (HttpClient client = new HttpClient())
            {
                client.DefaultRequestHeaders.Add("Authorization", $"Bearer {OpenAIApiKey}");

                string prompt = $"Find the best movie match for the search query: {searchQuery}. Movie Title: {movieTitle}";

                var requestData = new
                {
                    prompt,
                    max_tokens = 50,
                    temperature = 0.7
                };

                string requestDataJson = Newtonsoft.Json.JsonConvert.SerializeObject(requestData);
                var content = new StringContent(requestDataJson, Encoding.UTF8, "application/json");

                HttpResponseMessage response = await client.PostAsync(OpenAIEndpoint, content);

                if (response.IsSuccessStatusCode)
                {
                    string responseJson = await response.Content.ReadAsStringAsync();
                    dynamic responseData = Newtonsoft.Json.JsonConvert.DeserializeObject(responseJson);
                    string generatedText = responseData.choices[0].text;

                    return generatedText;
                }
                else
                {
                    throw new Exception($"ChatGPT API request failed. Status code: {response.StatusCode}");
                }
            }
        }
    }
}

Current problem is, that always the error "No Data" is thrown. Solved
Does anyone have a OpenAI API (1/user is free) and can support me writing/testing?
Make sure to replace "database.db" with the actual path to your SQLite database file (I kept mine in the code, as this will not change for most users) and "your_openai_api_key" with your OpenAI API key.

 

[ge2301]

I changed the code using the OpenAI nugget package and got it running But the maximum token were exceeded with my collection, so I limited it to the first 500 movies of my DB. I wrote as test query „movie with a clown“ and it listed „It“ in the output area

 

[Brownard]

Looks good. I'm not sure if its a limitation of the OpenAI API but your current code will run into rate limiting and performance issues because you perform a new web request for every individual movie, is there a way to pass a list of movies in a single request instead of each movie individually?

 

[ge2301]

Looks good. I'm not sure if its a limitation of the OpenAI API but your current code will run into rate limiting and performance issues because you perform a new web request for every individual movie, is there a way to pass a list of movies in a single request instead of each movie individually?

Yes, it was initially due to the token limitation, but then I changed it with manual limitation of media items. In the end a flexible solution, that counts the needed token and splits the query into jobs would be best. Also the merge into movieListText could be combined into the SQLite DB import function, it was just a trial to check if it'S generally doable


Current code is as below:

using System;
using System.Collections.Generic;
using System.Data.SQLite;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using OpenAI_API.Completions;
using OpenAI_API;

namespace MovieSearchApp
{

    public partial class MainWindow : Window
    {
        private const string ConnectionString = "Data Source=C:\\ProgramData\\Team MediaPortal\\MP2-Server\\Database\\Datastore.s3db;Version=3;";
        private const string OpenAIApiKey = "*******************";

        private string movieListText; // Variable to hold the concatenated movie titles


        public MainWindow()
        {
            InitializeComponent();
            PopulateMovieList();
        }

        private void PopulateMovieList()
        {
            using (SQLiteConnection connection = new SQLiteConnection(ConnectionString))
            {
                connection.Open();

                string sql = "SELECT MOVIENAME FROM M_MOVIEITEM;";
                using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                using (SQLiteDataReader reader = command.ExecuteReader())
                {
                    while (reader.Read())
                    {
                        string movieTitle = reader["MOVIENAME"].ToString();
                        movieListBox.Items.Add(movieTitle);
                    }
                }
            }

            // Call the method to join ListBox items into a string
            movieListText = JoinListBoxItemsToString(movieListBox);
        }

        string JoinListBoxItemsToString(ListBox listBox)
        {
            // Create a StringBuilder to efficiently concatenate strings
            StringBuilder resultBuilder = new StringBuilder();

            // Iterate through each item in the ListBox
            //foreach (object item in listBox.Items)
            //{
            // Append the item to the StringBuilder
            //    resultBuilder.Append(item.ToString());

            // Add ";" as a separator for the next item
            //    resultBuilder.Append(";");
            //}

            // Remove the trailing ";" if there are items in the ListBox
            //if (listBox.Items.Count > 0)
            //
            //    resultBuilder.Length--; // Remove the last character
            //}


            // Limit the loop to the first 300 items
            int itemsToConcatenate = Math.Min(listBox.Items.Count, 300);

            // Iterate through each item in the ListBox
            for (int i = 0; i < itemsToConcatenate; i++)
            {
                // Append the item to the StringBuilder
                resultBuilder.Append(listBox.Items[i].ToString());

                // Add ";" as a separator for the next item
                resultBuilder.Append(";");
            }

            // Remove the trailing ";" if there are items in the ListBox
            if (itemsToConcatenate > 0)
            {
                resultBuilder.Length--; // Remove the last character
            }

            // Return the final joined string
            return resultBuilder.ToString();
        }

        private async void SearchButton_Click(object sender, RoutedEventArgs e)
        {
            try
            {
                string searchQuery = searchTextBox.Text.Trim();

                if (string.IsNullOrEmpty(searchQuery))
                {
                    MessageBox.Show("Please enter a search query.");
                    return;
                }

                // Use ChatGPT to generate responses based on the search question
                List<string> bestMatches = await GetBestMovieMatches(searchQuery);

                // Display the results
                resultListBox.ItemsSource = bestMatches;
            }
            catch (Exception ex)
            {
                MessageBox.Show($"An error occurred: {ex.Message}");
            }
        }

        private async Task<List<string>> GetBestMovieMatches(string searchQuery)
        {
            // Use ChatGPT to determine the matches
            string generatedText = await GenerateResponseWithGPT(searchQuery, movieListText);

            // Split the generated text into a list of matches
            List<string> bestMatches = new List<string>(generatedText.Split(';'));

            return bestMatches;
        }

        private async Task<string> GenerateResponseWithGPT(string searchQuery, string movieListText)
        {
            APIAuthentication aPIAuthentication = new APIAuthentication(OpenAIApiKey);
            OpenAIAPI openAiApi = new OpenAIAPI(aPIAuthentication);
            try
            {
                string prompt = $"List all movies from the list fitting to the search query: {searchQuery}. Movies: {movieListText}";
                string model = "text-davinci-003";
                int maxTokens = 50;

                var completionRequest = new CompletionRequest
                {
                    Prompt = prompt,
                    Model = model,
                    MaxTokens = maxTokens
                };

                var completionResult = await openAiApi.Completions.CreateCompletionAsync(completionRequest);
                var generatedText = completionResult.Completions[0].Text; //completionResult.Choices[0].Text.Trim();

                return generatedText;
            }
            catch (Exception ex)
            {
                MessageBox.Show($"An error occurred: {ex.Message}");
                return "error";
            }
        }
    }
}

 

[ge2301]

Small update. I improved the code further and it seems the response is quite ok, if the criteria are clearly defined.
For example " Yellow characters" does not show good results, but "Characters with yellow skin" is fine and includes Minions etc.

Here some examples of the current state

 

[ge2301]

Next update: Sometimes ChatGPT made comments, which were not related to the filtered media items.
E.g "No results found", "Horror genre movies: ..., Half-Horror genre movies: ...". Those comments are filtered out now. I added a new function, which considers only items in the result list, that are also appearing in the input list. As many code parts changed, I attached the code in case you want to try it

You need to add your API KEY in "ChatGPT_MovieMatcher.csproj" and check right above this code line, if the path to the MP2 data base is correct.
Then just build and run.

 

[ge2301]

So far I needed to limit the media items to respect the token limit of ChatGPT.
Now the complete media items can be used, because the movie list is splitted into chunks of a specified size and responses for each chunk are concentrated into the result list

 private async Task<string> GenerateResponseWithGPT(string searchQuery, List<string> movieList)
 {
     // Chunk the movie list into smaller segments
     int chunkSize = 500; // Adjust the chunk size based on your requirements
     List<List<string>> chunks = SplitList(movieList, chunkSize);

     // Generate responses for each chunk and concatenate the results
     List<string> generatedResponses = new List<string>();
     foreach (var chunk in chunks)
     {
         string movieListText = string.Join(";", chunk);
         string prompt = $"Filter for only movies from <{movieListText}> if following is true for the movie: {searchQuery}. Use a simikolon as separator and do not write any summary comments. No result means the output is simply empty";

         string responseChunk = await GenerateResponseChunk(prompt);
         generatedResponses.Add(responseChunk);
     }

     // Concatenate the generated responses into a single string
     string finalResponse = string.Concat(generatedResponses);

     return finalResponse;
 }

 private async Task<string> GenerateResponseChunk(string prompt)
 {
     APIAuthentication aPIAuthentication = new APIAuthentication(OpenAIApiKey);
     OpenAIAPI openAiApi = new OpenAIAPI(aPIAuthentication);
     //string movieListText = string.Join(";", movieList);
     try
     {
         string model = "text-davinci-003";
         int maxTokens = 700;

         var completionRequest = new CompletionRequest
         {
             Prompt = prompt,
             Model = model,
             MaxTokens = maxTokens
         };

         var completionResult = await openAiApi.Completions.CreateCompletionAsync(completionRequest);
         var generatedText = completionResult.Completions[0].Text; //completionResult.Choices[0].Text.Trim();
         MessageBox.Show($"Filtered Movies: {completionResult}");
         return generatedText;
     }
     catch (Exception ex)
     {
         MessageBox.Show($"An error occurred: {ex.Message}");
         return "error";
     }
 }

 private void FilterResultListBox()
 {
     // Filter items in resultListBox based on items present in movieListBox
     List<string> filteredResult = new List<string>();
     foreach (var item in resultListBox.Items)
     {
         if (movieListBox.Items.Contains(item))
         {
             filteredResult.Add(item.ToString());
         }
     }

     // Display the filtered items in resultListBox
     DisplayMovieList(resultListBox, filteredResult);
 }

 private async Task<List<string>> FilterMoviesWithGPT(string searchQuery)
 {
     // Use ChatGPT to determine if each movie fits the search query
     string generatedText = await GenerateResponseWithGPT(searchQuery, movieList);
     //MessageBox.Show($"Filtered Movies: {generatedText}");
     // Split the generated text into a list of matches
     List<string> filteredMovies = new List<string>(generatedText.Split(';'));

     return filteredMovies;

 }
 private void DisplayMovieList(ListBox listBox, List<string> movies)
 {
     listBox.ItemsSource = movies;
 }

 // Split a list into chunks of a specified size
 private List<List<T>> SplitList<T>(List<T> source, int chunkSize)
 {
     return source
         .Select((x, i) => new { Index = i, Value = x })
         .GroupBy(x => x.Index / chunkSize)
         .Select(x => x.Select(v => v.Value).ToList())
         .ToList();
 }