diff --git a/README.md b/README.md index 272cc16..031df3d 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ red-plex now includes a comprehensive web-based GUI for users who prefer a visua - **⚙️ Configuration Management**: View and edit all settings (API keys, Plex config, rate limits) through web forms - **🎨 Collage Operations**: Convert new collages and view existing collections - **🔖 Bookmark Operations**: Convert bookmarks from RED and OPS trackers +- **🏷️ Site Tags Operations**: Scan albums using album/artist names and create tag-based collections - **🗄️ Database Management**: View statistics, update albums, reset tables - **⚡ Real-time Updates**: Live status updates during long operations via WebSocket - **📱 Mobile-Responsive**: Bootstrap-based design that works on all devices @@ -45,12 +46,6 @@ red-plex gui --debug The web interface provides the same functionality as the CLI commands but with a user-friendly visual interface, real-time progress updates, and intuitive navigation. -## What are RED and OPS? - -- **Redacted (RED)**: A private music tracker focused on high-quality audio files -- **Orpheus Network (OPS)**: Another private music tracker with curated content -- Both use the Gazelle framework and offer "collages" (curated collections) and personal bookmarks - ## Table of Contents - [Prerequisites](#prerequisites) - [Installation](#installation) @@ -62,7 +57,10 @@ The web interface provides the same functionality as the CLI commands but with a - [Usage & Commands](#usage--commands) - [Configuration Commands](#configuration-commands) - [Collages](#collages) + - [Upstream Sync](#upstream-sync) - [Bookmarks](#bookmarks) + - [Site Tags](#site-tags) + - [Remote Mappings](#remote-mappings) - [Fetch Mode (-fm)](#fetch-mode--fm) - [Database Commands](#database-commands) - [Examples](#examples) @@ -152,9 +150,10 @@ OPS: seconds: 15 ``` -### Getting Your Plex Token +### Getting Your Plex HTTPs URL -Visit: https://plex.tv/api/resources?includeHttps=1&X-Plex-Token={YOUR_TOKEN} +Visit: https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token +Also visit: https://plex.tv/api/resources?includeHttps=1&X-Plex-Token={YOUR_TOKEN} ## Overview @@ -169,7 +168,10 @@ Visit: https://plex.tv/api/resources?includeHttps=1&X-Plex-Token={YOUR_TOKEN} - **Multi-Site**: Works with Redacted ("red") and Orpheus Network ("ops"). - **Web Interface**: Modern Flask-based GUI with Bootstrap styling and real-time updates. - **Collections from Collages/Bookmarks**: Create or update entire Plex collections for each collage or bookmarked set. -- **Local SQLite Database**: All data (albums, collages, bookmarks) is kept in one DB, no more CSV. +- **Site Tags Mapping**: Map your Plex albums to site groups using album and artist names, then create collections based on specific tags. +- **Upstream Sync**: Push local Plex collection changes back to upstream collages on RED, enabling bidirectional synchronization between Plex collections and site collages. +- **Remote Mappings**: Core functionality that links Plex collections with site group IDs, enabling features like site tags and upstream sync. +- **Local SQLite Database**: All data (albums, collages, bookmarks, site mappings) is kept in one DB, no more CSV. - **Two Fetch Modes**: Choose between `torrent_name` (default) for direct path matching or `query` for metadata-based searches in Plex. - **Configurable Logging**: Choose between INFO, DEBUG, etc., in `config.yml`. - **Rate Limiting**: Respects site rate limits and retries on errors. @@ -216,6 +218,40 @@ red-plex collages convert [COLLAGE_IDS] --site [red|ops] --fetch-mode [torrent_n red-plex collages update --fetch-mode [torrent_name|query] ``` +### Upstream Sync + +**⚠️ Important Prerequisites:** +- Only works with **RED** (Redacted) - OPS doesn't have the required API operations +- You must **own** the collages on the tracker site +- First **convert the collage** from the site using `red-plex collages convert` (even if the collage is empty) +- Run **remote mappings scan** to link Plex items with site group IDs: `red-plex db remote-mappings scan -s red` + +**How it works:** +- Pushes local Plex collection changes back to upstream collages on RED +- Uses fuzzy string matching to find albums on the site (may cause occasional mismatches) +- Shows confirmation dialog with exactly what will be added before making changes +- **Never deletes** anything from upstream collages, only adds missing items +- If you update your Plex library, run a new remote-mappings scan to link new albums + +```bash +# Sync all collections to upstream collages +red-plex collages update --push + +# Sync specific collections to upstream collages +red-plex collages update 12345 67890 --push + +# Alternative flag name +red-plex collages update --update-upstream +``` + +**Scanning Process:** +- Can be cancelled anytime with **Ctrl+C** and resumed later +- Each mapping is saved to the database during scanning (no data loss) +- Scans latest added entries in Plex, so interrupted scans can continue from where they left off +- ⚠️ **Current limitation**: If you have many unmatched albums (e.g., 200 albums with no matches), they'll appear in the scan queue each time. This is being improved to ignore previously failed matches. + +**Available in both CLI and Web Interface** + ### Bookmarks ```bash @@ -226,6 +262,32 @@ red-plex bookmarks convert --site [red|ops] --fetch-mode [torrent_name|query] red-plex bookmarks update --fetch-mode [torrent_name|query] ``` +### Site Tags + +```bash +# Create collections from albums matching specific tags +red-plex extras site-tags convert --tags [tag1,tag2,...] --collection-name [name] +``` + +### Remote Mappings + +Remote mappings are the core functionality that links your Plex music library with site group IDs, enabling features like site tags and upstream sync. + +```bash +# Scan albums and create remote mappings using album and artist names +red-plex db remote-mappings scan --site [red|ops] [--always-skip] + +# Reset remote mappings (clears the relationship data) +red-plex db remote-mappings reset +``` + +**Scanning Features:** +- **Interruptible**: Can be cancelled with **Ctrl+C** at any time and resumed later +- **No data loss**: Each mapping is saved to database immediately during the scan +- **Incremental**: Processes latest Plex additions first, so you can resume interrupted scans +- **Fuzzy matching**: Uses string similarity to match Plex albums with site releases +- ⚠️ **Current limitation**: Albums with no matches will reappear in future scans (improvement planned) + ### Fetch Mode (-fm) The `--fetch-mode` (or `-fm`) option controls how red-plex locates albums in Plex: @@ -249,6 +311,10 @@ red-plex db collections reset # Clear the collage collections table # Manage bookmarks table red-plex db bookmarks reset # Clear the bookmark collections table + +# Manage remote mappings (core feature for site tags and upstream sync) +red-plex db remote-mappings scan --site [red|ops] # Create Plex-to-site mappings +red-plex db remote-mappings reset # Clear remote mapping data ``` ## Examples @@ -264,6 +330,16 @@ red-plex collages convert 1111 2222 3333 --site ops # From bookmarks (RED or OPS), default mode red-plex bookmarks convert --site red + +# Remote mappings - scan albums and create mappings using album/artist names +red-plex db remote-mappings scan --site red + +# Site tags - create collection from specific tags +red-plex extras site-tags convert --tags "electronic,ambient" --collection-name "Electronic Ambient" + +# Upstream sync - push Plex collection changes back to RED collages (RED only) +red-plex collages update 12345 --push # Sync specific collage +red-plex collages update --push # Sync all collages ``` ### Updating Collections @@ -277,6 +353,9 @@ red-plex bookmarks update # Update albums from Plex red-plex db albums update + +# Sync collections to upstream collages (RED only) +red-plex collages update --push ``` ### Using Query Fetch Mode @@ -305,9 +384,21 @@ red-plex collages convert 12345 67890 --site red # 4. Create collection from your bookmarks red-plex bookmarks convert --site red -# 5. Later, update all collections with new releases +# 5. Scan albums for remote mappings (required for site tags and upstream sync) +red-plex db remote-mappings scan --site red + +# 6. Create collections from specific tags +red-plex extras site-tags convert --tags "electronic,downtempo" --collection-name "Electronic Downtempo" + +# 7. Sync collections back to upstream collages (RED only, requires ownership) +red-plex collages update --push + +# 8. Later, update all collections with new releases red-plex collages update red-plex bookmarks update + +# 9. If you add new music to Plex, re-scan for new mappings +red-plex db remote-mappings scan --site red ``` #### Web Interface @@ -325,7 +416,13 @@ red-plex gui # 6. Use the Bookmarks page to convert your bookmarks -# 7. Return to Database page later to update all collections +# 7. Use the Remote Mappings page to scan albums and create mappings + +# 8. Use Site Tags to create collections from specific tags + +# 9. Use the Collages page to sync collections to upstream (RED only) + +# 10. Return to Database page later to update all collections ``` ## Configuration Details @@ -365,13 +462,6 @@ SECTION_NAME: Music ### Common Issues -#### "No module named 'plexapi'" Error -```bash -pip install plexapi -# or -pip install red-plex --upgrade -``` - #### Authentication Errors - Verify your API keys are correct in `config.yml` - Check that your Plex token is valid @@ -417,6 +507,16 @@ pip install red-plex --upgrade - `INFO`: Standard information (default) - `WARNING`: Minimal output - **Collection Updates**: When you run `collages update` or `bookmarks update`, new albums are added to existing Plex collections, but removed items from tracker collages are not automatically removed from Plex collections +- **Upstream Sync (RED Only)**: + - Only works with RED (Redacted) - OPS doesn't support the required API operations + - You must own the collages you want to sync to + - Convert collages from the site first, even if they're empty + - Run `red-plex db remote-mappings scan --site red` to create the necessary mappings + - Uses fuzzy string matching which may occasionally cause mismatches + - Never deletes items from upstream collages, only adds missing ones + - Can be interrupted with Ctrl+C and resumed later with no data loss + - Re-scan after adding new music to your Plex library +- **Remote Mappings**: Core functionality that links Plex items with site group IDs. Required for both site tags and upstream sync features ## Contributing @@ -431,5 +531,3 @@ pip install -e . ``` --- - -**Disclaimer**: This tool is for personal use with your own music library and tracker accounts. Respect the rules and terms of service of the private trackers you use. \ No newline at end of file diff --git a/SITE_TAGS_USAGE.md b/SITE_TAGS_USAGE.md new file mode 100644 index 0000000..cf1682f --- /dev/null +++ b/SITE_TAGS_USAGE.md @@ -0,0 +1,90 @@ +# Site Tags Feature Usage + +This document explains how to use the new site tags functionality to create Plex collections based on site tags. + +## Overview + +The site tags feature allows you to: +1. Scan your Plex albums and map them to site groups based on filename searches +2. Create Plex collections from albums that match specific tags + +## Prerequisites + +1. Ensure your Plex server is configured and accessible +2. Configure your RED/OPS API credentials in the config file +3. Populate your albums database with `red-plex db albums update` + +## Usage + +### 1. Scan Albums for Site Tags + +This command scans your albums and creates mappings to site groups: + +```bash +# Scan albums for RED +red-plex extras site-tags scan --site red + +# Scan albums for OPS +red-plex extras site-tags scan --site ops +``` + +The scan process: +- Fetches track file paths from each album in Plex +- Searches the site using filenames +- If multiple matches are found, prompts you to choose +- Creates mappings between Plex rating_key and site group_id + tags +- Is incremental - only processes albums not yet scanned + +### 2. Create Collections from Tags + +Create Plex collections based on tag filters: + +```bash +# Create a collection of electronic music +red-plex extras site-tags convert --site red --tags "electronic" --collection-name "Electronic Music" + +# Create a collection with multiple tag requirements +red-plex extras site-tags convert --site red --tags "drum.and.bass,liquid" --collection-name "Liquid DNB" +``` + +### 3. Reset Site Tag Mappings + +If you need to reset the mappings: + +```bash +# Reset mappings for a specific site +red-plex extras site-tags reset --site red + +# Reset all mappings +red-plex extras site-tags reset +``` + +## Examples + +### Example Workflow + +1. **Initial scan:** + ```bash + red-plex db albums update + red-plex extras site-tags scan --site red + ``` + +2. **Create genre-based collections:** + ```bash + red-plex extras site-tags convert --site red --tags "electronic" --collection-name "Electronic" + red-plex extras site-tags convert --site red --tags "jazz" --collection-name "Jazz" + red-plex extras site-tags convert --site red --tags "rock" --collection-name "Rock" + ``` + +3. **Create specific sub-genre collections:** + ```bash + red-plex extras site-tags convert --site red --tags "drum.and.bass,liquid" --collection-name "Liquid DNB" + red-plex extras site-tags convert --site red --tags "ambient,dark" --collection-name "Dark Ambient" + ``` + +## Notes + +- The scan process handles filename variations (e.g., removes track numbers like "1. " from the beginning) +- Multiple matches require user confirmation to ensure accuracy +- Collections are created/updated in Plex - existing collections with the same name will be replaced +- The tag matching uses AND logic - all specified tags must be present for an album to match \ No newline at end of file diff --git a/red_plex/domain/models.py b/red_plex/domain/models.py index 423cd96..4bbe99d 100644 --- a/red_plex/domain/models.py +++ b/red_plex/domain/models.py @@ -30,6 +30,7 @@ class TorrentGroup: artists: List[str] = field(default_factory=list) album_name: str = "" file_paths: List[str] = field(default_factory=list) + tags: List[str] = field(default_factory=list) def __hash__(self) -> int: return hash((self.id, tuple(self.artists), self.album_name, tuple(self.file_paths))) diff --git a/red_plex/infrastructure/cli/cli.py b/red_plex/infrastructure/cli/cli.py index f539e3d..763ffe6 100644 --- a/red_plex/infrastructure/cli/cli.py +++ b/red_plex/infrastructure/cli/cli.py @@ -1,30 +1,16 @@ """Collection creator CLI.""" -import os -import subprocess import sys -from typing import List import click -import yaml -from red_plex.domain.models import Collection -from red_plex.infrastructure.config.config import ( - CONFIG_FILE_PATH, - load_config, - save_config, - ensure_config_exists -) -from red_plex.infrastructure.config.models import Configuration +from red_plex.infrastructure.cli.commands.bookmarks import bookmarks +from red_plex.infrastructure.cli.commands.collages import collages +from red_plex.infrastructure.cli.commands.config import config +from red_plex.infrastructure.cli.commands.db import db +from red_plex.infrastructure.cli.commands.extras import extras +from red_plex.infrastructure.cli.commands.gui import gui from red_plex.infrastructure.db.local_database import LocalDatabase -from red_plex.infrastructure.logger.logger import logger, configure_logger -from red_plex.infrastructure.plex.plex_manager import PlexManager -from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI -from red_plex.infrastructure.service.collection_processor import CollectionProcessingService -from red_plex.use_case.create_collection.album_fetch_mode import AlbumFetchMode -from red_plex.use_case.create_collection.query.query_sync_collection import ( - QuerySyncCollectionUseCase) -from red_plex.use_case.create_collection.torrent_name.torrent_name_sync_collection import ( - TorrentNameCollectionCreatorUseCase) +from red_plex.infrastructure.logger.logger import configure_logger @click.group() @@ -35,459 +21,13 @@ def cli(ctx): ctx.obj['db'] = LocalDatabase() -# config -@cli.group() -def config(): - """View or edit configuration settings.""" - - -# config show -@config.command('show') -def show_config(): - """Display the current configuration.""" - config_data = load_config() - path_with_config = ( - f"Configuration path: {CONFIG_FILE_PATH}\n\n" + - yaml.dump(config_data.to_dict(), default_flow_style=False) - ) - click.echo(path_with_config) - - -# config edit -@config.command('edit') -def edit_config(): - """Open the configuration file in the default editor.""" - # Ensure the configuration file exists - ensure_config_exists() - - # Default to 'nano' if EDITOR is not set - editor = os.environ.get('EDITOR', 'notepad' if os.name == 'nt' else 'nano') - click.echo(f"Opening config file at {CONFIG_FILE_PATH}...") - try: - subprocess.call([editor, CONFIG_FILE_PATH]) - except FileNotFoundError: - message = f"Editor '{editor}' not found. \ - Please set the EDITOR environment variable to a valid editor." - logger.error(message) - click.echo(message) - except Exception as exc: # pylint: disable=W0718 - logger.exception('Failed to open editor: %s', exc) - click.echo(f"An error occurred while opening the editor: {exc}") - - -# config reset -@config.command('reset') -def reset_config(): - """Reset the configuration to default values.""" - if click.confirm('Are you sure you want to reset the configuration to default values?'): - save_config(Configuration.default()) - click.echo(f"Configuration reset to default values at {CONFIG_FILE_PATH}") - - -# collages -@cli.group('collages') -def collages(): - """Possible operations with site collages.""" - - -# collages update -@collages.command('update') -@click.pass_context -@click.option( - '--fetch-mode', '-fm', - type=click.Choice(['torrent_name', 'query']), - default='torrent_name', - show_default=True, - help=( - '(Optional) Album lookup strategy:\n' - '\n- torrent_name: uses torrent dir name to search in Plex, ' - 'if you don\'t use Beets/Lidarr \n' - '\n- query: uses queries to Plex instead of searching by path name ' - '(if you use Beets/Lidarr)\n' - ) -) -def update_collages(ctx, fetch_mode: str): - """Synchronize all stored collections with their source collages.""" - fetch_mode = map_fetch_mode(fetch_mode) - try: - local_database = ctx.obj.get('db', None) - local_database: LocalDatabase - all_collages = local_database.get_all_collage_collections() - - if not all_collages: - click.echo("No collages found in the db.") - return - - # Initialize PlexManager once, populate its db once - plex_manager = PlexManager(local_database) - if not plex_manager: - return - plex_manager.populate_album_table() - - update_collections_from_collages( - local_database=local_database, - collage_list=all_collages, - plex_manager=plex_manager, - fetch_bookmarks=False, - fetch_mode=fetch_mode) - except Exception as exc: # pylint: disable=W0718 - logger.exception('Failed to update stored collections: %s', exc) - click.echo(f"An error occurred while updating stored collections: {exc}") - - -# convert collection -@collages.command('convert') -@click.argument('collage_ids', nargs=-1) -@click.option('--site', '-s', - type=click.Choice(['red', 'ops']), - required=True, - help='Specify the site: red (Redacted) or ops (Orpheus).') -@click.option( - '--fetch-mode', '-fm', - type=click.Choice(['torrent_name', 'query'], case_sensitive=False), # Added case_sensitive - default='torrent_name', - show_default=True, - help=( - '(Optional) Album lookup strategy:\n' - '\n- torrent_name: uses torrent dir name (original behavior).\n' - '\n- query: uses Plex queries (Beets/Lidarr friendly).\n' - ) -) -@click.pass_context -def convert_collages(ctx, collage_ids, site, fetch_mode): - """ - Create/Update Plex collections from given COLLAGE_IDS. - """ - if not collage_ids: - click.echo("Please provide at least one COLLAGE_ID.") - ctx.exit(1) # Exit with an error code - - album_fetch_mode_enum = map_fetch_mode(fetch_mode) - - # --- Dependency Setup --- - local_database = ctx.obj.get('db') - if not local_database: - click.echo("Error: Database not initialized.", err=True) - ctx.exit(1) - - plex_manager, gazelle_api = None, None - try: - plex_manager = PlexManager(db=local_database) - gazelle_api = GazelleAPI(site) - except Exception as e: # pylint: disable=W0718 - logger.error("Failed to initialize dependencies: %s", e, exc_info=True) - click.echo(f"Error: Failed to initialize dependencies - {e}", err=True) - ctx.exit(1) - - # --- Service Instantiation and Execution --- - processor = CollectionProcessingService(local_database, plex_manager, gazelle_api) - - # Call the service, passing the necessary functions from click - processor.process_collages( - collage_ids=collage_ids, - album_fetch_mode=album_fetch_mode_enum, - echo_func=click.echo, - confirm_func=click.confirm # Pass the actual click.confirm - ) - - click.echo("Processing finished.") - - -# bookmarks -@cli.group() -def bookmarks(): - """Possible operations with your site bookmarks.""" - - -# bookmarks update -@bookmarks.command('update') -@click.pass_context -@click.option( - '--fetch-mode', '-fm', - type=click.Choice(['torrent_name', 'query']), - default='torrent_name', - show_default=True, - help=( - '(Optional) Album lookup strategy:\n' - '\n- torrent_name: uses torrent dir name to search in Plex, ' - 'if you don\'t use Beets/Lidarr \n' - '\n- query: uses queries to Plex instead of searching by path name ' - '(if you use Beets/Lidarr)\n' - ) -) -def update_bookmarks_collection(ctx, fetch_mode: str): - """Synchronize all stored bookmarks with their source collages.""" - fetch_mode = map_fetch_mode(fetch_mode) - try: - local_database = ctx.obj.get('db', None) - local_database: LocalDatabase - all_bookmarks = local_database.get_all_bookmark_collections() - - if not all_bookmarks: - click.echo("No bookmarks found in the db.") - return - - plex_manager = PlexManager(local_database) - if not plex_manager: - return - plex_manager.populate_album_table() - - update_collections_from_collages( - local_database, - all_bookmarks, - plex_manager, - fetch_bookmarks=True) - - except Exception as exc: # pylint: disable=W0718 - logger.exception('Failed to update stored bookmarks: %s', exc) - click.echo(f"An error occurred while updating stored bookmarks: {exc}") - - -# bookmarks convert -@bookmarks.command('convert') -@click.option('--site', '-s', - type=click.Choice(['red', 'ops'], case_sensitive=False), - required=True, - help='Specify the site: red (Redacted) or ops (Orpheus).') -@click.option( - '--fetch-mode', '-fm', - type=click.Choice(['torrent_name', 'query'], case_sensitive=False), - default='torrent_name', - show_default=True, - help=( - '(Optional) Album lookup strategy:\n' - '\n- torrent_name: uses torrent dir name (original behavior).\n' - '\n- query: uses Plex queries (Beets/Lidarr friendly).\n' - ) -) -@click.pass_context -def convert_collection_from_bookmarks(ctx, site: str, fetch_mode: str): - """ - Create/Update a Plex collection based on your site bookmarks. - """ - album_fetch_mode_enum = map_fetch_mode(fetch_mode) - - # --- Dependency Setup --- - local_database = ctx.obj.get('db') - if not local_database: - click.echo("Error: Database not initialized.", err=True) - ctx.exit(1) - - plex_manager, gazelle_api = None, None - try: - plex_manager = PlexManager(db=local_database) - gazelle_api = GazelleAPI(site) # Create GazelleAPI based on site - except Exception as e: # pylint: disable=W0718 - logger.error("Failed to initialize dependencies: %s", e, exc_info=True) - click.echo(f"Error: Failed to initialize dependencies - {e}", err=True) - ctx.exit(1) - - # --- Service Instantiation and Execution --- - processor = CollectionProcessingService(local_database, plex_manager, gazelle_api) - - try: - # Call the specific bookmark processing method - processor.process_bookmarks( - album_fetch_mode=album_fetch_mode_enum, - echo_func=click.echo, - confirm_func=click.confirm - ) - except Exception as exc: # pylint: disable=W0718 - logger.exception( - 'Failed to create collection from bookmarks on site %s: %s', - site.upper(), exc - ) - click.echo( - f'Failed to create collection from bookmarks on site {site.upper()}: {exc}', - err=True - ) - ctx.exit(1) - - click.echo("Bookmark processing finished.") - - -# db -@cli.group() -def db(): - """Manage database.""" - - -# db location -@db.command('location') -@click.pass_context -def db_location(ctx): - """Returns the location to the database.""" - local_database = ctx.obj.get('db', None) - local_database: LocalDatabase - db_path = local_database.db_path - if os.path.exists(db_path): - click.echo(f"Database exists at: {db_path}") - else: - click.echo("Database file does not exist.") - - -# db albums -@db.group('albums') -def db_albums(): - """Manage albums inside database.""" - - -# db albums reset -@db_albums.command('reset') -@click.pass_context -def db_albums_reset(ctx): - """Resets albums table from database.""" - if click.confirm('Are you sure you want to reset the db?'): - try: - local_database = ctx.obj.get('db', None) - local_database: LocalDatabase - local_database.reset_albums() - click.echo("Albums table has been reset successfully.") - except Exception as exc: # pylint: disable=W0718 - click.echo(f"An error occurred while resetting the album table: {exc}") - - -@db_albums.command('update') -@click.pass_context -def db_albums_update(ctx): - """Updates albums table from Plex.""" - try: - local_database = ctx.obj.get('db', None) - local_database: LocalDatabase - plex_manager = PlexManager(db=local_database) - plex_manager.populate_album_table() - click.echo("Albums table has been updated successfully.") - except Exception as exc: # pylint: disable=W0703 - click.echo(f"An error occurred while updating the album table: {exc}") - - -# db collections -@db.group('collections') -def db_collections(): - """Manage albums inside database.""" - - -# db collections reset -@db_collections.command('reset') -@click.pass_context -def db_collections_reset(ctx): - """Resets collections table from database.""" - if click.confirm('Are you sure you want to reset the collection db?'): - try: - local_database = ctx.obj.get('db', None) - local_database: LocalDatabase - local_database.reset_collage_collections() - click.echo("Collage collection db has been reset successfully.") - except Exception as exc: # pylint: disable=W0718 - logger.exception('Failed to reset collage collection db: %s', exc) - click.echo( - f"An error occurred while resetting the collage collection db: {exc}") - - -# db bookmarks -@db.group('bookmarks') -def db_bookmarks(): - """Manage bookmarks inside database.""" - - -# db bookmarks reset -@db_bookmarks.command('reset') -@click.pass_context -def db_bookmarks_reset(ctx): - """Resets bookmarks table from database.""" - if click.confirm('Are you sure you want to reset the collection bookmarks db?'): - try: - local_database = ctx.obj.get('db', None) - local_database: LocalDatabase - local_database.reset_bookmark_collections() - click.echo("Collection bookmarks db has been reset successfully.") - except Exception as exc: # pylint: disable=W0718 - logger.exception('Failed to reset collection bookmarks db: %s', exc) - click.echo(f"An error occurred while resetting the collection bookmarks db: {exc}") - - -def update_collections_from_collages(local_database: LocalDatabase, - collage_list: List[Collection], - plex_manager: PlexManager, - fetch_bookmarks=False, - fetch_mode: AlbumFetchMode = AlbumFetchMode.TORRENT_NAME): - """ - Forces the update of each collage (force_update=True) - """ - - for collage in collage_list: - logger.info('Updating collection for collage "%s"...', collage.name) - gazelle_api = GazelleAPI(collage.site) - - if AlbumFetchMode.TORRENT_NAME == fetch_mode: - collection_creator = TorrentNameCollectionCreatorUseCase(local_database, - plex_manager, - gazelle_api) - result = collection_creator.execute( - collage_id=collage.external_id, - site=collage.site, - fetch_bookmarks=fetch_bookmarks, - force_update=True - ) - else: - collection_creator = QuerySyncCollectionUseCase(local_database, - plex_manager, - gazelle_api) - result = collection_creator.execute( - collage_id=collage.external_id, - site=collage.site, - fetch_bookmarks=fetch_bookmarks, - force_update=True - ) - - if result.response_status is None: - logger.info('No valid data found for collage "%s".', collage.name) - else: - logger.info('Collection for collage "%s" created/updated successfully with %s entries.', - collage.name, len(result.albums)) - - -# gui -@cli.command() -@click.option('--host', '-h', default='127.0.0.1', help='Host to bind to (default: 127.0.0.1)') -@click.option('--port', '-p', default=8000, type=int, help='Port to bind to (default: 8000)') -@click.option('--debug', is_flag=True, help='Run in debug mode with auto-reloading.') -def gui(host, port, debug): - """Launches the web interface using the Gunicorn production server.""" - - # Build the bind address for Gunicorn - bind_address = f"{host}:{port}" - - click.echo(f"🚀 Launching Gunicorn server at http://{bind_address}") - - # Create the list of arguments for the Gunicorn command - command = [ - 'gunicorn', - '--worker-class', 'eventlet', - '-w', '1', - '--bind', bind_address, - ] - - # If the --debug flag is passed, add Gunicorn's reload option - if debug: - click.echo(" -> Debug mode on (auto-reload enabled).") - command.append('--reload') - - command.append('wsgi:app') - - try: - # Execute the Gunicorn command - subprocess.run(command, check=True) - except FileNotFoundError: - click.echo( - "Error: 'gunicorn' command not found.", - err=True - ) - click.echo( - "Please make sure you have installed it with: pip install gunicorn eventlet", - err=True - ) - sys.exit(1) +# Add all command groups +cli.add_command(config) +cli.add_command(collages) +cli.add_command(bookmarks) +cli.add_command(db) +cli.add_command(gui) +cli.add_command(extras) @cli.result_callback() @@ -499,13 +39,6 @@ def finalize_cli(ctx, _result, *_args, **_kwargs): local_database.close() -def map_fetch_mode(fetch_mode) -> AlbumFetchMode: - """Map the fetch mode string to an AlbumFetchMode enum.""" - if fetch_mode == 'query': - return AlbumFetchMode.QUERY - return AlbumFetchMode.TORRENT_NAME - - def main(): """Actual entry point for the CLI when installed.""" if 'gui' not in sys.argv: diff --git a/red_plex/infrastructure/cli/commands/bookmarks.py b/red_plex/infrastructure/cli/commands/bookmarks.py new file mode 100644 index 0000000..3781c3d --- /dev/null +++ b/red_plex/infrastructure/cli/commands/bookmarks.py @@ -0,0 +1,124 @@ +"""Bookmark management CLI commands.""" + +import click + +from red_plex.infrastructure.cli.utils import update_collections_from_collages, map_fetch_mode +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.logger.logger import logger +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.infrastructure.service.collection_processor import CollectionProcessingService + + +@click.group() +def bookmarks(): + """Possible operations with your site bookmarks.""" + + +@bookmarks.command('update') +@click.pass_context +@click.option( + '--fetch-mode', '-fm', + type=click.Choice(['torrent_name', 'query']), + default='torrent_name', + show_default=True, + help=( + '(Optional) Album lookup strategy:\n' + '\n- torrent_name: uses torrent dir name to search in Plex, ' + 'if you don\'t use Beets/Lidarr \n' + '\n- query: uses queries to Plex instead of searching by path name ' + '(if you use Beets/Lidarr)\n' + ) +) +# pylint: disable=R0801 +def update_bookmarks_collection(ctx, fetch_mode: str): + """Synchronize all stored bookmarks with their source collages.""" + # Import here to avoid circular imports with cli.py + + fetch_mode = map_fetch_mode(fetch_mode) + try: + local_database = ctx.obj.get('db', None) + local_database: LocalDatabase + all_bookmarks = local_database.get_all_bookmark_collections() + + if not all_bookmarks: + click.echo("No bookmarks found in the db.") + return + + plex_manager = PlexManager(local_database) + if not plex_manager: + return + plex_manager.populate_album_table() + + update_collections_from_collages( + local_database, + all_bookmarks, + plex_manager, + fetch_bookmarks=True) + + except Exception as exc: # pylint: disable=W0718 + logger.exception('Failed to update stored bookmarks: %s', exc) + click.echo(f"An error occurred while updating stored bookmarks: {exc}") + + +@bookmarks.command('convert') +@click.option('--site', '-s', + type=click.Choice(['red', 'ops'], case_sensitive=False), + required=True, + help='Specify the site: red (Redacted) or ops (Orpheus).') +@click.option( + '--fetch-mode', '-fm', + type=click.Choice(['torrent_name', 'query'], case_sensitive=False), + default='torrent_name', + show_default=True, + help=( + '(Optional) Album lookup strategy:\n' + '\n- torrent_name: uses torrent dir name (original behavior).\n' + '\n- query: uses Plex queries (Beets/Lidarr friendly).\n' + ) +) +@click.pass_context +# pylint: disable=R0801 +def convert_collection_from_bookmarks(ctx, site: str, fetch_mode: str): + """ + Create/Update a Plex collection based on your site bookmarks. + """ + album_fetch_mode_enum = map_fetch_mode(fetch_mode) + + # --- Dependency Setup --- + local_database = ctx.obj.get('db') + if not local_database: + click.echo("Error: Database not initialized.", err=True) + ctx.exit(1) + + plex_manager, gazelle_api = None, None + try: + plex_manager = PlexManager(db=local_database) + gazelle_api = GazelleAPI(site) # Create GazelleAPI based on site + except Exception as e: # pylint: disable=W0718 + logger.error("Failed to initialize dependencies: %s", e, exc_info=True) + click.echo(f"Error: Failed to initialize dependencies - {e}", err=True) + ctx.exit(1) + + # --- Service Instantiation and Execution --- + processor = CollectionProcessingService(local_database, plex_manager, gazelle_api) + + try: + # Call the specific bookmark processing method + processor.process_bookmarks( + album_fetch_mode=album_fetch_mode_enum, + echo_func=click.echo, + confirm_func=click.confirm + ) + except Exception as exc: # pylint: disable=W0718 + logger.exception( + 'Failed to create collection from bookmarks on site %s: %s', + site.upper(), exc + ) + click.echo( + f'Failed to create collection from bookmarks on site {site.upper()}: {exc}', + err=True + ) + ctx.exit(1) + + click.echo("Bookmark processing finished.") diff --git a/red_plex/infrastructure/cli/commands/collages.py b/red_plex/infrastructure/cli/commands/collages.py new file mode 100644 index 0000000..4265e0e --- /dev/null +++ b/red_plex/infrastructure/cli/commands/collages.py @@ -0,0 +1,173 @@ +"""Collage management CLI commands.""" + +import click + +from red_plex.infrastructure.cli.utils import (update_collections_from_collages, + map_fetch_mode, + push_collections_to_upstream) +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.logger.logger import logger +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.infrastructure.service.collection_processor import CollectionProcessingService + + +@click.group('collages') +def collages(): + """Possible operations with site collages.""" + + +@collages.command('update') +@click.pass_context +@click.argument('collage_ids', nargs=-1) +@click.option( + '--fetch-mode', '-fm', + type=click.Choice(['torrent_name', 'query']), + default='torrent_name', + show_default=True, + help=( + '(Optional) Album lookup strategy:\n' + '\n- torrent_name: uses torrent dir name to search in Plex, ' + 'if you don\'t use Beets/Lidarr \n' + '\n- query: uses queries to Plex instead of searching by path name ' + '(if you use Beets/Lidarr)\n' + ) +) +@click.option( + '--push', '--update-upstream', + is_flag=True, + default=False, + help='Push local collection changes back to upstream collages on the site' +) +# pylint: disable=R0912 +def update_collages(ctx, collage_ids, fetch_mode: str, push: bool): + """ + Synchronize stored collections with their source collages. + + If COLLAGE_IDS are provided, only those collages will be processed. + If no COLLAGE_IDS are provided, all stored collages will be processed. + """ + # Import here to avoid circular imports with cli.py + + fetch_mode = map_fetch_mode(fetch_mode) + try: + local_database = ctx.obj.get('db', None) + local_database: LocalDatabase + + if collage_ids: + # Filter to only the specified collage IDs + all_collages = local_database.get_all_collage_collections() + collage_ids_set = set(collage_ids) + filtered_collages = [c for c in all_collages if c.external_id in collage_ids_set] + + if not filtered_collages: + click.echo(f"No collages found in the database with IDs: {', '.join(collage_ids)}") + return + + # Check if any requested IDs were not found + found_ids = {c.external_id for c in filtered_collages} + missing_ids = collage_ids_set - found_ids + if missing_ids: + click.echo(f"Warning: Collage IDs not found in database: {', '.join(missing_ids)}") + + target_collages = filtered_collages + else: + # Process all collages + target_collages = local_database.get_all_collage_collections() + + if not target_collages: + click.echo("No collages found to process.") + return + + # Initialize PlexManager once, populate its db once + plex_manager = PlexManager(local_database) + if not plex_manager: + return + plex_manager.populate_album_table() + + if push: + # Push mode: sync local collections to upstream + logger.info("Pushing local collection updates to upstream collages...") + if collage_ids: + logger.info("Processing specific collages: %s", + ', '.join(c.name for c in target_collages)) + success = push_collections_to_upstream( + local_database=local_database, + collage_list=target_collages, + plex_manager=plex_manager + ) + if success: + logger.info("All collections successfully synced to upstream.") + else: + logger.info("Some collections failed to sync. Check logs for details.") + else: + if collage_ids: + click.echo(f"Updating specific collages: " + f"{', '.join(c.name for c in target_collages)}") + update_collections_from_collages( + local_database=local_database, + collage_list=target_collages, + plex_manager=plex_manager, + fetch_bookmarks=False, + fetch_mode=fetch_mode) + + except Exception as exc: # pylint: disable=W0718 + logger.exception('Failed to update stored collections: %s', exc) + click.echo(f"An error occurred while updating stored collections: {exc}") + + +@collages.command('convert') +@click.argument('collage_ids', nargs=-1) +@click.option('--site', '-s', + type=click.Choice(['red', 'ops']), + required=True, + help='Specify the site: red (Redacted) or ops (Orpheus).') +@click.option( + '--fetch-mode', '-fm', + type=click.Choice(['torrent_name', 'query'], case_sensitive=False), # Added case_sensitive + default='torrent_name', + show_default=True, + help=( + '(Optional) Album lookup strategy:\n' + '\n- torrent_name: uses torrent dir name (original behavior).\n' + '\n- query: uses Plex queries (Beets/Lidarr friendly).\n' + ) +) +@click.pass_context +def convert_collages(ctx, collage_ids, site, fetch_mode): + """ + Create/Update Plex collections from given COLLAGE_IDS. + """ + if not collage_ids: + click.echo("Please provide at least one COLLAGE_ID.") + ctx.exit(1) # Exit with an error code + + album_fetch_mode_enum = map_fetch_mode(fetch_mode) + + # --- Dependency Setup --- + local_database = ctx.obj.get('db') + if not local_database: + click.echo("Error: Database not initialized.", err=True) + ctx.exit(1) + + plex_manager, gazelle_api = None, None + try: + plex_manager = PlexManager(db=local_database) + gazelle_api = GazelleAPI(site) + except Exception as e: # pylint: disable=W0718 + logger.error("Failed to initialize dependencies: %s", e, exc_info=True) + click.echo(f"Error: Failed to initialize dependencies - {e}", err=True) + ctx.exit(1) + + # --- Service Instantiation and Execution --- + processor = CollectionProcessingService(local_database, plex_manager, gazelle_api) + + # Call the service, passing the necessary functions from click + processor.process_collages( + collage_ids=collage_ids, + album_fetch_mode=album_fetch_mode_enum, + echo_func=click.echo, + confirm_func=click.confirm # Pass the actual click.confirm + ) + + click.echo("Processing finished.") diff --git a/red_plex/infrastructure/cli/commands/config.py b/red_plex/infrastructure/cli/commands/config.py new file mode 100644 index 0000000..7d7cda8 --- /dev/null +++ b/red_plex/infrastructure/cli/commands/config.py @@ -0,0 +1,61 @@ +"""Configuration management CLI commands.""" + +import os +import subprocess + +import click +import yaml + +from red_plex.infrastructure.config.config import ( + CONFIG_FILE_PATH, + load_config, + save_config, + ensure_config_exists +) +from red_plex.infrastructure.config.models import Configuration +from red_plex.infrastructure.logger.logger import logger + + +@click.group() +def config(): + """View or edit configuration settings.""" + + +@config.command('show') +def show_config(): + """Display the current configuration.""" + config_data = load_config() + path_with_config = ( + f"Configuration path: {CONFIG_FILE_PATH}\n\n" + + yaml.dump(config_data.to_dict(), default_flow_style=False) + ) + click.echo(path_with_config) + + +@config.command('edit') +def edit_config(): + """Open the configuration file in the default editor.""" + # Ensure the configuration file exists + ensure_config_exists() + + # Default to 'nano' if EDITOR is not set + editor = os.environ.get('EDITOR', 'notepad' if os.name == 'nt' else 'nano') + click.echo(f"Opening config file at {CONFIG_FILE_PATH}...") + try: + subprocess.call([editor, CONFIG_FILE_PATH]) + except FileNotFoundError: + message = f"Editor '{editor}' not found. \ + Please set the EDITOR environment variable to a valid editor." + logger.error(message) + click.echo(message) + except Exception as exc: # pylint: disable=W0718 + logger.exception('Failed to open editor: %s', exc) + click.echo(f"An error occurred while opening the editor: {exc}") + + +@config.command('reset') +def reset_config(): + """Reset the configuration to default values.""" + if click.confirm('Are you sure you want to reset the configuration to default values?'): + save_config(Configuration.default()) + click.echo(f"Configuration reset to default values at {CONFIG_FILE_PATH}") diff --git a/red_plex/infrastructure/cli/commands/db.py b/red_plex/infrastructure/cli/commands/db.py new file mode 100644 index 0000000..511304d --- /dev/null +++ b/red_plex/infrastructure/cli/commands/db.py @@ -0,0 +1,159 @@ +"""Database management CLI commands.""" + +import os + +import click + +from red_plex.infrastructure.cli.utils import get_database_from_context, create_plex_manager +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.logger.logger import logger +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.use_case.site_tags.site_tags_use_case import SiteTagsUseCase + + +@click.group() +def db(): + """Manage database.""" + + +@db.command('location') +@click.pass_context +def db_location(ctx): + """Returns the location to the database.""" + local_database = ctx.obj.get('db', None) + local_database: LocalDatabase + db_path = local_database.db_path + if os.path.exists(db_path): + click.echo(f"Database exists at: {db_path}") + else: + click.echo("Database file does not exist.") + + +@db.group('albums') +def db_albums(): + """Manage albums inside database.""" + + +@db_albums.command('reset') +@click.pass_context +def db_albums_reset(ctx): + """Resets albums table from database.""" + if click.confirm('Are you sure you want to reset the db?'): + try: + local_database = ctx.obj.get('db', None) + local_database: LocalDatabase + local_database.reset_albums() + click.echo("Albums table has been reset successfully.") + except Exception as exc: # pylint: disable=W0718 + click.echo(f"An error occurred while resetting the album table: {exc}") + + +@db_albums.command('update') +@click.pass_context +def db_albums_update(ctx): + """Updates albums table from Plex.""" + try: + local_database = ctx.obj.get('db', None) + local_database: LocalDatabase + plex_manager = create_plex_manager(local_database) + plex_manager.populate_album_table() + click.echo("Albums table has been updated successfully.") + except Exception as exc: # pylint: disable=W0703 + click.echo(f"An error occurred while updating the album table: {exc}") + + +@db.group('collections') +def db_collections(): + """Manage albums inside database.""" + + +@db_collections.command('reset') +@click.pass_context +def db_collections_reset(ctx): + """Resets collections table from database.""" + if click.confirm('Are you sure you want to reset the collection db?'): + try: + local_database = ctx.obj.get('db', None) + local_database: LocalDatabase + local_database.reset_collage_collections() + click.echo("Collage collection db has been reset successfully.") + except Exception as exc: # pylint: disable=W0718 + logger.exception('Failed to reset collage collection db: %s', exc) + click.echo( + f"An error occurred while resetting the collage collection db: {exc}") + + +@db.group('bookmarks') +def db_bookmarks(): + """Manage bookmarks inside database.""" + + +@db_bookmarks.command('reset') +@click.pass_context +def db_bookmarks_reset(ctx): + """Resets bookmarks table from database.""" + if click.confirm('Are you sure you want to reset the collection bookmarks db?'): + try: + local_database = ctx.obj.get('db', None) + local_database: LocalDatabase + local_database.reset_bookmark_collections() + click.echo("Collection bookmarks db has been reset successfully.") + except Exception as exc: # pylint: disable=W0718 + logger.exception('Failed to reset collection bookmarks db: %s', exc) + click.echo(f"An error occurred while resetting the collection bookmarks db: {exc}") + + +@db.group('remote-mappings') +def db_remote_mappings(): + """Manage core remote mappings between Plex items and site group IDs.""" + + +@db_remote_mappings.command('scan') +@click.option('--site', '-s', + type=click.Choice(['red', 'ops'], case_sensitive=False), + required=True, + help='Specify the site: red (Redacted) or ops (Orpheus).') +@click.option('--always-skip', is_flag=True, help='Always skip albums with multiple matches.') +@click.pass_context +def scan_albums(ctx, site: str, always_skip: bool): + """ + Scan albums and create remote mappings by searching filenames on the site. + This is an incremental process - only unscanned albums will be processed. + """ + try: + # Get dependencies from context using shared utilities + local_database = get_database_from_context(ctx) + plex_manager = create_plex_manager(local_database) + gazelle_api = GazelleAPI(site) + + # Ensure albums table is populated + click.echo("Updating album database from Plex...") + plex_manager.populate_album_table() + + # Create use case and execute scan + site_tags_use_case = SiteTagsUseCase(local_database, plex_manager, gazelle_api) + site_tags_use_case.scan_albums_for_site_tags( + echo_func=click.echo, + confirm_func=click.confirm, + always_skip=always_skip + ) + + except Exception as e: # pylint: disable=W0703 + logger.exception("Error during album scan: %s", e) + click.echo(f"Error during album scan: {e}", err=True) + ctx.exit(1) + + +@db_remote_mappings.command('reset') +@click.pass_context +def reset_remote_mappings(ctx): + """Reset remote mappings. Use with caution!""" + if click.confirm('Are you sure you want to reset remote mappings?'): + try: + local_database = get_database_from_context(ctx) + local_database.reset_tag_mappings() + click.echo("Remote mappings have been reset successfully.") + except Exception as e: # pylint: disable=W0703 + logger.exception("Error resetting remote mappings: %s", e) + click.echo(f"Error resetting remote mappings: {e}", err=True) + ctx.exit(1) diff --git a/red_plex/infrastructure/cli/commands/extras.py b/red_plex/infrastructure/cli/commands/extras.py new file mode 100644 index 0000000..ab83e7c --- /dev/null +++ b/red_plex/infrastructure/cli/commands/extras.py @@ -0,0 +1,61 @@ +"""Extra CLI commands for advanced features.""" + +import click + +from red_plex.infrastructure.logger.logger import logger +from red_plex.use_case.site_tags.site_tags_use_case import SiteTagsUseCase +from red_plex.infrastructure.cli.utils import get_database_from_context, create_plex_manager + + +@click.group() +def extras(): + """Extra features and advanced functionality.""" + +@extras.group('site-tags') +def site_tags(): + """Site tags functionality for creating collections from tagged albums.""" + + +@site_tags.command('convert') +@click.option('--tags', '-t', + required=True, + help='Comma-separated list of tags to filter by.') +@click.option('--collection-name', '-n', + required=True, + help='Name for the Plex collection to create/update.') +@click.pass_context +def convert_tags_to_collection(ctx, tags: str, collection_name: str): + """ + Create a Plex collection from albums matching the specified tags. + """ + try: + # Parse tags + tag_list = [tag.strip() for tag in tags.split(',') if tag.strip()] + if not tag_list: + click.echo("Error: No valid tags provided.", err=True) + ctx.exit(1) + + # Get dependencies from context using shared utility + local_database = get_database_from_context(ctx) + plex_manager = create_plex_manager(local_database) + + # Create the use case and execute conversion + # No need of gazelle_api here since we're using the local database + site_tags_use_case = SiteTagsUseCase(local_database=local_database, + plex_manager=plex_manager) + success = site_tags_use_case.create_collection_from_tags( + tags=tag_list, + collection_name=collection_name, + echo_func=click.echo + ) + + if success: + click.echo("Collection creation completed successfully.") + else: + click.echo("Collection creation failed.", err=True) + ctx.exit(1) + + except Exception as e: # pylint: disable=W0703 + logger.exception("Error during collection creation: %s", e) + click.echo(f"Error during collection creation: {e}", err=True) + ctx.exit(1) diff --git a/red_plex/infrastructure/cli/commands/gui.py b/red_plex/infrastructure/cli/commands/gui.py new file mode 100644 index 0000000..676e43d --- /dev/null +++ b/red_plex/infrastructure/cli/commands/gui.py @@ -0,0 +1,42 @@ +"""GUI launcher CLI command.""" +import subprocess +import sys + +import click + +from red_plex.infrastructure.logger.logger import logger + + +@click.command() +@click.option('--host', '-h', default='127.0.0.1', help='Host to bind to (default: 127.0.0.1)') +@click.option('--port', '-p', default=8000, type=int, help='Port to bind to (default: 8000)') +@click.option('--debug', is_flag=True, help='Run in debug mode with auto-reloading.') +def gui(host, port, debug): + """Launches the web interface using the Gunicorn production server.""" + + # Build the bind address for Gunicorn + bind_address = f"{host}:{port}" + + click.echo(f"🚀 Launching Gunicorn server at http://{bind_address}") + + # Create the list of arguments for the Gunicorn command + command = [ + 'gunicorn', + '--worker-class', 'eventlet', + '-w', '1', + '--bind', bind_address, + ] + + # If the --debug flag is passed, add Gunicorn's reload option + if debug: + logger.info("Debug mode enabled: auto-reloading will be active.") + command.append('--reload') + + command.append('wsgi:app') + + try: + # Execute the Gunicorn command + subprocess.run(command, check=True) + except FileNotFoundError: + logger.error("Gunicorn is not installed. Please install it with: pip install gunicorn") + sys.exit(1) diff --git a/red_plex/infrastructure/cli/utils.py b/red_plex/infrastructure/cli/utils.py new file mode 100644 index 0000000..6a7728e --- /dev/null +++ b/red_plex/infrastructure/cli/utils.py @@ -0,0 +1,195 @@ +"""Shared utilities for CLI commands.""" +from typing import List + +import click + +from red_plex.domain.models import Collection +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.logger.logger import logger +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.use_case.create_collection.album_fetch_mode import AlbumFetchMode +from red_plex.use_case.create_collection.query.query_sync_collection import ( + QuerySyncCollectionUseCase) +from red_plex.use_case.create_collection.torrent_name.torrent_name_sync_collection import ( + TorrentNameCollectionCreatorUseCase) +from red_plex.use_case.upstream_sync.upstream_sync_use_case import UpstreamSyncUseCase + + +def get_database_from_context(ctx) -> LocalDatabase: + """ + Get database instance from Click context with error handling. + + Args: + ctx: Click context object + + Returns: + LocalDatabase instance + + Raises: + SystemExit: If database is not initialized + """ + local_database = ctx.obj.get('db') + if not local_database: + click.echo("Error: Database not initialized.", err=True) + ctx.exit(1) + return local_database + + +def create_plex_manager(local_database: LocalDatabase) -> PlexManager: + """ + Create a PlexManager instance. + + Args: + local_database: LocalDatabase instance + + Returns: + PlexManager instance + """ + return PlexManager(db=local_database) + + +def map_fetch_mode(fetch_mode: str) -> AlbumFetchMode: + """Map the fetch mode string to an AlbumFetchMode enum.""" + if fetch_mode == 'query': + return AlbumFetchMode.QUERY + return AlbumFetchMode.TORRENT_NAME + + +def update_collections_from_collages(local_database: LocalDatabase, + collage_list: List[Collection], + plex_manager: PlexManager, + fetch_bookmarks=False, + fetch_mode: AlbumFetchMode = AlbumFetchMode.TORRENT_NAME): + """ + Forces the update of each collage (force_update=True) + """ + + for collage in collage_list: + logger.info('Updating collection for collage "%s"...', collage.name) + gazelle_api = GazelleAPI(collage.site) + + if AlbumFetchMode.TORRENT_NAME == fetch_mode: + collection_creator = TorrentNameCollectionCreatorUseCase(local_database, + plex_manager, + gazelle_api) + result = collection_creator.execute( + collage_id=collage.external_id, + site=collage.site, + fetch_bookmarks=fetch_bookmarks, + force_update=True + ) + else: + collection_creator = QuerySyncCollectionUseCase(local_database, + plex_manager, + gazelle_api) + result = collection_creator.execute( + collage_id=collage.external_id, + site=collage.site, + fetch_bookmarks=fetch_bookmarks, + force_update=True + ) + + if result.response_status is None: + logger.info('No valid data found for collage "%s".', collage.name) + else: + logger.info('Collection for collage "%s" created/updated successfully with %s entries.', + collage.name, len(result.albums)) + + +# pylint: disable=R0914,W0718,R0912,R0915 +def push_collections_to_upstream(local_database: LocalDatabase, + collage_list: List[Collection], + plex_manager: PlexManager) -> bool: + """ + Push local collection updates back to upstream collages. + + Args: + local_database: Database instance + collage_list: List of collections to sync upstream + plex_manager: Plex manager instance + + Returns: + True if all syncs were successful, False otherwise + """ + # Initialize upstream sync use case + upstream_sync = UpstreamSyncUseCase(local_database, plex_manager) + + # Get preview of what would be synced + preview_response = upstream_sync.get_sync_preview(collage_list) + + if not preview_response.success: + logger.error('Failed to get sync preview: %s', preview_response.error_message) + return False + + if not preview_response.preview_data: + logger.info('No collages need syncing - all are up to date') + return True + + # Show confirmation dialog for each collage + selected_albums = {} + + for collage_preview in preview_response.preview_data: + click.echo( + f'\nCollage "{collage_preview.collage_name}" ({collage_preview.external_id}) ' + f'will have {len(collage_preview.albums_to_add)} new items added:') + + # Show albums with numbers + for i, album in enumerate(collage_preview.albums_to_add, 1): + click.echo(f' {i}. {album.display_name}') + + # Ask for album selection + click.echo('\nSpecify which albums to add (e.g., "1,3,4" for albums 1, 3, and 4)') + click.echo('Leave empty to add all albums, or "skip" to skip this collage:') + + selection = click.prompt('Albums to add', type=str, default='', show_default=False) + selection = selection.strip() + + if selection.lower() == 'skip': + click.echo('Skipping this collage.') + continue + + # Parse selection + selected_group_ids = [album.group_id + for album in collage_preview.albums_to_add] # Default to all + if selection: + try: + # Parse comma-separated numbers + selected_indices = [int(x.strip()) - 1 for x in selection.split(',') + if x.strip().isdigit()] + + # Validate indices + valid_indices = [i for i in selected_indices + if 0 <= i < len(collage_preview.albums_to_add)] + + if not valid_indices: + click.echo('No valid album numbers specified. Adding all albums.') + else: + selected_group_ids = [collage_preview.albums_to_add[i].group_id + for i in valid_indices] + click.echo(f'Selected {len(selected_group_ids)} album(s) to add.') + + except ValueError: + click.echo('Invalid selection format. Adding all albums.') + + selected_albums[collage_preview.collage_id] = selected_group_ids + + if not selected_albums: + logger.info('No albums selected for sync') + return True + + # Filter collages to only sync those with selected albums + collages_to_sync = [c for c in collage_list if c.id in selected_albums] + + # Perform the sync + sync_response = upstream_sync.sync_collections_upstream(collages_to_sync, selected_albums) + + # Log results + logger.info('Upstream sync completed: %d/%d collections synced successfully', + sync_response.synced_collages, sync_response.total_collages) + + if sync_response.errors: + for error in sync_response.errors: + logger.error(error) + + return sync_response.success diff --git a/red_plex/infrastructure/constants/constants.py b/red_plex/infrastructure/constants/constants.py new file mode 100644 index 0000000..f2a819a --- /dev/null +++ b/red_plex/infrastructure/constants/constants.py @@ -0,0 +1,12 @@ +""" This module stores constants""" + +ALBUM_TAGS = [ + "EP", "E.P", "E.P.", "Single", "Album", "Soundtrack", "Anthology", + "Compilation", "Live Album", "Remix", "Bootleg", "Interview", + "Mixtape", "Demo", "Concert Recording", "DJ Mix", "Original Mix", + "Remastered", "Deluxe Edition", "Limited Edition", "Bonus Track", + "Instrumental", "Acapella" +] + +VARIOUS_ARTISTS_TAGS = [ + "various artists", "va", "v.a", "v/a", "various"] diff --git a/red_plex/infrastructure/db/albums.py b/red_plex/infrastructure/db/albums.py new file mode 100644 index 0000000..6078fc6 --- /dev/null +++ b/red_plex/infrastructure/db/albums.py @@ -0,0 +1,215 @@ +"""Album database operations.""" + +from collections import defaultdict +from datetime import datetime +from typing import List, Optional + +from red_plex.domain.models import Album +from red_plex.infrastructure.logger.logger import logger + + +class AlbumDatabaseManager: + """Manages album-related database operations.""" + + def __init__(self, conn): + self.conn = conn + + def insert_or_update_album(self, album: Album) -> None: + """ + Insert or update an album, including its artists, in the database. + Uses a transaction to ensure atomicity. + """ + logger.debug("Inserting/updating album with ID %s", album.id) + with self.conn: + # 1. Insert or replace the album itself + self.conn.execute( + """ + INSERT OR REPLACE INTO albums(album_id, name, path, added_at) + VALUES (?, ?, ?, ?) + """, + (album.id, + album.name, + album.path, + album.added_at.isoformat() if album.added_at else None) + ) + + # 2. Delete old artist associations for this album + self.conn.execute("DELETE FROM album_artists WHERE album_id = ?", (album.id,)) + + # 3. Insert artists and their new associations + if album.artists: + # Filter out None and empty artist names + valid_artists = [artist + for artist in album.artists + if artist is not None and artist.strip()] + + if valid_artists: + # Insert artists if they don't exist (IGNORE on conflict) + self.conn.executemany( + "INSERT OR IGNORE INTO artists(artist_name) VALUES (?)", + [(artist,) for artist in valid_artists] + ) + + # Get artist IDs + cur = self.conn.cursor() + artist_ids = dict(cur.execute( + f"SELECT artist_name, artist_id FROM artists WHERE artist_name IN " + f"({','.join('?' * len(valid_artists))})", + valid_artists + )) + + # Insert into linking table + self.conn.executemany( + "INSERT INTO album_artists(album_id, artist_id) VALUES (?, ?)", + [(album.id, artist_ids[name]) for name in valid_artists] + ) + + def insert_albums_bulk(self, albums: List[Album]) -> None: + """ + Inserts or updates a list of albums in bulk using a single transaction. + Handles albums, artists, and their relationships efficiently. + """ + logger.debug("Inserting/updating %d albums in bulk.", len(albums)) + + album_rows = [] + all_artists = set() + album_id_to_artists = {} + + for album in albums: + album_rows.append(( + album.id, + album.name, + album.path, + album.added_at.isoformat() if album.added_at else None + )) + if album.artists: + # Filter out None and empty artist names + valid_artists = [artist + for artist in album.artists + if artist is not None and artist.strip()] + if valid_artists: + all_artists.update(valid_artists) + album_id_to_artists[album.id] = valid_artists + + with self.conn: + # 1. Insert/update all albums + self.conn.executemany( + "INSERT OR REPLACE INTO albums(album_id, name, path, added_at) VALUES (?, ?, ?, ?)", + album_rows + ) + + album_ids = [album.id for album in albums] + + # 2. Delete all existing artist links for the albums being updated + self.conn.execute( + f"DELETE FROM album_artists WHERE album_id IN ({','.join('?' * len(album_ids))})", + album_ids + ) + + # 3. Insert all new unique artists + if all_artists: + self.conn.executemany( + "INSERT OR IGNORE INTO artists(artist_name) VALUES (?)", + [(artist,) for artist in all_artists] + ) + + # 4. Get all required artist IDs in one query + cur = self.conn.cursor() + artist_name_to_id = dict(cur.execute( + f"SELECT artist_name, artist_id FROM artists WHERE artist_name IN " + f"({','.join('?' * len(all_artists))})", + list(all_artists) + )) + + # 5. Prepare and insert all album-artist links + album_artist_links = [] + for album_id, artist_names in album_id_to_artists.items(): + for name in artist_names: + if name in artist_name_to_id: + album_artist_links.append((album_id, artist_name_to_id[name])) + + self.conn.executemany( + "INSERT INTO album_artists(album_id, artist_id) VALUES (?, ?)", + album_artist_links + ) + + def get_album(self, album_id: str) -> Optional[Album]: + """ + Retrieve a single album by its ID, including its list of artists. + """ + cur = self.conn.cursor() + + # Get album details + cur.execute("SELECT album_id, name, path, added_at " + "FROM albums WHERE album_id = ?", (album_id,)) + row = cur.fetchone() + if not row: + return None + + _id, _name, _path, _added_at_str = row + added_at = datetime.fromisoformat(_added_at_str) if _added_at_str else None + + # Get associated artists + cur.execute(""" + SELECT ar.artist_name + FROM artists ar + JOIN album_artists aa ON ar.artist_id = aa.artist_id + WHERE aa.album_id = ? + """, (album_id,)) + artists = [row[0] for row in cur.fetchall()] + + return Album(id=_id, name=_name, path=_path, added_at=added_at, artists=artists) + + def get_all_albums(self) -> List[Album]: + """ + Retrieve all albums from the database, including their lists of artists. + This is done efficiently to avoid the N+1 query problem. + """ + cur = self.conn.cursor() + + # 1. Fetch all albums + cur.execute("SELECT album_id, name, path, added_at FROM albums") + album_rows = cur.fetchall() + if not album_rows: + return [] + + # 2. Fetch all artist relationships in a single query + artist_map = defaultdict(list) + cur.execute(""" + SELECT aa.album_id, ar.artist_name + FROM album_artists aa + JOIN artists ar ON aa.artist_id = ar.artist_id + """) + for album_id, artist_name in cur.fetchall(): + artist_map[album_id].append(artist_name) + + # 3. Construct Album objects + albums = [] + for _id, _name, _path, _added_at_str in album_rows: + added_at = datetime.fromisoformat(_added_at_str) if _added_at_str else None + albums.append( + Album(id=_id, name=_name, path=_path, added_at=added_at, artists=artist_map[_id]) + ) + return albums + + def delete_album(self, album_id: str) -> None: + """ + Delete an album. Associated artist links are removed automatically + due to 'ON DELETE CASCADE' in the foreign key constraint. + """ + logger.debug("Deleting album with ID %s", album_id) + with self.conn: + self.conn.execute("DELETE FROM albums WHERE album_id = ?", (album_id,)) + # Note: We might want to clean up orphan artists later, but for now, this is fine. + + def reset_albums(self): + """ + Deletes all records from 'albums', 'album_artists', and 'artists'. + """ + logger.info("Resetting albums: dropping and recreating tables.") + with self.conn: + self.conn.execute("DROP TABLE IF EXISTS album_artists") + self.conn.execute("DROP TABLE IF EXISTS artists") + self.conn.execute("DROP TABLE IF EXISTS albums") + # Note: We need to recreate tables elsewhere since this manager + # doesn't handle table creation diff --git a/red_plex/infrastructure/db/base.py b/red_plex/infrastructure/db/base.py new file mode 100644 index 0000000..d172c0c --- /dev/null +++ b/red_plex/infrastructure/db/base.py @@ -0,0 +1,135 @@ +"""Base database functionality for LocalDatabase.""" + +import os +import sqlite3 + +from red_plex.infrastructure.db.utils.csv_to_db_migrator import CsvToDbMigrator + + +#pylint: disable=R0903 +class BaseDatabaseManager: + """Base database manager with connection and table creation functionality.""" + + def __init__(self): + self.db_path = self._get_database_directory() + os.makedirs(self.db_path, exist_ok=True) + db_file_path = os.path.join(self.db_path, 'red_plex.db') + + # If the database file doesn't exist, create and run migrations + if not os.path.isfile(db_file_path): + # Create a temp connection for initialization + self.conn = sqlite3.connect(db_file_path) + self.conn.execute("PRAGMA journal_mode=WAL;") + # Migrate existing CSV data into the new DB + # This can be removed in future releases + migrator = CsvToDbMigrator(db_file_path=db_file_path) + migrator.migrate_from_csv_to_db() + self.conn.commit() + else: + self.conn = sqlite3.connect(db_file_path) + self.conn.execute("PRAGMA journal_mode=WAL;") + self._create_tables() + + def _create_tables(self): + """Create necessary tables if they do not exist.""" + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS albums ( + album_id TEXT PRIMARY KEY, + name TEXT, + path TEXT NOT NULL, + added_at TEXT + ); + """) + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS artists ( + artist_id INTEGER PRIMARY KEY AUTOINCREMENT, + artist_name TEXT NOT NULL UNIQUE + ); + """) + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS album_artists ( + album_id TEXT, + artist_id INTEGER, + PRIMARY KEY (album_id, artist_id), + FOREIGN KEY (album_id) REFERENCES albums(album_id) ON DELETE CASCADE, + FOREIGN KEY (artist_id) REFERENCES artists(artist_id) ON DELETE CASCADE + ); + """) + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS collage_collections ( + rating_key TEXT PRIMARY KEY, + name TEXT, + site TEXT, + external_id TEXT + ); + """) + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS bookmark_collections ( + rating_key TEXT PRIMARY KEY, + site TEXT + ); + """) + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS collection_torrent_groups ( + rating_key TEXT, + group_id INTEGER + ); + """) + + # Tables for site tag mappings + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS rating_key_group_id_mappings ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rating_key TEXT NOT NULL, + group_id INTEGER NOT NULL, + site TEXT NOT NULL, + UNIQUE(rating_key, group_id, site) + ); + """) + + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS site_tags ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + tag_name TEXT NOT NULL UNIQUE + ); + """) + + self.conn.execute(""" + CREATE TABLE IF NOT EXISTS mapping_tags ( + mapping_id INTEGER, + tag_id INTEGER, + PRIMARY KEY (mapping_id, tag_id), + FOREIGN KEY (mapping_id) REFERENCES rating_key_group_id_mappings(id) ON DELETE CASCADE, + FOREIGN KEY (tag_id) REFERENCES site_tags(id) ON DELETE CASCADE + ); + """) + + self.conn.execute("DROP TABLE IF EXISTS beets_mappings;") + + @staticmethod + def _get_database_directory(): + """ + Return the directory path where the database file should live, + based on each OS's convention for local application data. + """ + if os.name == 'nt': # Windows + # Typically, LOCALAPPDATA or APPDATA for user-level application data + return os.path.join( + os.getenv('LOCALAPPDATA', os.path.expanduser('~\\AppData\\Local')), + 'red-plex' + ) + try: + if os.uname().sysname == 'Darwin': # macOS + + # Commonly used for persistent data: + return os.path.join(os.path.expanduser('~/Library/Application Support'), 'red-plex') + except ImportError: + pass + + # Linux / other Unix: use ~/.local/share/red-plex by XDG spec + data_home = os.getenv('XDG_DATA_HOME', os.path.expanduser('~/.local/share')) + return os.path.join(data_home, 'red-plex') + + def close(self): + """Close the database connection.""" + self.conn.close() diff --git a/red_plex/infrastructure/db/collection.py b/red_plex/infrastructure/db/collection.py new file mode 100644 index 0000000..ba71f1e --- /dev/null +++ b/red_plex/infrastructure/db/collection.py @@ -0,0 +1,310 @@ +"""Collection database operations for collage and bookmark collections.""" + +from typing import List, Optional, Set + +from red_plex.domain.models import Collection, TorrentGroup +from red_plex.infrastructure.logger.logger import logger + + +class CollectionDatabaseManager: + """Manages collection-related database operations.""" + + def __init__(self, conn): + self.conn = conn + + def insert_or_update_collage_collection(self, coll: Collection) -> None: + """ + Insert or update a collage-based collection, along with its torrent groups. + We'll do an upsert in collage_collections, then remove all old group_ids + from collection_torrent_groups for that rating_key, and re-insert them. + """ + logger.debug("Inserting/updating collage collection with rating_key %s", coll.id) + self.conn.execute( + """ + INSERT OR REPLACE INTO collage_collections(rating_key, name, site, external_id) + VALUES (?, ?, ?, ?) + """, + (coll.id, coll.name, coll.site, coll.external_id) + ) + # Remove old group_ids for that rating_key + self.conn.execute( + "DELETE FROM collection_torrent_groups WHERE rating_key = ?", + (coll.id,) + ) + # Insert new group_ids + if coll.torrent_groups: + group_data = [(coll.id, tg.id) for tg in coll.torrent_groups] + self.conn.executemany( + "INSERT INTO collection_torrent_groups(rating_key, group_id) VALUES(?, ?)", + group_data + ) + self.conn.commit() + + def merge_torrent_groups_for_collage_collection(self, rating_key: str, + new_group_ids: Set[int]) -> None: + """ + Merges a new set of torrent group IDs with the existing ones for a collage collection. + + This method fetches the current torrent groups for a given collection, + merges them with the new set of group IDs provided, and updates the + database with the combined list, ensuring no duplicates. + + Args: + rating_key: The ID of the collage collection to update. + new_group_ids: A set of new torrent group IDs to merge. + """ + logger.debug( + "Merging torrent groups for collage collection with rating_key %s", + rating_key + ) + + # First, ensure the collage collection exists. + cur = self.conn.cursor() + cur.execute( + "SELECT 1 FROM collage_collections WHERE rating_key = ?", (rating_key,) + ) + if not cur.fetchone(): + logger.warning( + "Attempted to merge torrent groups for a non-existent collage collection: %s", + rating_key + ) + return + + # Get existing group IDs and form a merged set to handle duplicates. + existing_group_ids = set(self._get_torrent_group_ids_for(rating_key)) + merged_group_ids = existing_group_ids.union(new_group_ids) + + # If there's no change, we can skip the database write. + if merged_group_ids == existing_group_ids: + logger.debug( + "No new torrent groups to add for rating_key %s. Skipping update.", + rating_key + ) + return + + # Use a transaction to perform the delete and bulk insert atomically. + try: + # Remove all old group_ids for the rating_key + self.conn.execute( + "DELETE FROM collection_torrent_groups WHERE rating_key = ?", + (rating_key,) + ) + + # Insert the new merged set of group_ids + if merged_group_ids: + data_to_insert = [(rating_key, group_id) for group_id in merged_group_ids] + self.conn.executemany( + "INSERT INTO collection_torrent_groups(rating_key, group_id) VALUES(?, ?)", + data_to_insert + ) + + self.conn.commit() + logger.debug( + "Successfully merged and updated torrent groups for rating_key %s. " + "Total groups: %d", + rating_key, + len(merged_group_ids) + ) + except self.conn.Error as e: + self.conn.rollback() + logger.error( + "Database error while merging torrent groups for rating_key %s: %s", + rating_key, + e + ) + raise e + + def get_collage_collection(self, rating_key: str) -> Optional[Collection]: + """ + Retrieve a single collage-based collection (and associated group_ids) by rating_key. + Returns a Collection or None if not found. + """ + cur = self.conn.cursor() + # Get collage collection fields + cur.execute( + """ + SELECT rating_key, name, site, external_id + FROM collage_collections + WHERE rating_key = ? + """, + (rating_key,) + ) + row = cur.fetchone() + if not row: + return None + rating_key_val, name, site, external_id = row + # Get associated group_ids + group_ids = self._get_torrent_group_ids_for(rating_key_val) + return Collection( + id=rating_key_val, + external_id=external_id, + name=name, + torrent_groups=[TorrentGroup(id=gid) for gid in group_ids], + site=site + ) + + def get_all_collage_collections(self) -> List[Collection]: + """ + Retrieve all collage-based collections from the DB, + along with their torrent groups. + """ + cur = self.conn.cursor() + cur.execute("SELECT rating_key, name, site, external_id FROM collage_collections") + rows = cur.fetchall() + collections = [] + for (rk, name, site, external_id) in rows: + group_ids = self._get_torrent_group_ids_for(rk) + collections.append(Collection( + id=rk, + external_id=external_id, + name=name, + torrent_groups=[TorrentGroup(id=gid) for gid in group_ids], + site=site + )) + return collections + + def delete_collage_collection(self, rating_key: str) -> None: + """ + Delete a collage-based collection and associated torrent group mappings. + """ + logger.debug("Deleting collage collection with rating_key %s", rating_key) + self.conn.execute( + "DELETE FROM collage_collections WHERE rating_key = ?", + (rating_key,) + ) + self.conn.execute( + "DELETE FROM collection_torrent_groups WHERE rating_key = ?", + (rating_key,) + ) + self.conn.commit() + + def reset_collage_collections(self): + """ + Deletes all records from 'collage_collections' and + their associated rows in 'collection_torrent_groups'. + """ + logger.info("Resetting collage collections (deleting all rows in 'collage_collections').") + self.conn.execute("DELETE FROM collage_collections") + logger.info("Removing associated torrent groups for collage collections.") + self.conn.execute(""" + DELETE FROM collection_torrent_groups + WHERE rating_key NOT IN (SELECT rating_key FROM bookmark_collections) + """) + + self.conn.commit() + + def insert_or_update_bookmark_collection(self, coll: Collection) -> None: + """ + Insert or update a bookmark-based collection (in bookmark_collections), + then remove all old group_ids from 'collection_torrent_groups' for that rating_key + and re-insert them. + """ + logger.debug("Inserting/updating bookmark collection with rating_key %s", coll.id) + self.conn.execute( + """ + INSERT OR REPLACE INTO bookmark_collections(rating_key, site) + VALUES (?, ?) + """, + (coll.id, coll.site) + ) + # Remove old group_ids for that rating_key + self.conn.execute( + "DELETE FROM collection_torrent_groups WHERE rating_key = ?", + (coll.id,) + ) + # Insert new group_ids + if coll.torrent_groups: + group_data = [(coll.id, tg.id) for tg in coll.torrent_groups] + self.conn.executemany( + "INSERT INTO collection_torrent_groups(rating_key, group_id) VALUES(?, ?)", + group_data + ) + self.conn.commit() + + def get_bookmark_collection(self, rating_key: str) -> Optional[Collection]: + """ + Retrieve a single bookmark collection (plus group_ids) by rating_key. + Returns a Collection or None if not found. + """ + cur = self.conn.cursor() + cur.execute( + """ + SELECT rating_key, site + FROM bookmark_collections + WHERE rating_key = ? + """, + (rating_key,) + ) + row = cur.fetchone() + if not row: + return None + rating_key_val, site = row + group_ids = self._get_torrent_group_ids_for(rating_key_val) + # We can store the name as something like f"{site.upper()} Bookmarks" + return Collection( + id=rating_key_val, + name=f"{site.upper()} Bookmarks", + site=site, + torrent_groups=[TorrentGroup(id=gid) for gid in group_ids] + ) + + def get_all_bookmark_collections(self) -> List[Collection]: + """ + Retrieve all bookmark collections from the DB, + along with their torrent groups. + """ + cur = self.conn.cursor() + cur.execute("SELECT rating_key, site FROM bookmark_collections") + rows = cur.fetchall() + bookmarks = [] + for (rk, site) in rows: + group_ids = self._get_torrent_group_ids_for(rk) + bookmarks.append(Collection( + id=rk, + name=f"{site.upper()} Bookmarks", + site=site, + torrent_groups=[TorrentGroup(id=gid) for gid in group_ids] + )) + return bookmarks + + def delete_bookmark_collection(self, rating_key: str) -> None: + """ + Delete a bookmark-based collection and associated torrent group mappings. + """ + logger.debug("Deleting bookmark collection with rating_key %s", rating_key) + self.conn.execute( + "DELETE FROM bookmark_collections WHERE rating_key = ?", + (rating_key,) + ) + self.conn.execute( + "DELETE FROM collection_torrent_groups WHERE rating_key = ?", + (rating_key,) + ) + self.conn.commit() + + def reset_bookmark_collections(self): + """ + Deletes all records from 'bookmark_collections' and + their associated rows in 'collection_torrent_groups'. + """ + logger.info("Resetting bookmark collections (deleting all rows in 'bookmark_collections').") + self.conn.execute("DELETE FROM bookmark_collections") + logger.info("Removing associated torrent groups for bookmark collections.") + self.conn.execute(""" + DELETE FROM collection_torrent_groups + WHERE rating_key NOT IN (SELECT rating_key FROM collage_collections) + """) + + self.conn.commit() + + def _get_torrent_group_ids_for(self, rating_key: str) -> List[int]: + """ + Retrieve all group_ids for a given rating_key from collection_torrent_groups. + """ + cur = self.conn.cursor() + cur.execute( + "SELECT group_id FROM collection_torrent_groups WHERE rating_key = ?", + (rating_key,) + ) + rows = cur.fetchall() + return [row[0] for row in rows] diff --git a/red_plex/infrastructure/db/local_database.py b/red_plex/infrastructure/db/local_database.py index 791c4a7..8458e21 100644 --- a/red_plex/infrastructure/db/local_database.py +++ b/red_plex/infrastructure/db/local_database.py @@ -1,17 +1,16 @@ """Defines the LocalDatabase class for managing a SQLite database.""" -import os -import sqlite3 -from collections import defaultdict -from datetime import datetime -from typing import List, Optional +from typing import List, Optional, Set -from red_plex.domain.models import Album, Collection, TorrentGroup -from red_plex.infrastructure.db.utils.csv_to_db_migrator import CsvToDbMigrator -from red_plex.infrastructure.logger.logger import logger +from red_plex.domain.models import Album, Collection +from red_plex.infrastructure.db.albums import AlbumDatabaseManager +from red_plex.infrastructure.db.base import BaseDatabaseManager +from red_plex.infrastructure.db.collection import CollectionDatabaseManager +from red_plex.infrastructure.db.remote_mappings import RemoteMappingDatabaseManager -class LocalDatabase: +# pylint: disable=R0904 +class LocalDatabase(BaseDatabaseManager): """ A class for managing local persistent storage in a SQLite database. It creates or reuses a 'red_plex.db' file and provides CRUD operations for: @@ -19,522 +18,133 @@ class LocalDatabase: - collage_collections - bookmark_collections - collection_torrent_groups (relational table for group IDs) + - site tag mappings """ def __init__(self): - self.db_path = self._get_database_directory() - os.makedirs(self.db_path, exist_ok=True) - db_file_path = os.path.join(self.db_path, 'red_plex.db') - - # If the database file doesn't exist, create and run migrations - if not os.path.isfile(db_file_path): - # Create a temp connection for initialization - self.conn = sqlite3.connect(db_file_path) - self.conn.execute("PRAGMA journal_mode=WAL;") - self._create_tables() - # Migrate existing CSV data into the new DB - # This can be removed in future releases - migrator = CsvToDbMigrator(db_file_path=db_file_path) - migrator.migrate_from_csv_to_db() - self.conn.commit() - else: - self.conn = sqlite3.connect(db_file_path) - self.conn.execute("PRAGMA journal_mode=WAL;") - - def _create_tables(self): - """Create necessary tables if they do not exist.""" - self.conn.execute(""" - CREATE TABLE IF NOT EXISTS albums ( - album_id TEXT PRIMARY KEY, - name TEXT, - path TEXT NOT NULL, - added_at TEXT - ); - """) - self.conn.execute(""" - CREATE TABLE IF NOT EXISTS artists ( - artist_id INTEGER PRIMARY KEY AUTOINCREMENT, - artist_name TEXT NOT NULL UNIQUE - ); - """) - self.conn.execute(""" - CREATE TABLE IF NOT EXISTS album_artists ( - album_id TEXT, - artist_id INTEGER, - PRIMARY KEY (album_id, artist_id), - FOREIGN KEY (album_id) REFERENCES albums(album_id) ON DELETE CASCADE, - FOREIGN KEY (artist_id) REFERENCES artists(artist_id) ON DELETE CASCADE - ); - """) - self.conn.execute(""" - CREATE TABLE IF NOT EXISTS collage_collections ( - rating_key TEXT PRIMARY KEY, - name TEXT, - site TEXT, - external_id TEXT - ); - """) - self.conn.execute(""" - CREATE TABLE IF NOT EXISTS bookmark_collections ( - rating_key TEXT PRIMARY KEY, - site TEXT - ); - """) - self.conn.execute(""" - CREATE TABLE IF NOT EXISTS collection_torrent_groups ( - rating_key TEXT, - group_id INTEGER - ); - """) - self.conn.execute("DROP TABLE IF EXISTS beets_mappings;") - - @staticmethod - def _get_database_directory(): - """ - Return the directory path where the database file should live, - based on each OS's convention for local application data. - """ - if os.name == 'nt': # Windows - # Typically, LOCALAPPDATA or APPDATA for user-level application data - return os.path.join( - os.getenv('LOCALAPPDATA', os.path.expanduser('~\\AppData\\Local')), - 'red-plex' - ) - try: - if os.uname().sysname == 'Darwin': # macOS - - # Commonly used for persistent data: - return os.path.join(os.path.expanduser('~/Library/Application Support'), 'red-plex') - except ImportError: - pass - - # Linux / other Unix: use ~/.local/share/red-plex by XDG spec - data_home = os.getenv('XDG_DATA_HOME', os.path.expanduser('~/.local/share')) - return os.path.join(data_home, 'red-plex') - - def close(self): - """Close the database connection.""" - self.conn.close() + super().__init__() + # Initialize specialized managers + self._album_manager = AlbumDatabaseManager(self.conn) + self._collection_manager = CollectionDatabaseManager(self.conn) + self._remote_mapping_manager = RemoteMappingDatabaseManager(self.conn) # -------------------------------------------------------------------------- # ALBUMS # -------------------------------------------------------------------------- def insert_or_update_album(self, album: Album) -> None: - """ - Insert or update an album, including its artists, in the database. - Uses a transaction to ensure atomicity. - """ - logger.debug("Inserting/updating album with ID %s", album.id) - with self.conn: - # 1. Insert or replace the album itself - self.conn.execute( - """ - INSERT OR REPLACE INTO albums(album_id, name, path, added_at) - VALUES (?, ?, ?, ?) - """, - (album.id, - album.name, - album.path, - album.added_at.isoformat() if album.added_at else None) - ) - - # 2. Delete old artist associations for this album - self.conn.execute("DELETE FROM album_artists WHERE album_id = ?", (album.id,)) - - # 3. Insert artists and their new associations - if album.artists: - # Insert artists if they don't exist (IGNORE on conflict) - self.conn.executemany( - "INSERT OR IGNORE INTO artists(artist_name) VALUES (?)", - [(artist,) for artist in album.artists] - ) - - # Get artist IDs - cur = self.conn.cursor() - artist_ids = dict(cur.execute( - f"SELECT artist_name, artist_id FROM artists WHERE artist_name IN " - f"({','.join('?' * len(album.artists))})", - album.artists - )) - - # Insert into linking table - self.conn.executemany( - "INSERT INTO album_artists(album_id, artist_id) VALUES (?, ?)", - [(album.id, artist_ids[name]) for name in album.artists] - ) + """Insert or update an album, including its artists, in the database.""" + return self._album_manager.insert_or_update_album(album) def insert_albums_bulk(self, albums: List[Album]) -> None: - """ - Inserts or updates a list of albums in bulk using a single transaction. - Handles albums, artists, and their relationships efficiently. - """ - logger.debug("Inserting/updating %d albums in bulk.", len(albums)) - - album_rows = [] - all_artists = set() - album_id_to_artists = {} - - for album in albums: - album_rows.append(( - album.id, - album.name, - album.path, - album.added_at.isoformat() if album.added_at else None - )) - if album.artists: - all_artists.update(album.artists) - album_id_to_artists[album.id] = album.artists - - with self.conn: - # 1. Insert/update all albums - self.conn.executemany( - "INSERT OR REPLACE INTO albums(album_id, name, path, added_at) VALUES (?, ?, ?, ?)", - album_rows - ) - - album_ids = [album.id for album in albums] - - # 2. Delete all existing artist links for the albums being updated - self.conn.execute( - f"DELETE FROM album_artists WHERE album_id IN ({','.join('?' * len(album_ids))})", - album_ids - ) - - # 3. Insert all new unique artists - if all_artists: - self.conn.executemany( - "INSERT OR IGNORE INTO artists(artist_name) VALUES (?)", - [(artist,) for artist in all_artists] - ) - - # 4. Get all required artist IDs in one query - cur = self.conn.cursor() - artist_name_to_id = dict(cur.execute( - f"SELECT artist_name, artist_id FROM artists WHERE artist_name IN " - f"({','.join('?' * len(all_artists))})", - list(all_artists) - )) - - # 5. Prepare and insert all album-artist links - album_artist_links = [] - for album_id, artist_names in album_id_to_artists.items(): - for name in artist_names: - album_artist_links.append((album_id, artist_name_to_id[name])) - - self.conn.executemany( - "INSERT INTO album_artists(album_id, artist_id) VALUES (?, ?)", - album_artist_links - ) + """Inserts or updates a list of albums in bulk using a single transaction.""" + return self._album_manager.insert_albums_bulk(albums) def get_album(self, album_id: str) -> Optional[Album]: - """ - Retrieve a single album by its ID, including its list of artists. - """ - cur = self.conn.cursor() - - # Get album details - cur.execute("SELECT album_id, name, path, added_at " - "FROM albums WHERE album_id = ?", (album_id,)) - row = cur.fetchone() - if not row: - return None - - _id, _name, _path, _added_at_str = row - added_at = datetime.fromisoformat(_added_at_str) if _added_at_str else None - - # Get associated artists - cur.execute(""" - SELECT ar.artist_name - FROM artists ar - JOIN album_artists aa ON ar.artist_id = aa.artist_id - WHERE aa.album_id = ? - """, (album_id,)) - artists = [row[0] for row in cur.fetchall()] - - return Album(id=_id, name=_name, path=_path, added_at=added_at, artists=artists) + """Retrieve a single album by its ID, including its list of artists.""" + return self._album_manager.get_album(album_id) def get_all_albums(self) -> List[Album]: - """ - Retrieve all albums from the database, including their lists of artists. - This is done efficiently to avoid the N+1 query problem. - """ - cur = self.conn.cursor() - - # 1. Fetch all albums - cur.execute("SELECT album_id, name, path, added_at FROM albums") - album_rows = cur.fetchall() - if not album_rows: - return [] - - # 2. Fetch all artist relationships in a single query - artist_map = defaultdict(list) - cur.execute(""" - SELECT aa.album_id, ar.artist_name - FROM album_artists aa - JOIN artists ar ON aa.artist_id = ar.artist_id - """) - for album_id, artist_name in cur.fetchall(): - artist_map[album_id].append(artist_name) - - # 3. Construct Album objects - albums = [] - for _id, _name, _path, _added_at_str in album_rows: - added_at = datetime.fromisoformat(_added_at_str) if _added_at_str else None - albums.append( - Album(id=_id, name=_name, path=_path, added_at=added_at, artists=artist_map[_id]) - ) - return albums + """Retrieve all albums from the database, including their lists of artists.""" + return self._album_manager.get_all_albums() def delete_album(self, album_id: str) -> None: - """ - Delete an album. Associated artist links are removed automatically - due to 'ON DELETE CASCADE' in the foreign key constraint. - """ - logger.debug("Deleting album with ID %s", album_id) - with self.conn: - self.conn.execute("DELETE FROM albums WHERE album_id = ?", (album_id,)) - # Note: We might want to clean up orphan artists later, but for now, this is fine. + """Delete an album.""" + return self._album_manager.delete_album(album_id) def reset_albums(self): - """ - Deletes all records from 'albums', 'album_artists', and 'artists'. - """ - logger.info("Resetting albums: dropping and recreating tables.") - with self.conn: - self.conn.execute("DROP TABLE IF EXISTS album_artists") - self.conn.execute("DROP TABLE IF EXISTS artists") - self.conn.execute("DROP TABLE IF EXISTS albums") + """Deletes all records from 'albums', 'album_artists', and 'artists'.""" + self._album_manager.reset_albums() + # Recreate tables after reset self._create_tables() - logger.info("Album-related tables have been successfully reset.") # -------------------------------------------------------------------------- # COLLAGE COLLECTIONS (and their groups) # -------------------------------------------------------------------------- def insert_or_update_collage_collection(self, coll: Collection) -> None: + """Insert or update a collage-based collection, along with its torrent groups.""" + return self._collection_manager.insert_or_update_collage_collection(coll) + + def merge_torrent_groups_for_collage_collection(self, rating_key: str, + new_group_ids: Set[int]) -> None: """ - Insert or update a collage-based collection, along with its torrent groups. - We'll do an upsert in collage_collections, then remove all old group_ids - from collection_torrent_groups for that rating_key, and re-insert them. + Merges a new set of torrent group IDs with existing ones for a collage collection. """ - logger.debug("Inserting/updating collage collection with rating_key %s", coll.id) - self.conn.execute( - """ - INSERT OR REPLACE INTO collage_collections(rating_key, name, site, external_id) - VALUES (?, ?, ?, ?) - """, - (coll.id, coll.name, coll.site, coll.external_id) + return self._collection_manager.merge_torrent_groups_for_collage_collection( + rating_key, + new_group_ids ) - # Remove old group_ids for that rating_key - self.conn.execute( - "DELETE FROM collection_torrent_groups WHERE rating_key = ?", - (coll.id,) - ) - # Insert new group_ids - if coll.torrent_groups: - for tg in coll.torrent_groups: - self.conn.execute( - "INSERT INTO collection_torrent_groups(rating_key, group_id) VALUES(?, ?)", - (coll.id, tg.id) - ) - self.conn.commit() def get_collage_collection(self, rating_key: str) -> Optional[Collection]: - """ - Retrieve a single collage-based collection (and associated group_ids) by rating_key. - Returns a Collection or None if not found. - """ - cur = self.conn.cursor() - # Get collage collection fields - cur.execute( - """ - SELECT rating_key, name, site, external_id - FROM collage_collections - WHERE rating_key = ? - """, - (rating_key,) - ) - row = cur.fetchone() - if not row: - return None - rating_key_val, name, site, external_id = row - # Get associated group_ids - group_ids = self._get_torrent_group_ids_for(rating_key_val) - return Collection( - id=rating_key_val, - external_id=external_id, - name=name, - torrent_groups=[TorrentGroup(id=gid) for gid in group_ids], - site=site - ) + """Retrieve a single collage-based collection by rating_key.""" + return self._collection_manager.get_collage_collection(rating_key) def get_all_collage_collections(self) -> List[Collection]: - """ - Retrieve all collage-based collections from the DB, - along with their torrent groups. - """ - cur = self.conn.cursor() - cur.execute("SELECT rating_key, name, site, external_id FROM collage_collections") - rows = cur.fetchall() - collections = [] - for (rk, name, site, external_id) in rows: - group_ids = self._get_torrent_group_ids_for(rk) - collections.append(Collection( - id=rk, - external_id=external_id, - name=name, - torrent_groups=[TorrentGroup(id=gid) for gid in group_ids], - site=site - )) - return collections + """Retrieve all collage-based collections from the DB.""" + return self._collection_manager.get_all_collage_collections() def delete_collage_collection(self, rating_key: str) -> None: - """ - Delete a collage-based collection and associated torrent group mappings. - """ - logger.debug("Deleting collage collection with rating_key %s", rating_key) - self.conn.execute( - "DELETE FROM collage_collections WHERE rating_key = ?", - (rating_key,) - ) - self.conn.execute( - "DELETE FROM collection_torrent_groups WHERE rating_key = ?", - (rating_key,) - ) - self.conn.commit() + """Delete a collage-based collection and associated torrent group mappings.""" + return self._collection_manager.delete_collage_collection(rating_key) def reset_collage_collections(self): - """ - Deletes all records from 'collage_collections' and - their associated rows in 'collection_torrent_groups'. - """ - logger.info("Resetting collage collections (deleting all rows in 'collage_collections').") - self.conn.execute("DELETE FROM collage_collections") - logger.info("Removing associated torrent groups for collage collections.") - self.conn.execute(""" - DELETE FROM collection_torrent_groups - WHERE rating_key NOT IN (SELECT rating_key FROM bookmark_collections) - """) - - self.conn.commit() + """Deletes all records from 'collage_collections'.""" + return self._collection_manager.reset_collage_collections() # -------------------------------------------------------------------------- # BOOKMARK COLLECTIONS # -------------------------------------------------------------------------- def insert_or_update_bookmark_collection(self, coll: Collection) -> None: - """ - Insert or update a bookmark-based collection (in bookmark_collections), - then remove all old group_ids from 'collection_torrent_groups' for that rating_key - and re-insert them. - """ - logger.debug("Inserting/updating bookmark collection with rating_key %s", coll.id) - self.conn.execute( - """ - INSERT OR REPLACE INTO bookmark_collections(rating_key, site) - VALUES (?, ?) - """, - (coll.id, coll.site) - ) - # Remove old group_ids for that rating_key - self.conn.execute( - "DELETE FROM collection_torrent_groups WHERE rating_key = ?", - (coll.id,) - ) - # Insert new group_ids - if coll.torrent_groups: - for tg in coll.torrent_groups: - self.conn.execute( - "INSERT INTO collection_torrent_groups(rating_key, group_id) VALUES(?, ?)", - (coll.id, tg.id) - ) - self.conn.commit() + """Insert or update a bookmark-based collection.""" + return self._collection_manager.insert_or_update_bookmark_collection(coll) def get_bookmark_collection(self, rating_key: str) -> Optional[Collection]: - """ - Retrieve a single bookmark collection (plus group_ids) by rating_key. - Returns a Collection or None if not found. - """ - cur = self.conn.cursor() - cur.execute( - """ - SELECT rating_key, site - FROM bookmark_collections - WHERE rating_key = ? - """, - (rating_key,) - ) - row = cur.fetchone() - if not row: - return None - rating_key_val, site = row - group_ids = self._get_torrent_group_ids_for(rating_key_val) - # We can store the name as something like f"{site.upper()} Bookmarks" - return Collection( - id=rating_key_val, - name=f"{site.upper()} Bookmarks", - site=site, - torrent_groups=[TorrentGroup(id=gid) for gid in group_ids] - ) + """Retrieve a single bookmark collection by rating_key.""" + return self._collection_manager.get_bookmark_collection(rating_key) def get_all_bookmark_collections(self) -> List[Collection]: - """ - Retrieve all bookmark collections from the DB, - along with their torrent groups. - """ - cur = self.conn.cursor() - cur.execute("SELECT rating_key, site FROM bookmark_collections") - rows = cur.fetchall() - bookmarks = [] - for (rk, site) in rows: - group_ids = self._get_torrent_group_ids_for(rk) - bookmarks.append(Collection( - id=rk, - name=f"{site.upper()} Bookmarks", - site=site, - torrent_groups=[TorrentGroup(id=gid) for gid in group_ids] - )) - return bookmarks + """Retrieve all bookmark collections from the DB.""" + return self._collection_manager.get_all_bookmark_collections() def delete_bookmark_collection(self, rating_key: str) -> None: - """ - Delete a bookmark-based collection and associated torrent group mappings. - """ - logger.debug("Deleting bookmark collection with rating_key %s", rating_key) - self.conn.execute( - "DELETE FROM bookmark_collections WHERE rating_key = ?", - (rating_key,) - ) - self.conn.execute( - "DELETE FROM collection_torrent_groups WHERE rating_key = ?", - (rating_key,) - ) - self.conn.commit() + """Delete a bookmark-based collection and associated torrent group mappings.""" + return self._collection_manager.delete_bookmark_collection(rating_key) def reset_bookmark_collections(self): - """ - Deletes all records from 'bookmark_collections' and - their associated rows in 'collection_torrent_groups'. - """ - logger.info("Resetting bookmark collections (deleting all rows in 'bookmark_collections').") - self.conn.execute("DELETE FROM bookmark_collections") - logger.info("Removing associated torrent groups for bookmark collections.") - self.conn.execute(""" - DELETE FROM collection_torrent_groups - WHERE rating_key NOT IN (SELECT rating_key FROM collage_collections) - """) - - self.conn.commit() + """Deletes all records from 'bookmark_collections'.""" + return self._collection_manager.reset_bookmark_collections() # -------------------------------------------------------------------------- - # HELPER: TORRENT GROUPS FOR A GIVEN RATING_KEY + # SITE TAG MAPPINGS # -------------------------------------------------------------------------- - def _get_torrent_group_ids_for(self, rating_key: str) -> List[int]: - """ - Retrieve all group_ids for a given rating_key from collection_torrent_groups. - """ - cur = self.conn.cursor() - cur.execute( - "SELECT group_id FROM collection_torrent_groups WHERE rating_key = ?", - (rating_key,) - ) - rows = cur.fetchall() - return [row[0] for row in rows] + def insert_site_tag_mapping(self, rating_key: str, + group_id: int, + site: str, + tags: List[str]) -> None: + """Insert or update a site tag mapping with its associated tags.""" + return self._remote_mapping_manager.insert_rating_key_group_id_mapping( + rating_key, + group_id, site, tags) + + def get_rating_keys_by_tags(self, tags: List[str]) -> List[str]: + """Get rating keys that have mappings containing all specified tags.""" + return self._remote_mapping_manager.get_rating_keys_by_tags(tags) + + def get_group_ids_by_rating_keys(self, rating_keys: List[str], site: str) -> List[str]: + """Get group IDs for given rating keys from a specific site.""" + return self._remote_mapping_manager.get_group_ids_by_rating_keys(rating_keys, site) + + def get_unscanned_albums(self) -> List[str]: + """Get rating keys from albums table that are not present in mappings.""" + return self._remote_mapping_manager.get_unscanned_albums() + + def get_site_tags_stats(self): + """Get statistics about site tag mappings.""" + return self._remote_mapping_manager.get_remote_mappings_stats() + + def get_recent_site_tag_mappings(self, limit: int = 20): + """Get recent site tag mappings for display.""" + return self._remote_mapping_manager.get_recent_remote_mappings(limit) + + def reset_tag_mappings(self): + """Reset site tag mappings.""" + return self._remote_mapping_manager.reset_remote_mappings() diff --git a/red_plex/infrastructure/db/remote_mappings.py b/red_plex/infrastructure/db/remote_mappings.py new file mode 100644 index 0000000..50bc692 --- /dev/null +++ b/red_plex/infrastructure/db/remote_mappings.py @@ -0,0 +1,189 @@ +"""Remote mapping database operations for rating key to group ID mappings.""" + +from typing import List + +from red_plex.infrastructure.logger.logger import logger + + +class RemoteMappingDatabaseManager: + """Manages remote mapping database operations between Plex rating keys and site group IDs.""" + + def __init__(self, conn): + self.conn = conn + + def insert_rating_key_group_id_mapping(self, rating_key: str, + group_id: int, + site: str, + tags: List[str]) -> None: + """ + Insert or update mapping with its associated tags. + """ + logger.debug("Inserting rating key -> group id mapping: " + "rating_key=%s, group_id=%s, site=%s", + rating_key, group_id, site) + + with self.conn: + # Insert or ignore the mapping + cur = self.conn.cursor() + cur.execute(""" + INSERT OR IGNORE INTO rating_key_group_id_mappings(rating_key, group_id, site) + VALUES (?, ?, ?) + """, (rating_key, group_id, site)) + + # Get the mapping ID + cur.execute(""" + SELECT id FROM rating_key_group_id_mappings + WHERE rating_key = ? AND group_id = ? AND site = ? + """, (rating_key, group_id, site)) + mapping_id = cur.fetchone()[0] + + # Delete existing tag associations for this mapping + cur.execute("DELETE FROM mapping_tags WHERE mapping_id = ?", (mapping_id,)) + + if tags: + # Insert tags if they don't exist + self.conn.executemany( + "INSERT OR IGNORE INTO site_tags(tag_name) VALUES (?)", + [(tag,) for tag in tags] + ) + + # Get tag IDs + tag_ids = dict(cur.execute( + f"SELECT tag_name, id FROM site_tags WHERE tag_name IN " + f"({','.join('?' * len(tags))})", + tags + )) + + # Insert tag associations + self.conn.executemany( + "INSERT INTO mapping_tags(mapping_id, tag_id) VALUES (?, ?)", + [(mapping_id, tag_ids[tag]) for tag in tags] + ) + + def get_rating_keys_by_tags(self, tags: List[str]) -> List[str]: + """ + Get rating keys that have mappings containing all specified tags. + """ + if not tags: + return [] + + cur = self.conn.cursor() + + placeholders = ','.join('?' * len(tags)) + cur.execute(f""" + SELECT DISTINCT stm.rating_key + FROM rating_key_group_id_mappings stm + JOIN mapping_tags mt ON stm.id = mt.mapping_id + JOIN site_tags st ON mt.tag_id = st.id + WHERE st.tag_name IN ({placeholders}) + GROUP BY stm.rating_key + HAVING COUNT(DISTINCT st.tag_name) = ? + """, tags + [len(tags)]) + + return [row[0] for row in cur.fetchall()] + + def get_unscanned_albums(self) -> List[str]: + """ + Get rating keys from albums table that are not present in + rating_key_group_id_mappings, ordered by most recently added. + """ + cur = self.conn.cursor() + cur.execute(""" + SELECT a.album_id + FROM albums a + LEFT JOIN rating_key_group_id_mappings stm ON a.album_id = stm.rating_key + WHERE stm.rating_key IS NULL + ORDER BY a.added_at DESC + """) + + return [row[0] for row in cur.fetchall()] + + def get_remote_mappings_stats(self): + """ + Get statistics about remote mappings. + Returns a tuple of (mapped_albums, total_tags, total_mappings). + """ + cur = self.conn.cursor() + cur.execute(""" + SELECT COUNT(DISTINCT stm.rating_key) as mapped_albums, + COUNT(DISTINCT st.tag_name) as total_tags, + COUNT(stm.id) as total_mappings + FROM rating_key_group_id_mappings stm + LEFT JOIN mapping_tags mt ON stm.id = mt.mapping_id + LEFT JOIN site_tags st ON mt.tag_id = st.id + """) + stats = cur.fetchone() + return (stats[0] or 0, stats[1] or 0, stats[2] or 0) + + def get_recent_remote_mappings(self, limit: int = 20): + """ + Get recent remote mappings for display. + Returns a list of dictionaries with album information. + """ + cur = self.conn.cursor() + cur.execute(""" + SELECT stm.rating_key, stm.group_id, stm.site, + a.name as album_name, + GROUP_CONCAT(DISTINCT ar.artist_name) as artists, + GROUP_CONCAT(DISTINCT st.tag_name) as tags + FROM rating_key_group_id_mappings stm + LEFT JOIN albums a ON stm.rating_key = a.album_id + LEFT JOIN album_artists aa ON a.album_id = aa.album_id + LEFT JOIN artists ar ON aa.artist_id = ar.artist_id + LEFT JOIN mapping_tags mt ON stm.id = mt.mapping_id + LEFT JOIN site_tags st ON mt.tag_id = st.id + GROUP BY stm.id + ORDER BY stm.id DESC + LIMIT ? + """, (limit,)) + + results = [] + for row in cur.fetchall(): + # Handle the comma separation manually since SQLite GROUP_CONCAT uses comma by default + artists = row[4].replace(',', ', ') if row[4] else 'Unknown Artist' + tags = row[5].replace(',', ', ') if row[5] else '' + + results.append({ + 'rating_key': row[0], + 'group_id': row[1], + 'site': row[2], + 'album_name': row[3] or 'Unknown Album', + 'artists': artists, + 'tags': tags + }) + return results + + def get_group_ids_by_rating_keys(self, rating_keys: List[str], site: str) -> List[str]: + """ + Get group IDs for given rating keys from a specific site. + + Args: + rating_keys: List of rating keys to look up + site: Site to filter by + + Returns: + List of group IDs as strings + """ + if not rating_keys: + return [] + + cur = self.conn.cursor() + placeholders = ','.join('?' * len(rating_keys)) + cur.execute(f""" + SELECT DISTINCT group_id + FROM rating_key_group_id_mappings + WHERE rating_key IN ({placeholders}) AND site = ? + """, rating_keys + [site.lower()]) + + return [str(row[0]) for row in cur.fetchall()] + + def reset_remote_mappings(self): + """ + Reset remote mappings. + """ + logger.info("Resetting remote mappings") + + with self.conn: + self.conn.execute("DELETE FROM mapping_tags") + self.conn.execute("DELETE FROM rating_key_group_id_mappings") + self.conn.execute("DELETE FROM site_tags") diff --git a/red_plex/infrastructure/gui/app.py b/red_plex/infrastructure/gui/app.py index 2425c4b..7072774 100644 --- a/red_plex/infrastructure/gui/app.py +++ b/red_plex/infrastructure/gui/app.py @@ -2,23 +2,20 @@ import logging import os -from flask import Flask, render_template, request, redirect, url_for, flash, g -from flask_socketio import SocketIO, emit +from flask import Flask, render_template, g +from flask_socketio import SocketIO -from red_plex.infrastructure.cli.cli import update_collections_from_collages -from red_plex.infrastructure.config.config import ( - load_config, - save_config -) -from red_plex.infrastructure.config.models import Configuration from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.gui.routes.bookmarks import register_bookmarks_routes +from red_plex.infrastructure.gui.routes.collages import register_collages_routes +from red_plex.infrastructure.gui.routes.config import register_config_routes +from red_plex.infrastructure.gui.routes.database import register_database_routes +from red_plex.infrastructure.gui.routes.remote_mappings import register_remote_mappings_routes +from red_plex.infrastructure.gui.routes.site_tags import register_site_tags_routes from red_plex.infrastructure.logger.logger import configure_logger -from red_plex.infrastructure.plex.plex_manager import PlexManager -from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI -from red_plex.infrastructure.service.collection_processor import CollectionProcessingService -from red_plex.use_case.create_collection.album_fetch_mode import AlbumFetchMode -# pylint: disable=W0703,W0718,R0914,R0915 + +# pylint: disable=W0703,W0718,R0914,R0915,W0511 class WebSocketHandler(logging.Handler): """Custom logging handler that sends log messages via WebSocket.""" @@ -75,380 +72,21 @@ def close_db(error): if error is not None: logger.error("Error during request teardown: %s", error) - def map_fetch_mode(fetch_mode_str) -> AlbumFetchMode: - """Map the fetch mode string to an AlbumFetchMode enum.""" - if fetch_mode_str == 'query': - return AlbumFetchMode.QUERY - return AlbumFetchMode.TORRENT_NAME - @app.route('/') def index(): """Home page.""" return render_template('index.html') - @app.route('/config') - def config_view(): - """View configuration.""" - try: - config_data = load_config() - return render_template('config.html', config=config_data.to_dict()) - except Exception as e: - flash(f'Error loading configuration: {str(e)}', 'error') - return render_template('config.html', config={}) - - @app.route('/config/edit', methods=['GET', 'POST']) - def config_edit(): - """Edit configuration.""" - if request.method == 'POST': - try: - config_data = request.form.to_dict() - - # Convert nested structure for site configs - sites_config = {} - for key, value in config_data.items(): - if key.startswith('RED_') or key.startswith('OPS_'): - site, field = key.split('_', 1) - if site not in sites_config: - sites_config[site] = {} - if field == 'RATE_LIMIT_CALLS': - sites_config[site].setdefault('RATE_LIMIT', {})['calls'] = int(value) - elif field == 'RATE_LIMIT_SECONDS': - sites_config[site].setdefault('RATE_LIMIT', {})['seconds'] = int(value) - else: - sites_config[site][field] = value - - # Build final config - final_config = { - 'LOG_LEVEL': config_data.get('LOG_LEVEL', 'INFO'), - 'PLEX_URL': config_data.get('PLEX_URL', ''), - 'PLEX_TOKEN': config_data.get('PLEX_TOKEN', ''), - 'SECTION_NAME': config_data.get('SECTION_NAME', 'Music'), - } - final_config.update(sites_config) - - # Save configuration - config = Configuration.from_dict(final_config) - save_config(config) - - flash('Configuration saved successfully!', 'success') - return redirect(url_for('config_view')) - except Exception as e: - flash(f'Error saving configuration: {str(e)}', 'error') - - try: - config_data = load_config() - return render_template('config_edit.html', config=config_data.to_dict()) - except Exception as e: - flash(f'Error loading configuration: {str(e)}', 'error') - return render_template('config_edit.html', config={}) - - @app.route('/collages') - def collages(): - """View collages.""" - try: - db = get_db() - collages = db.get_all_collage_collections() - return render_template('collages.html', collages=collages) - except Exception as e: - flash(f'Error loading collages: {str(e)}', 'error') - return render_template('collages.html', collages=[]) - - @app.route('/collages/convert', methods=['GET', 'POST']) - def collages_convert(): - """Convert collages.""" - if request.method == 'POST': - try: - collage_ids = request.form.get('collage_ids', '').split() - site = request.form.get('site') - fetch_mode = request.form.get('fetch_mode', 'torrent_name') - - if not collage_ids: - flash('Please provide at least one collage ID.', 'error') - return render_template('collages_convert.html') - - if not site: - flash('Please select a site.', 'error') - return render_template('collages_convert.html') - - # Start processing in background - def process_collages(): - logger = logging.getLogger('red_plex') - thread_db = None - try: - thread_db = LocalDatabase() - album_fetch_mode = map_fetch_mode(fetch_mode) - - with app.app_context(): - socketio.emit('status_update', - {'message': 'Starting collage conversion process...'}) - - logger.info("WebSocket logging is configured and ready.") - logger.info("Connecting to Plex server...") - - try: - plex_manager = PlexManager(db=thread_db) - except Exception as e: - logger.error('Failed to initialize PlexManager: %s', e) - with app.app_context(): - socketio.emit('status_update', { - 'message': f'Failed to connect to Plex server: {str(e)}', - 'error': True - }) - return - - logger.info("Successfully connected to Plex server.") - - gazelle_api = GazelleAPI(site) - processor = CollectionProcessingService(thread_db, - plex_manager, - gazelle_api) - - def web_echo(message): - logger.info(message) - - def web_confirm(message): - logger.info('Auto-confirming: %s', message) - return True - - processor.process_collages( - collage_ids=collage_ids, - album_fetch_mode=album_fetch_mode, - echo_func=web_echo, - confirm_func=web_confirm - ) - - with app.app_context(): - socketio.emit('status_update', { - 'message': 'Collage processing completed successfully!', - 'finished': True - }) - - except Exception as e: - logger.critical('An unhandled error occurred: %s', e, exc_info=True) - with app.app_context(): - socketio.emit('status_update', { - 'message': f'Error: {str(e)}', - 'error': True - }) - finally: - if thread_db: - thread_db.close() - - socketio.start_background_task(target=process_collages) - - flash('Processing started! Check the status below.', 'info') - return render_template('collages_convert.html', - processing=True) - - except Exception as e: - flash(f'Error starting collage conversion: {str(e)}', 'error') - - return render_template('collages_convert.html') - - @app.route('/bookmarks') - def bookmarks(): - """View bookmarks.""" - try: - db = get_db() - bookmarks = db.get_all_bookmark_collections() - return render_template('bookmarks.html', bookmarks=bookmarks) - except Exception as e: - flash(f'Error loading bookmarks: {str(e)}', 'error') - return render_template('bookmarks.html', bookmarks=[]) - - @app.route('/bookmarks/convert', methods=['GET', 'POST']) - def bookmarks_convert(): - """Convert bookmarks.""" - if request.method == 'POST': - try: - site = request.form.get('site') - fetch_mode = request.form.get('fetch_mode', 'torrent_name') - - if not site: - flash('Please select a site.', 'error') - return render_template('bookmarks_convert.html') - - # Start processing in background - def process_bookmarks(): - logger = logging.getLogger('red_plex') - thread_db = None - try: - thread_db = LocalDatabase() - album_fetch_mode = map_fetch_mode(fetch_mode) - - with app.app_context(): - socketio.emit('status_update', - {'message': 'Starting bookmark conversion process...'}) - - gazelle_api = GazelleAPI(site) - plex_manager = PlexManager(db=thread_db) - processor = CollectionProcessingService(thread_db, - plex_manager, - gazelle_api) - - def web_echo(message): - logger.info(message) - - def web_confirm(message): - logger.info('Auto-confirming: %s', message) - return True - - processor.process_bookmarks( - album_fetch_mode=album_fetch_mode, - echo_func=web_echo, - confirm_func=web_confirm - ) - - with app.app_context(): - socketio.emit('status_update', { - 'message': 'Bookmark processing completed successfully!', - 'finished': True - }) - except Exception as e: - logger.critical( - 'An unhandled error occurred during bookmark processing: %s', - e, - exc_info=True) - with app.app_context(): - socketio.emit('status_update', { - 'message': f'Error: {str(e)}', - 'error': True - }) - finally: - if thread_db: - thread_db.close() - - socketio.start_background_task(target=process_bookmarks) - - flash('Processing started! Check the status below.', 'info') - return render_template('bookmarks_convert.html', - processing=True) - - except Exception as e: - flash(f'Error starting bookmark conversion: {str(e)}', 'error') - - return render_template('bookmarks_convert.html') - - @app.route('/database') - def database(): - """View database status.""" - try: - db = get_db() - db_path = db.db_path - db_exists = os.path.exists(db_path) - - # Get some basic stats - stats = {} - if db_exists: - try: - stats['albums'] = len(db.get_all_albums()) - stats['collages'] = len(db.get_all_collage_collections()) - stats['bookmarks'] = len(db.get_all_bookmark_collections()) - except Exception as e: - logger.warning('Error getting database stats: %s', e) - stats = {'albums': 0, 'collages': 0, 'bookmarks': 0} - else: - stats = {'albums': 0, 'collages': 0, 'bookmarks': 0} - - return render_template('database.html', - db_path=db_path, - db_exists=db_exists, - stats=stats) - except Exception as e: - flash(f'Error loading database status: {str(e)}', 'error') - return render_template('database.html', - db_path="Unknown", - db_exists=False, - stats={}) - - @app.route('/database/albums/update', methods=['POST']) - def database_albums_update(): - """Update albums from Plex and update collections from collages.""" - try: - def update_albums(): - logger = logging.getLogger('red_plex') - thread_db = None - try: - thread_db = LocalDatabase() - - with app.app_context(): - socketio.emit('status_update', - {'message': 'Starting albums update from Plex...'}) - - plex_manager = PlexManager(db=thread_db) - - plex_manager.populate_album_table() - - with app.app_context(): - socketio.emit('status_update', - {'message': 'Albums update completed. ' - 'Starting collections update...'}) - - all_collages = thread_db.get_all_collage_collections() - if all_collages: - logger.info('Updating %s collage collections...', len(all_collages)) - - update_collections_from_collages( - local_database=thread_db, - collage_list=all_collages, - plex_manager=plex_manager, - fetch_bookmarks=False, - fetch_mode=AlbumFetchMode.TORRENT_NAME - ) - logger.info('Collections update from collages completed!') - else: - logger.info('No stored collages found to update.') - - with app.app_context(): - socketio.emit('status_update', { - 'message': 'Albums and collections update completed successfully!', - 'finished': True - }) - except Exception as e: - logger.critical('An unhandled error occurred during album update: %s', - e, - exc_info=True) - with app.app_context(): - socketio.emit('status_update', { - 'message': f'Error updating albums: {str(e)}', - 'error': True - }) - finally: - if thread_db: - thread_db.close() - - socketio.start_background_task(target=update_albums) - - flash('Albums and collections update started!', 'info') - except Exception as e: - flash(f'Error starting albums update: {str(e)}', 'error') - - return redirect(url_for('database')) - - @app.route('/database//reset', methods=['POST']) - def database_reset(table): - """Reset database table.""" - try: - db = get_db() - if table == 'albums': - db.reset_albums() - flash('Albums table reset successfully!', 'success') - elif table == 'collages': - db.reset_collage_collections() - flash('Collages table reset successfully!', 'success') - elif table == 'bookmarks': - db.reset_bookmark_collections() - flash('Bookmarks table reset successfully!', 'success') - else: - flash(f'Unknown table: {table}', 'error') - except Exception as e: - flash(f'Error resetting {table} table: {str(e)}', 'error') - - return redirect(url_for('database')) + # Register all route modules + register_config_routes(app) + register_collages_routes(app, socketio, get_db) + register_bookmarks_routes(app, socketio, get_db) + register_remote_mappings_routes(app, socketio, get_db) + register_site_tags_routes(app, socketio, get_db) + register_database_routes(app, socketio, get_db) @socketio.on('connect') def handle_connect(): """Handle WebSocket connection.""" - emit('status_update', {'message': 'Connected to red-plex server'}) return app, socketio diff --git a/red_plex/infrastructure/gui/routes/bookmarks.py b/red_plex/infrastructure/gui/routes/bookmarks.py new file mode 100644 index 0000000..61fae9a --- /dev/null +++ b/red_plex/infrastructure/gui/routes/bookmarks.py @@ -0,0 +1,105 @@ +"""Bookmark route handlers.""" +import logging + +from flask import render_template, request, flash + +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.infrastructure.service.collection_processor import CollectionProcessingService +from red_plex.use_case.create_collection.album_fetch_mode import AlbumFetchMode + + +def map_fetch_mode(fetch_mode_str) -> AlbumFetchMode: + """Map the fetch mode string to an AlbumFetchMode enum.""" + if fetch_mode_str == 'query': + return AlbumFetchMode.QUERY + return AlbumFetchMode.TORRENT_NAME + +# pylint: disable=R0801,W0718 +def register_bookmarks_routes(app, socketio, get_db): + """Register bookmark-related routes.""" + + @app.route('/bookmarks') + def bookmarks(): + """View bookmarks.""" + try: + db = get_db() + bookmarks = db.get_all_bookmark_collections() + return render_template('bookmarks.html', bookmarks=bookmarks) + except Exception as e: + flash(f'Error loading bookmarks: {str(e)}', 'error') + return render_template('bookmarks.html', bookmarks=[]) + + @app.route('/bookmarks/convert', methods=['GET', 'POST']) + def bookmarks_convert(): + """Convert bookmarks.""" + if request.method == 'POST': + try: + site = request.form.get('site') + fetch_mode = request.form.get('fetch_mode', 'torrent_name') + + if not site: + flash('Please select a site.', 'error') + return render_template('bookmarks_convert.html') + + # Start processing in background + def process_bookmarks(): + logger = logging.getLogger('red_plex') + thread_db = None + try: + thread_db = LocalDatabase() + album_fetch_mode = map_fetch_mode(fetch_mode) + + with app.app_context(): + socketio.emit('status_update', + {'message': 'Starting bookmark conversion process...'}) + + gazelle_api = GazelleAPI(site) + plex_manager = PlexManager(db=thread_db) + processor = CollectionProcessingService(thread_db, + plex_manager, + gazelle_api) + + def web_echo(message): + logger.info(message) + + def web_confirm(message): + logger.info('Auto-confirming: %s', message) + return True + + processor.process_bookmarks( + album_fetch_mode=album_fetch_mode, + echo_func=web_echo, + confirm_func=web_confirm + ) + + with app.app_context(): + socketio.emit('status_update', { + 'message': 'Bookmark processing completed successfully!', + 'finished': True + }) + except Exception as e: + logger.critical( + 'An unhandled error occurred during bookmark processing: %s', + e, + exc_info=True) + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Error: {str(e)}', + 'error': True + }) + finally: + if thread_db: + thread_db.close() + + socketio.start_background_task(target=process_bookmarks) + + flash('Processing started! Check the status below.', 'info') + return render_template('bookmarks_convert.html', + processing=True) + + except Exception as e: + flash(f'Error starting bookmark conversion: {str(e)}', 'error') + + return render_template('bookmarks_convert.html') diff --git a/red_plex/infrastructure/gui/routes/collages.py b/red_plex/infrastructure/gui/routes/collages.py new file mode 100644 index 0000000..b0c4b5d --- /dev/null +++ b/red_plex/infrastructure/gui/routes/collages.py @@ -0,0 +1,381 @@ +"""Collage route handlers.""" +import json +import logging + +from flask import render_template, request, flash + +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.infrastructure.service.collection_processor import CollectionProcessingService +from red_plex.use_case.create_collection.album_fetch_mode import AlbumFetchMode +from red_plex.use_case.upstream_sync.upstream_sync_use_case import UpstreamSyncUseCase + +logger = logging.getLogger('red_plex') + + +def map_fetch_mode(fetch_mode_str) -> AlbumFetchMode: + """Map the fetch mode string to an AlbumFetchMode enum.""" + if fetch_mode_str == 'query': + return AlbumFetchMode.QUERY + return AlbumFetchMode.TORRENT_NAME + + +# pylint: disable=R0801,W0718,R0914,R1702,R0915,R0912,R0911,R1705,R1702 +def register_collages_routes(app, socketio, get_db): + """Register collage-related routes.""" + + @app.route('/collages') + def collages(): + """View collages.""" + try: + db = get_db() + collages = db.get_all_collage_collections() + + # Determine which collages the user owns for each site + user_owned_collages = {} # site -> set of external_ids + + # Group collages by site to minimize API calls + collages_by_site = {} + for collage in collages: + site = collage.site + if site not in collages_by_site: + collages_by_site[site] = [] + collages_by_site[site].append(collage) + + # For each site, check user ownership + for site, _ in collages_by_site.items(): + try: + api = GazelleAPI(site) + user_info = api.get_user_info() + + if user_info and 'id' in user_info: + user_id = str(user_info['id']) + user_collages = api.get_user_collages(user_id) + + if user_collages is not None: + # Extract external IDs of owned collages + owned_external_ids = {uc.external_id for uc in user_collages} + user_owned_collages[site] = owned_external_ids + else: + # get_user_collages returned None (not supported for this site) + user_owned_collages[site] = set() + else: + # Failed to get user info + user_owned_collages[site] = set() + + except Exception as e: + logger.error('Error checking collage ownership for site %s: %s', site, e) + # On error, assume no ownership + user_owned_collages[site.upper()] = set() + return render_template('collages.html', + collages=collages, + user_owned_collages=user_owned_collages) + except Exception as e: + flash(f'Error loading collages: {str(e)}', 'error') + logger.error('Error loading collages: %s', e, exc_info=True) + return render_template('collages.html', collages=[], user_owned_collages={}) + + @app.route('/collages/convert', methods=['GET', 'POST']) + def collages_convert(): + """Convert collages.""" + if request.method == 'POST': + try: + collage_ids = request.form.get('collage_ids', '').split() + site = request.form.get('site') + fetch_mode = request.form.get('fetch_mode', 'torrent_name') + + if not collage_ids: + flash('Please provide at least one collage ID.', 'error') + return render_template('collages_convert.html') + + if not site: + flash('Please select a site.', 'error') + return render_template('collages_convert.html') + + # Start processing in background + def process_collages(): + thread_db = None + try: + thread_db = LocalDatabase() + album_fetch_mode = map_fetch_mode(fetch_mode) + + with app.app_context(): + socketio.emit('status_update', + {'message': 'Starting collage conversion process...'}) + + logger.info("WebSocket logging is configured and ready.") + logger.info("Connecting to Plex server...") + + try: + plex_manager = PlexManager(db=thread_db) + except Exception as e: + logger.error('Failed to initialize PlexManager: %s', e) + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Failed to connect to Plex server: {str(e)}', + 'error': True + }) + return + + logger.info("Successfully connected to Plex server.") + + gazelle_api = GazelleAPI(site) + processor = CollectionProcessingService(thread_db, + plex_manager, + gazelle_api) + + def web_echo(message): + logger.info(message) + + def web_confirm(message): + logger.info('Auto-confirming: %s', message) + return True + + processor.process_collages( + collage_ids=collage_ids, + album_fetch_mode=album_fetch_mode, + echo_func=web_echo, + confirm_func=web_confirm + ) + + with app.app_context(): + socketio.emit('status_update', { + 'message': 'Collage processing completed successfully!', + 'finished': True + }) + + except Exception as e: + logger.critical('An unhandled error occurred: %s', e, exc_info=True) + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Error: {str(e)}', + 'error': True + }) + finally: + if thread_db: + thread_db.close() + + socketio.start_background_task(target=process_collages) + + flash('Processing started! Check the status below.', 'info') + return render_template('collages_convert.html', + processing=True) + + except Exception as e: + flash(f'Error starting collage conversion: {str(e)}', 'error') + + return render_template('collages_convert.html') + + @app.route('/collages/upstream-sync', methods=['GET', 'POST']) + def collages_upstream_sync(): + """Sync collections to upstream collages.""" + if request.method == 'GET': + try: + db = get_db() + all_collages = db.get_all_collage_collections() + + # Determine which collages the user owns for each site + user_owned_collages = {} # site -> set of external_ids + user_collages = [] # Only collages owned by the user + + # Group collages by site to minimize API calls + collages_by_site = {} + for collage in all_collages: + site = collage.site + if site not in collages_by_site: + collages_by_site[site] = [] + collages_by_site[site].append(collage) + + # For each site, check user ownership + for site, site_collages in collages_by_site.items(): + try: + api = GazelleAPI(site) + user_info = api.get_user_info() + + if user_info and 'id' in user_info: + user_id = str(user_info['id']) + user_owned_collages_list = api.get_user_collages(user_id) + + if user_owned_collages_list is not None: + # Extract external IDs of owned collages + owned_external_ids = {uc.external_id + for uc in user_owned_collages_list} + user_owned_collages[site] = owned_external_ids + + # Add owned collages to the filtered list + for collage in site_collages: + if collage.external_id in owned_external_ids: + user_collages.append(collage) + else: + # get_user_collages returned None (not supported for this site) + user_owned_collages[site] = set() + else: + # Failed to get user info + user_owned_collages[site] = set() + + except Exception as e: + logger.error('Error checking collage ownership for site %s: %s', site, e) + # On error, assume no ownership + user_owned_collages[site] = set() + + # Get collage_ids from query parameters for specific collages + collage_ids_param = request.args.get('collage_ids', '') + selected_collage_ids = [] + if collage_ids_param: + selected_collage_ids = collage_ids_param.split(',') + + return render_template('collages_upstream_sync.html', + collages=user_collages, # Only show user-owned collages + selected_collage_ids=selected_collage_ids, + user_owned_collages=user_owned_collages) + except Exception as e: + flash(f'Error loading collages: {str(e)}', 'error') + return render_template('collages_upstream_sync.html', + collages=[], + user_owned_collages={}) + + elif request.method == 'POST': + try: + action = request.form.get('action') + + if action == 'get_preview': + # Get preview of what would be synced + collage_ids = request.form.getlist('collage_ids') + if not collage_ids: + flash('Please select at least one collage.', 'error') + return render_template('collages_upstream_sync.html', collages=[]) + + db = get_db() + plex_manager = PlexManager(db=db) + + # Get selected collages + selected_collages = [] + for collage_id in collage_ids: + # collage_id is actually the rating_key (Collection.id) + collage = db.get_collage_collection(collage_id) + if collage: + selected_collages.append(collage) + + # Use the upstream sync use case to get preview + logger.info('Getting upstream sync preview for collages: %s', collage_ids) + upstream_sync = UpstreamSyncUseCase(db, plex_manager) + preview_response = upstream_sync.get_sync_preview(selected_collages) + + if not preview_response.success: + flash(f'Error getting sync preview: {preview_response.error_message}', + 'error') + return render_template('collages_upstream_sync.html', + collages=db.get_all_collage_collections()) + + # Convert response to template format + preview_data = [] + for collage_preview in preview_response.preview_data: + collage = db.get_collage_collection(collage_preview.collage_id) + if collage: + album_details = [ + { + 'group_id': album.group_id, + 'display_name': album.display_name + } + for album in collage_preview.albums_to_add + ] + preview_data.append({ + 'collage': collage, + 'album_details': album_details + }) + + return render_template('collages_upstream_sync.html', + collages=db.get_all_collage_collections(), + preview_data=preview_data, + show_confirmation=True) + + elif action == 'confirm_sync': + # Perform the actual sync with selected albums + selected_albums = request.form.get('selected_albums', '') + + if not selected_albums: + flash('No albums selected for sync.', 'info') + return render_template('collages_upstream_sync.html', collages=[]) + + # Parse selected albums data (JSON format) + try: + albums_data = json.loads(selected_albums) + except json.JSONDecodeError: + flash('Invalid album selection data.', 'error') + return render_template('collages_upstream_sync.html', collages=[]) + + # Start sync process in background + def process_upstream_sync(): + thread_db = None + try: + thread_db = LocalDatabase() + plex_manager = PlexManager(db=thread_db) + + with app.app_context(): + socketio.emit('status_update', + {'message': 'Starting upstream sync process...'}) + + # Get collages to sync + collages_to_sync = [] + for collage_id in albums_data.keys(): + collage = thread_db.get_collage_collection(collage_id) + if collage: + collages_to_sync.append(collage) + + if not collages_to_sync: + logger.warning('No valid collages found for sync') + return + + # Use upstream sync use case + upstream_sync = UpstreamSyncUseCase(thread_db, plex_manager) + sync_response = upstream_sync.sync_collections_upstream( + collages_to_sync, albums_data) + + # Log results + for collage_id, result in sync_response.sync_results.items(): + collage = thread_db.get_collage_collection(collage_id) + collage_name = collage.name if collage else collage_id + + if result.get('success', False): + added = result.get('added', 0) + rejected = result.get('rejected', 0) + duplicated = result.get('duplicated', 0) + logger.info('Collage "%s": %d added, %d rejected, ' + '%d duplicated', + collage_name, added, rejected, duplicated) + else: + logger.error('Failed to sync collage "%s": %s', + collage_name, result.get('error', 'Unknown error')) + + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Upstream sync completed! ' + f'{sync_response.synced_collages}' + f'/{sync_response.total_collages}' + f' collages synced successfully.', + 'finished': True + }) + + except Exception as e: + logger.critical('Error in upstream sync: %s', e, exc_info=True) + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Error: {str(e)}', + 'error': True + }) + finally: + if thread_db: + thread_db.close() + + socketio.start_background_task(target=process_upstream_sync) + flash('Upstream sync started! Check the status below.', + 'info') + return render_template('collages_upstream_sync.html', + collages=[], + processing=True) + + except Exception as e: + flash(f'Error in upstream sync: {str(e)}', 'error') + + return render_template('collages_upstream_sync.html', collages=[]) diff --git a/red_plex/infrastructure/gui/routes/config.py b/red_plex/infrastructure/gui/routes/config.py new file mode 100644 index 0000000..9f84d91 --- /dev/null +++ b/red_plex/infrastructure/gui/routes/config.py @@ -0,0 +1,65 @@ +"""Configuration route handlers.""" +from flask import render_template, request, redirect, url_for, flash + +from red_plex.infrastructure.config.config import load_config, save_config +from red_plex.infrastructure.config.models import Configuration + +# pylint: disable=W0718 +def register_config_routes(app): + """Register configuration-related routes.""" + + @app.route('/config') + def config_view(): + """View configuration.""" + try: + config_data = load_config() + return render_template('config.html', config=config_data.to_dict()) + except Exception as e: + flash(f'Error loading configuration: {str(e)}', 'error') + return render_template('config.html', config={}) + + @app.route('/config/edit', methods=['GET', 'POST']) + def config_edit(): + """Edit configuration.""" + if request.method == 'POST': + try: + config_data = request.form.to_dict() + + # Convert nested structure for site configs + sites_config = {} + for key, value in config_data.items(): + if key.startswith('RED_') or key.startswith('OPS_'): + site, field = key.split('_', 1) + if site not in sites_config: + sites_config[site] = {} + if field == 'RATE_LIMIT_CALLS': + sites_config[site].setdefault('RATE_LIMIT', {})['calls'] = int(value) + elif field == 'RATE_LIMIT_SECONDS': + sites_config[site].setdefault('RATE_LIMIT', {})['seconds'] = int(value) + else: + sites_config[site][field] = value + + # Build final config + final_config = { + 'LOG_LEVEL': config_data.get('LOG_LEVEL', 'INFO'), + 'PLEX_URL': config_data.get('PLEX_URL', ''), + 'PLEX_TOKEN': config_data.get('PLEX_TOKEN', ''), + 'SECTION_NAME': config_data.get('SECTION_NAME', 'Music'), + } + final_config.update(sites_config) + + # Save configuration + config = Configuration.from_dict(final_config) + save_config(config) + + flash('Configuration saved successfully!', 'success') + return redirect(url_for('config_view')) + except Exception as e: + flash(f'Error saving configuration: {str(e)}', 'error') + + try: + config_data = load_config() + return render_template('config_edit.html', config=config_data.to_dict()) + except Exception as e: + flash(f'Error loading configuration: {str(e)}', 'error') + return render_template('config_edit.html', config={}) diff --git a/red_plex/infrastructure/gui/routes/database.py b/red_plex/infrastructure/gui/routes/database.py new file mode 100644 index 0000000..e92147a --- /dev/null +++ b/red_plex/infrastructure/gui/routes/database.py @@ -0,0 +1,163 @@ +"""Database route handlers.""" +import asyncio +import os + +from flask import render_template, flash, redirect, url_for + +from red_plex.infrastructure.cli.utils import update_collections_from_collages +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.logger.logger import logger +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.use_case.create_collection.album_fetch_mode import AlbumFetchMode + + +# pylint: disable=R0801,W0718,R0915 +def register_database_routes(app, socketio, get_db): + """Register database-related routes.""" + + @app.route('/database') + def database(): + """View database status.""" + try: + db = get_db() + db_path = db.db_path + db_exists = os.path.exists(db_path) + + # Get some basic stats + stats = {} + if db_exists: + try: + stats['albums'] = len(db.get_all_albums()) + stats['collages'] = len(db.get_all_collage_collections()) + stats['bookmarks'] = len(db.get_all_bookmark_collections()) + + # Get remote mappings stats + cur = db.conn.cursor() + cur.execute("SELECT COUNT(*) FROM rating_key_group_id_mappings") + stats['remote_mappings'] = cur.fetchone()[0] + except Exception as e: + logger.warning('Error getting database stats: %s', e) + stats = {'albums': 0, 'collages': 0, 'bookmarks': 0, 'remote_mappings': 0} + else: + stats = {'albums': 0, 'collages': 0, 'bookmarks': 0, 'remote_mappings': 0} + + return render_template('database.html', + db_path=db_path, + db_exists=db_exists, + stats=stats) + except Exception as e: + flash(f'Error loading database status: {str(e)}', 'error') + return render_template('database.html', + db_path="Unknown", + db_exists=False, + stats={'albums': 0, + 'collages': 0, + 'bookmarks': 0, + 'remote_mappings': 0}) + + @app.route('/database/albums/update', methods=['POST']) + def database_albums_update(): + """Update albums from Plex and update collections from collages.""" + try: + def run_update(): + """Run the update in a background thread with asyncio isolation.""" + + # Force no event loop in this thread by setting the policy + if hasattr(asyncio, 'WindowsSelectorEventLoopPolicy'): + asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) + + # Clear any existing event loop + try: + loop = asyncio.get_event_loop() + if loop.is_running(): + asyncio.set_event_loop(None) + except RuntimeError: + pass + + # Disable PyrateLimiter async features by setting environment variable + os.environ['PYRATE_LIMITER_DISABLE_ASYNC'] = '1' + + thread_db = None + try: + thread_db = LocalDatabase() + + with app.app_context(): + socketio.emit('status_update', {'message': + 'Starting albums update from Plex...'}) + + plex_manager = PlexManager(db=thread_db) + plex_manager.populate_album_table() + + with app.app_context(): + socketio.emit('status_update', + {'message': 'Albums update completed. ' + 'Starting collections update...'}) + + all_collages = thread_db.get_all_collage_collections() + if all_collages: + logger.info('Updating %s collage collections...', len(all_collages)) + + update_collections_from_collages( + local_database=thread_db, + collage_list=all_collages, + plex_manager=plex_manager, + fetch_bookmarks=False, + fetch_mode=AlbumFetchMode.TORRENT_NAME + ) + logger.info('Collections update from collages completed!') + else: + logger.info('No stored collages found to update.') + + with app.app_context(): + socketio.emit('status_update', { + 'message': 'Albums and collections update completed successfully!', + 'finished': True + }) + + except Exception as e: + logger.critical('An unhandled error occurred during album update: %s', + e, exc_info=True) + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Error updating albums: {str(e)}', + 'error': True + }) + finally: + if thread_db: + thread_db.close() + # Clean up environment variable + os.environ.pop('PYRATE_LIMITER_DISABLE_ASYNC', None) + + # Use socketio's background task instead of threading + socketio.start_background_task(target=run_update) + + flash('Albums and collections update started!', 'info') + except Exception as e: + flash(f'Error starting albums update: {str(e)}', 'error') + + return redirect(url_for('database')) + + @app.route('/database/
/reset', methods=['POST']) + # pylint: disable=W0718 + def database_reset(table): + """Reset database table.""" + try: + db = get_db() + if table == 'albums': + db.reset_albums() + flash('Albums table reset successfully!', 'success') + elif table == 'collages': + db.reset_collage_collections() + flash('Collages table reset successfully!', 'success') + elif table == 'bookmarks': + db.reset_bookmark_collections() + flash('Bookmarks table reset successfully!', 'success') + elif table == 'remote-mappings': + db.reset_tag_mappings() + flash('Remote mappings reset successfully!', 'success') + else: + flash(f'Unknown table: {table}', 'error') + except Exception as e: + flash(f'Error resetting {table} table: {str(e)}', 'error') + + return redirect(url_for('database')) diff --git a/red_plex/infrastructure/gui/routes/remote_mappings.py b/red_plex/infrastructure/gui/routes/remote_mappings.py new file mode 100644 index 0000000..89215a0 --- /dev/null +++ b/red_plex/infrastructure/gui/routes/remote_mappings.py @@ -0,0 +1,77 @@ +"""Remote mappings route handlers.""" +import logging + +from flask import render_template, request, flash + +from red_plex.infrastructure.gui.routes.utils import ( + get_mapping_stats_and_recent, + execute_background_task) +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.use_case.site_tags.site_tags_use_case import SiteTagsUseCase + + +# pylint: disable=W0718,R0915 +def register_remote_mappings_routes(app, socketio, get_db): + """Register remote mappings-related routes.""" + + @app.route('/remote-mappings') + def remote_mappings(): + """View remote mappings.""" + stats, recent_mappings, _ = get_mapping_stats_and_recent(get_db) + return render_template('remote_mappings.html', + stats=stats, + recent_mappings=recent_mappings) + + @app.route('/remote-mappings/scan', methods=['GET', 'POST']) + def remote_mappings_scan(): + """Scan albums for remote mappings.""" + if request.method == 'POST': + try: + site = request.form.get('site') + always_skip = request.form.get('always_skip') == 'on' + + if not site: + flash('Please select a site.', 'error') + return render_template('remote_mappings_scan.html') + + # Define the scan task function + def scan_task(thread_db, web_echo): + logger = logging.getLogger('red_plex') + + logger.info("Connecting to Plex server...") + plex_manager = PlexManager(db=thread_db) + + logger.info("Updating album database from Plex...") + plex_manager.populate_album_table() + + gazelle_api = GazelleAPI(site) + site_tags_use_case = SiteTagsUseCase(thread_db, plex_manager, gazelle_api) + + def web_confirm(message): + logger.info('Auto-confirming: %s', message) + return True + + return site_tags_use_case.scan_albums_for_site_tags( + echo_func=web_echo, + confirm_func=web_confirm, + always_skip=always_skip + ) + + # Execute background task + execute_background_task( + socketio=socketio, + app=app, + task_func=scan_task, + success_message="Remote mappings scan completed successfully!", + error_prefix="Remote mappings scan" + ) + + flash('Remote mappings scan started! Check the status below.', 'info') + return render_template('remote_mappings_scan.html', + processing=True) + + except Exception as e: + flash(f'Error starting remote mappings scan: {str(e)}', 'error') + + return render_template('remote_mappings_scan.html') diff --git a/red_plex/infrastructure/gui/routes/site_tags.py b/red_plex/infrastructure/gui/routes/site_tags.py new file mode 100644 index 0000000..aae7aa8 --- /dev/null +++ b/red_plex/infrastructure/gui/routes/site_tags.py @@ -0,0 +1,70 @@ +"""Site tags route handlers.""" + +from flask import render_template, request, flash + +from red_plex.infrastructure.gui.routes.utils import ( + get_mapping_stats_and_recent, + execute_background_task) +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.use_case.site_tags.site_tags_use_case import SiteTagsUseCase + + +# pylint: disable=W0718,R0915 +def register_site_tags_routes(app, socketio, get_db): + """Register site tags-related routes.""" + + @app.route('/site-tags') + def site_tags(): + """View site tags functionality.""" + stats, recent_mappings, _ = get_mapping_stats_and_recent(get_db) + return render_template('site_tags.html', + stats=stats, + recent_mappings=recent_mappings) + + @app.route('/site-tags/convert', methods=['GET', 'POST']) + def site_tags_convert(): + """Convert tags to Plex collections.""" + if request.method == 'POST': + try: + tags = request.form.get('tags', '').strip() + collection_name = request.form.get('collection_name', '').strip() + + if not tags: + flash('Please provide tags.', 'error') + return render_template('site_tags_convert.html') + + if not collection_name: + flash('Please provide a collection name.', 'error') + return render_template('site_tags_convert.html') + + # Parse tags + tag_list = [tag.strip() for tag in tags.split(',') if tag.strip()] + + # Define the task function + def convert_task(thread_db, web_echo): + plex_manager = PlexManager(db=thread_db) + site_tags_use_case = SiteTagsUseCase(thread_db, plex_manager) + return site_tags_use_case.create_collection_from_tags( + tags=tag_list, + collection_name=collection_name, + echo_func=web_echo + ) + + # Execute background task + execute_background_task( + socketio=socketio, + app=app, + task_func=convert_task, + success_message="Collection created successfully!", + error_prefix="Collection creation" + ) + + flash('Collection creation started! ' + 'Check the log monitor below for progress.', 'info') + return render_template('site_tags_convert.html', + processing=True) + + except Exception as e: + flash(f'Error starting collection creation: {str(e)}', 'error') + + return render_template('site_tags_convert.html') diff --git a/red_plex/infrastructure/gui/routes/utils.py b/red_plex/infrastructure/gui/routes/utils.py new file mode 100644 index 0000000..b40aa21 --- /dev/null +++ b/red_plex/infrastructure/gui/routes/utils.py @@ -0,0 +1,102 @@ +"""Shared utilities for GUI routes to reduce code duplication.""" +import logging +from typing import Callable + +from flask import flash + +from red_plex.infrastructure.db.local_database import LocalDatabase + + +# pylint: disable=W0718 +def get_mapping_stats_and_recent(get_db: Callable, limit: int = 20) -> tuple: + """ + Get mapping statistics and recent mappings for display. + + Returns: + Tuple of (stats_dict, recent_mappings_list, error_occurred) + """ + try: + db = get_db() + # Get basic stats about mappings + mapped_albums, total_tags, total_mappings = db.get_site_tags_stats() + stats = { + 'mapped_albums': mapped_albums, + 'total_tags': total_tags, + 'total_mappings': total_mappings + } + + # Get recent mappings for display + recent_mappings = db.get_recent_site_tag_mappings(limit=limit) + + return stats, recent_mappings, False + + except Exception as e: + flash(f'Error loading data: {str(e)}', 'error') + default_stats = {'mapped_albums': 0, 'total_tags': 0, 'total_mappings': 0} + return default_stats, [], True + + +def execute_background_task( + socketio, + app, + task_func: Callable[[LocalDatabase, Callable], bool], + success_message: str = "Operation completed successfully!", + error_prefix: str = "Error" +) -> None: + """ + Execute a background task with standardized error handling and status updates. + + Args: + socketio: The SocketIO instance + app: The Flask app instance + task_func: Function that takes (db, echo_func) and returns success boolean + success_message: Message to emit on success + error_prefix: Prefix for error messages + """ + + def process_task(): + logger = logging.getLogger('red_plex') + thread_db = None + try: + thread_db = LocalDatabase() + + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Starting {error_prefix.lower()}...' + }) + + def web_echo(message): + logger.info(message) + with app.app_context(): + socketio.emit('status_update', {'message': message}) + + success = task_func(thread_db, web_echo) + + if success: + with app.app_context(): + socketio.emit('status_update', { + 'message': success_message, + 'finished': True + }) + else: + with app.app_context(): + socketio.emit('status_update', { + 'message': f'{error_prefix} failed.', + 'error': True + }) + + except Exception as e: + logger.critical('An unhandled error occurred during %s: %s', + error_prefix.lower(), + e, + exc_info=True) + with app.app_context(): + socketio.emit('status_update', { + 'message': f'Error: {str(e)}', + 'error': True + }) + finally: + if thread_db: + thread_db.close() + + socketio.start_background_task(target=process_task) diff --git a/red_plex/infrastructure/gui/templates/base.html b/red_plex/infrastructure/gui/templates/base.html index 967be28..0a7216f 100644 --- a/red_plex/infrastructure/gui/templates/base.html +++ b/red_plex/infrastructure/gui/templates/base.html @@ -29,6 +29,7 @@ margin-left: 250px; padding: 2rem; padding-top: 80px; + padding-bottom: 300px; /* Space for log monitor */ } .nav-link.active { background-color: #dc3545; @@ -53,6 +54,41 @@ .success-message { color: #198754; } + .log-monitor { + position: fixed; + bottom: 0; + left: 250px; + right: 0; + background: white; + border-top: 2px solid #dee2e6; + z-index: 1040; + transition: transform 0.3s ease; + } + .log-monitor.collapsed { + transform: translateY(calc(100% - 40px)); + } + .log-monitor-header { + background: #f8f9fa; + padding: 8px 15px; + border-bottom: 1px solid #dee2e6; + cursor: pointer; + user-select: none; + display: flex; + justify-content: space-between; + align-items: center; + } + .log-monitor-content { + height: 250px; + overflow-y: auto; + padding: 10px; + font-family: monospace; + font-size: 0.875rem; + background-color: #f5f5f5; + } + .log-monitor-toggle { + font-size: 1rem; + color: #6c757d; + } @@ -84,6 +120,14 @@ href="{{ url_for('bookmarks') }}"> Bookmarks + + Remote Mappings + + + Site Tags + Database @@ -108,34 +152,152 @@ {% block content %}{% endblock %} + + + {% block scripts %}{% endblock %} - \ No newline at end of file + + diff --git a/red_plex/infrastructure/gui/templates/bookmarks_convert.html b/red_plex/infrastructure/gui/templates/bookmarks_convert.html index cb80437..8b652a4 100644 --- a/red_plex/infrastructure/gui/templates/bookmarks_convert.html +++ b/red_plex/infrastructure/gui/templates/bookmarks_convert.html @@ -102,27 +102,6 @@
- {% if processing %} -
-
-
-
-
- Processing Status -
-
-
-
-
- Starting bookmark conversion... -
-
-
-
-
-
- {% endif %} -
diff --git a/red_plex/infrastructure/gui/templates/collages_convert.html b/red_plex/infrastructure/gui/templates/collages_convert.html index 28cc81e..ead16c9 100644 --- a/red_plex/infrastructure/gui/templates/collages_convert.html +++ b/red_plex/infrastructure/gui/templates/collages_convert.html @@ -129,24 +129,6 @@
Query
- {% if processing %} -
-
-
-
-
- Processing Status -
-
-
-
-
-
-
-
-
- {% endif %} -
-{% block scripts %} -{% endblock %} - {% endblock %} \ No newline at end of file diff --git a/red_plex/infrastructure/gui/templates/collages_upstream_sync.html b/red_plex/infrastructure/gui/templates/collages_upstream_sync.html new file mode 100644 index 0000000..4457572 --- /dev/null +++ b/red_plex/infrastructure/gui/templates/collages_upstream_sync.html @@ -0,0 +1,268 @@ +{% extends "base.html" %} + +{% block title %}Upstream Sync - red-plex{% endblock %} + +{% block content %} +
+
+

+ Upstream Sync +

+ + Back to Collages + +
+ + {% if processing %} + + {% elif show_confirmation and preview_data %} + +
+
+
Confirm Albums to Sync
+
+
+

+ Select which albums you want to add to the upstream collages. + By default, all albums are selected. +

+ +
+ + + + {% for item in preview_data %} +
+
+
+ {{ item.collage.name }} + + {{ item.collage.site.upper() }} + + (ID: {{ item.collage.external_id }}) +
+
+
+

+ {{ item.album_details|length }} new album(s) will be added: +

+ +
+
+ + +
+
+ +
+ {% for album in item.album_details %} +
+
+ + +
+
+ {% endfor %} +
+
+
+ {% endfor %} + +
+ + + Cancel + +
+ +
+
+ {% else %} + +
+
+
+
+
Select Collages to Sync
+
+
+

+ Choose which collages you want to sync to their upstream sources. + This will add any albums from your Plex collections that are missing from the upstream collages. +

+ + {% if collages %} +
+ + +
+
+ + +
+
+ +
+ {% for collage in collages %} +
+
+ + +
+
+ {% endfor %} +
+ +
+ +
+ + {% else %} +
+ +
No Owned Collages Found
+

You don't own any collages that can be synced upstream. Only collages you created can be synchronized to their source sites.

+ + Convert New Collages + + + Back to Collages + +
+ {% endif %} +
+
+
+
+ {% endif %} + + {% if not processing %} +
+
+ +
+
+ {% endif %} +
+ + + +{% endblock %} diff --git a/red_plex/infrastructure/gui/templates/database.html b/red_plex/infrastructure/gui/templates/database.html index cfe0ffc..8ab2aad 100644 --- a/red_plex/infrastructure/gui/templates/database.html +++ b/red_plex/infrastructure/gui/templates/database.html @@ -48,27 +48,34 @@
Database Statistics
-
+

{{ stats.get('albums', 0) }}

Albums
-
+

{{ stats.get('collages', 0) }}

Collages
-
+

{{ stats.get('bookmarks', 0) }}

Bookmarks
+
+
+ +
+

{{ stats.get('remote_mappings', 0) }}

+ Remote Mappings +
@@ -77,7 +84,7 @@

{{ stats.get('bookmarks', 0) }}

-
+
@@ -107,7 +114,7 @@
-
+
@@ -135,7 +142,7 @@
-
+
@@ -161,19 +168,31 @@
-
- - + +
+
+
+
+ Site Tags +
+

+ Map albums to site groups using album/artist names and create collections from tags. +

+ + Manage Site Tags + +
+
+
+
@@ -112,6 +129,9 @@
Convert Bookmarks + + Scan Albums for Tags +
diff --git a/red_plex/infrastructure/gui/templates/remote_mappings.html b/red_plex/infrastructure/gui/templates/remote_mappings.html new file mode 100644 index 0000000..0fdd5c1 --- /dev/null +++ b/red_plex/infrastructure/gui/templates/remote_mappings.html @@ -0,0 +1,141 @@ +{% extends "base.html" %} + +{% block title %}Remote Mappings - red-plex{% endblock %} + +{% block content %} +
+
+

Remote Mappings

+ +
+ + +
+
+
+
+
+
+
Mapped Albums
+

{{ stats.mapped_albums }}

+
+
+ +
+
+
+
+
+
+
+
+
+
+
Total Tags
+

{{ stats.total_tags }}

+
+
+ +
+
+
+
+
+
+
+
+
+
+
Total Mappings
+

{{ stats.total_mappings }}

+
+
+ +
+
+
+
+
+
+ + + {% if recent_mappings %} +
+
+
Recent Mappings
+
+
+
+
+ + + + + + + + + + {% for mapping in recent_mappings %} + + + + + + + {% endfor %} + +
Rating KeyGroup IDSiteTags
{{ mapping.rating_key }} + {{ mapping.group_id }} + + + {{ mapping.site|upper }} + + + {% if mapping.tags %} + {% for tag in mapping.tags.split(', ') %} + {{ tag }} + {% endfor %} + {% else %} + No tags + {% endif %} +
+ + + + {% else %} + + {% endif %} + + +
+
+
+
+
About Remote Mappings
+
+
+

Remote mappings are the core functionality that connects your local Plex albums to torrent groups on RED/OPS sites. This enables various features throughout red-plex:

+
    +
  • Site Tags: Create Plex collections based on site tags
    • +
  • Upstream Sync: Push your collections back to site collages
    • +
  • Metadata Enrichment: Use site metadata to organize your library
    • +
+

+ Getting Started: Click "Scan Albums" to begin mapping your Plex library to site content. + This process searches for your albums on the selected site and creates the necessary mappings. + Once you have mappings, you can use the Site Tags feature to create collections. +

+
+
+
+
+ +{% endblock %} \ No newline at end of file diff --git a/red_plex/infrastructure/gui/templates/remote_mappings_scan.html b/red_plex/infrastructure/gui/templates/remote_mappings_scan.html new file mode 100644 index 0000000..b56f40f --- /dev/null +++ b/red_plex/infrastructure/gui/templates/remote_mappings_scan.html @@ -0,0 +1,80 @@ +{% extends "base.html" %} + +{% block title %}Scan Remote Mappings - red-plex{% endblock %} + +{% block content %} +
+
+

Scan Remote Mappings

+ + Back to Remote Mappings + +
+ + {% if not processing %} +
+
+
+
+
Scan Configuration
+
+
+
+
+ + +
Choose which site to scan against for mapping creation
+
+ +
+ + +
When enabled, albums with multiple search results will be automatically skipped without prompting
+
+ + +
+
+
+
+ +
+
+
+
Scan Process
+
+
+

The scan process will:

+
    +
  1. Identify unscanned albums in your Plex library
    2. +
  2. Search for each album on the selected site
    3. +
  3. Create mappings between Plex items and site groups
    4. +
  4. Store associated tags for collection creation
    5. +
+ +
+
+
+
+ {% endif %} +
+ + + +{% endblock %} \ No newline at end of file diff --git a/red_plex/infrastructure/gui/templates/site_tags.html b/red_plex/infrastructure/gui/templates/site_tags.html new file mode 100644 index 0000000..e223da3 --- /dev/null +++ b/red_plex/infrastructure/gui/templates/site_tags.html @@ -0,0 +1,97 @@ +{% extends "base.html" %} + +{% block title %}Site Tags{% endblock %} + +{% block content %} +
+

Site Tags

+ +
+
+
+
+
Overview
+
+
+

Site tags functionality allows you to create Plex collections from albums that have been tagged on RED/OPS sites.

+ +
Statistics
+
    +
  • Albums with mappings: {{ stats.mapped_albums }}
    • +
  • Total tags: {{ stats.total_tags }}
    • +
  • Total mappings: {{ stats.total_mappings }}
    • +
+ +
+ + Convert Tags to Collection + +
+
+
+
+ +
+
+
+
How it Works
+
+
+
    +
  1. Scan albums: Use red-plex db remote-mappings scan to create mappings between your Plex albums and site group IDs
    2. +
  2. Tags are collected: During the scan process, tags from the site are automatically stored
    3. +
  3. Convert to collections: Use this interface or CLI to create Plex collections based on specific tags
    4. +
+ +
+ + Prerequisites: You need to have remote mappings data first. Use the Remote Mappings section to scan your albums. +
+
+
+
+
+ + {% if recent_mappings %} +
+
+
+
+
Recent Tagged Albums
+
+
+
+ + + + + + + + + + + {% for mapping in recent_mappings %} + + + + + + + {% endfor %} + +
AlbumArtist(s)SiteTags
{{ mapping.album_name }}{{ mapping.artists }}{{ mapping.site.upper() }} + {% if mapping.tags %} + {{ mapping.tags[:50] }}{% if mapping.tags|length > 50 %}...{% endif %} + {% else %} + No tags + {% endif %} +
+
+
+
+
+
+ {% endif %} +
+{% endblock %} \ No newline at end of file diff --git a/red_plex/infrastructure/gui/templates/site_tags_convert.html b/red_plex/infrastructure/gui/templates/site_tags_convert.html new file mode 100644 index 0000000..0aa8998 --- /dev/null +++ b/red_plex/infrastructure/gui/templates/site_tags_convert.html @@ -0,0 +1,84 @@ +{% extends "base.html" %} + +{% block title %}Convert Tags to Collection{% endblock %} + +{% block content %} +
+

Convert Tags to Collection

+

Create a Plex collection from albums matching specific tags.

+ + + +
+
+
+
+
Collection Settings
+
+
+
+
+ + +
+ Enter comma-separated tags to filter albums by. Only albums with these tags will be included. +
+
+ +
+ + +
+ Name for the Plex collection. If a collection with this name exists, it will be updated. +
+
+ + + + Back to Site Tags + +
+
+
+
+ +
+
+
+
How it Works
+
+
+
    +
  1. Enter tags: Specify the tags you want to filter by
    2. +
  2. Name the collection: Choose a name for your Plex collection
    3. +
  3. Processing: The system will find all albums matching those tags
    4. +
  4. Collection created: A new collection will be created in Plex with the matching albums
    5. +
+ +
+ + Note: Only albums that have been scanned and have remote mappings will be included. +
+
+
+
+
+
+{% endblock %} \ No newline at end of file diff --git a/red_plex/infrastructure/plex/plex_manager.py b/red_plex/infrastructure/plex/plex_manager.py index 0d8ae84..b43f72e 100644 --- a/red_plex/infrastructure/plex/plex_manager.py +++ b/red_plex/infrastructure/plex/plex_manager.py @@ -2,6 +2,7 @@ import os import re +import time from datetime import datetime, timezone from typing import List from typing import Optional @@ -10,16 +11,21 @@ from plexapi.audio import Album as PlexAlbum from plexapi.base import MediaContainer from plexapi.collection import Collection as PlexCollection +from plexapi.exceptions import PlexApiException from plexapi.library import MusicSection from plexapi.server import PlexServer +from requests.exceptions import (ConnectionError as RequestsConnectionError, + Timeout, RequestException) from red_plex.domain.models import Collection, Album from red_plex.infrastructure.config.config import load_config +from red_plex.infrastructure.constants.constants import ALBUM_TAGS from red_plex.infrastructure.db.local_database import LocalDatabase from red_plex.infrastructure.logger.logger import logger from red_plex.infrastructure.plex.mapper.plex_mapper import PlexMapper +# pylint: disable=W0718 class PlexManager: """Handles operations related to Plex.""" @@ -30,7 +36,8 @@ def __init__(self, db: LocalDatabase): self.url = config_data.plex_url self.token = config_data.plex_token self.section_name = config_data.section_name - self.plex = PlexServer(self.url, self.token, timeout=1200) + # Increase timeout to 30 minutes for large operations + self.plex = PlexServer(self.url, self.token, timeout=1800) self.library_section: MusicSection self.library_section = self.plex.library.section(self.section_name) @@ -39,6 +46,44 @@ def __init__(self, db: LocalDatabase): self.local_database = db self.album_data = self.local_database.get_all_albums() + def _retry_with_backoff(self, func, max_retries=3, base_delay=1): + """Retry a function with exponential backoff.""" + for attempt in range(max_retries): + try: + return func() + except (RequestsConnectionError, Timeout, RequestException, PlexApiException) as e: + if attempt == max_retries - 1: + raise e + delay = base_delay * (2 ** attempt) + logger.warning("Attempt %d failed: %s. Retrying in %d seconds...", + attempt + 1, e, delay) + time.sleep(delay) + return None # Add explicit return for consistency + + def _get_album_path_safely(self, album: PlexAlbum) -> Optional[str]: + """Safely get album path with retry logic.""" + + def _get_tracks(): + return album.tracks() + + try: + tracks = self._retry_with_backoff(_get_tracks) + if tracks and tracks[0].media and tracks[0].media[0].parts: + media_path = tracks[0].media[0].parts[0].file + return os.path.dirname(media_path) + except Exception as e: + logger.warning("Failed to get path for album %s (ID: %s): %s", + album.title, album.ratingKey, e) + + # Fallback: try to construct path from album metadata if available + try: + if hasattr(album, 'locations') and album.locations: + return album.locations[0] + except Exception: + pass + + return None + def populate_album_table(self): """Fetches new albums from Plex and updates the db.""" logger.info('Updating album db...') @@ -64,25 +109,63 @@ def populate_album_table(self): def get_albums_given_filter(self, plex_filter: dict) -> List[Album]: """Returns a list of albums that match the specified filter.""" - albums: List[PlexAlbum] + + def _search_albums(): + return self.library_section.searchAlbums(filters=plex_filter) + try: - albums = self.library_section.searchAlbums(filters=plex_filter) - except Exception as e: # pylint: disable=W0718 - logger.warning('An error occurred while fetching albums given filter: %s', e) + albums = self._retry_with_backoff(_search_albums) + except Exception as e: + logger.error('Failed to fetch albums after retries: %s', e) return [] - domain_albums: List[Album] - domain_albums = [] - for album in albums: - tracks = album.tracks() - if tracks: - media_path = tracks[0].media[0].parts[0].file - album_folder_path = os.path.dirname(media_path) - domain_albums.append(Album(id=album.ratingKey, - name=album.title, - added_at=album.addedAt, - path=album_folder_path)) + + domain_albums: List[Album] = [] + batch_size = 100 # Process albums in batches + total_albums = len(albums) + + for i, album in enumerate(albums): + try: + # Log progress every 100 albums + if i % batch_size == 0: + logger.info('Processing album %d/%d: %s', i + 1, total_albums, album.title) + + album_path = self._get_album_path_safely(album) + + # Create album even if path is None (we'll skip those with None paths later) + domain_album = Album( + id=album.ratingKey, + name=album.title, + artists=[album.parentTitle] if album.parentTitle else [], + added_at=album.addedAt, + path=album_path + ) + + # Only add albums that have a valid path + if album_path: + domain_albums.append(domain_album) + else: + logger.warning("Skipping album '%s' - no valid path found", album.title) + + # Small delay every 50 albums to be nice to the server + if i % 50 == 0 and i > 0: + time.sleep(0.1) + + except Exception as e: + logger.warning('Error processing album %s (ID: %s): %s', + album.title, album.ratingKey, e) + continue + + logger.info('Successfully processed %d out of %d albums', + len(domain_albums), total_albums) return domain_albums + def get_album_by_rating_key(self, rating_key: int) -> Optional[Album]: + """ Queries Plex for the album that matches the given rating_key """ + album = self.library_section.fetchItem(rating_key) + if album: + return PlexMapper.map_plex_album_to_domain(album) + return None + # If multiple matches are found, prompt the user to choose def query_for_albums(self, album_name: str, artists: List[str]) -> List[Album]: """Queries Plex for the rating keys of albums that match the given name and artists.""" @@ -131,7 +214,7 @@ def query_for_albums(self, album_name: str, artists: List[str]) -> List[Album]: "Invalid input. Please enter valid numbers separated by commas or 'A' for all, " "'N' to select none." ) - except Exception as e: # pylint: disable=W0718 + except Exception as e: logger.warning('An error occurred while searching for albums: %s', e) return [] @@ -204,27 +287,78 @@ def find_matching_rating_keys(self, path): return matched_rating_keys def _fetch_albums_by_keys(self, albums: List[Album]) -> List[MediaContainer]: - """Fetches album objects from Plex using their rating keys.""" - logger.debug('Fetching albums from Plex: %s', albums) - rating_keys = [int(album.id) for album in albums] - try: - fetched_albums = self.plex.fetchItems(rating_keys) - except Exception as e: # pylint: disable=W0718 - logger.warning('An error occurred while fetching albums: %s', e) + """Fetches album objects from Plex in batches using their rating keys.""" + if not albums: return [] - return fetched_albums + + logger.debug('Preparing to fetch %d albums from Plex.', len(albums)) + rating_keys = [int(album.id) for album in albums] + all_fetched_albums = [] + batch_size = 1000 + + for i in range(0, len(rating_keys), batch_size): + batch_keys = rating_keys[i:i + batch_size] + logger.debug('Fetching batch of %d albums, starting from index %d.', len(batch_keys), i) + + try: + # The API call and error handling are now in one place + fetched_batch = self.plex.fetchItems(batch_keys) + if fetched_batch: + all_fetched_albums.extend(fetched_batch) + else: + logger.warning('No albums found for rating keys in this batch.') + except Exception as e: + logger.warning('An error occurred while fetching a batch of albums: %s', e) + + logger.debug('Finished fetching. Total albums retrieved: %d.', len(all_fetched_albums)) + return all_fetched_albums def create_collection(self, name: str, albums: List[Album]) -> Optional[Collection]: - """Creates a collection in Plex.""" - logger.info('Creating collection with name "%s" and %d albums.', name, len(albums)) - albums_media = self._fetch_albums_by_keys(albums) + """ + Creates a collection in Plex, adding albums in batches to avoid request limits. + """ + if not albums: + logger.warning('Cannot create a collection with no albums.') + return None + logger.info('Creating collection "%s" with %d total albums.', name, len(albums)) + batch_size = 1000 + + # Step 1: Create the collection with the first batch of albums. + first_batch = albums[:batch_size] + albums_media = self._fetch_albums_by_keys(first_batch) + + if not albums_media: + logger.error('Failed to fetch initial batch of albums. Aborting collection creation.') + return None + created_collection: Optional[Collection] = None try: - collection = self.library_section.createCollection(name, items=albums_media) - except Exception as e: # pylint: disable=W0718 - logger.warning('An error occurred while creating the collection: %s', e) + logger.debug('Creating collection with the first %d items.', len(albums_media)) + plex_collection_obj = self.library_section.createCollection(name, items=albums_media) + created_collection = PlexMapper.map_plex_collection_to_domain(plex_collection_obj) + except Exception as e: + logger.error('An error occurred while creating the collection ' + 'with the first batch: %s', e) return None - return PlexMapper.map_plex_collection_to_domain(collection) + + # Step 2: If there are more albums, add them in subsequent batches. + if len(albums) > batch_size: + for i in range(batch_size, len(albums), batch_size): + remaining_batch = albums[i:i + batch_size] + logger.debug('Adding batch of %d albums to collection "%s".', + len(remaining_batch), name) + try: + self.add_items_to_collection(created_collection, + remaining_batch) + except Exception as e: + logger.warning( + 'Failed to add a batch to collection "%s". ' + 'The collection may be incomplete. Error: %s', + name, e + ) + + logger.info('Successfully finished processing collection "%s".', name) + return created_collection def get_collection_by_name(self, name: str) -> Optional[Collection]: """Finds a collection by name.""" @@ -232,7 +366,7 @@ def get_collection_by_name(self, name: str) -> Optional[Collection]: try: collection = self.library_section.collection(name) # pylint: disable=broad-except - # pylint: disable=W0718 + except Exception: # If the collection doesn't exist, collection will be set to None collection = None @@ -244,6 +378,19 @@ def get_collection_by_name(self, name: str) -> Optional[Collection]: logger.info('No existing collection found with name "%s" in Plex.', name) return None + def get_collection_by_rating_key(self, rating_key: str) -> Optional[PlexCollection]: + """Finds a Plex collection by rating key.""" + try: + collection = self.library_section.fetchItem(int(rating_key)) + if collection and hasattr(collection, 'TYPE') and collection.TYPE == 'collection': + return collection + logger.warning('Item with rating key "%s" is not a collection ' + 'or does not exist.', rating_key) + return None + except Exception as e: + logger.error('Error fetching collection with rating key "%s": %s', rating_key, e) + return None + def add_items_to_collection(self, collection: Collection, albums: List[Album]) -> None: """Adds albums to an existing collection.""" logger.debug('Adding %d albums to collection "%s".', len(albums), collection.name) @@ -251,7 +398,7 @@ def add_items_to_collection(self, collection: Collection, albums: List[Album]) - collection_from_plex: Optional[PlexCollection] try: collection_from_plex = self.library_section.collection(collection.name) - except Exception as e: # pylint: disable=W0718 + except Exception as e: logger.warning('An error occurred while trying to fetch the collection: %s', e) collection_from_plex = None if collection_from_plex: @@ -259,8 +406,7 @@ def add_items_to_collection(self, collection: Collection, albums: List[Album]) - else: logger.warning('Collection "%s" not found.', collection.name) - @staticmethod - def _get_album_transformations(album_name: str) -> List[str]: + def _get_album_transformations(self, album_name: str) -> List[str]: """ Returns a list of album name transformations for use in Plex queries, increasing the chances of a successful match. @@ -274,14 +420,7 @@ def _get_album_transformations(album_name: str) -> List[str]: if not album_name: return [] - tags = [ - "EP", "E.P", "E.P.", "Single", "Album", "Soundtrack", "Anthology", - "Compilation", "Live Album", "Remix", "Bootleg", "Interview", - "Mixtape", "Demo", "Concert Recording", "DJ Mix", "Original Mix", - "Remastered", "Deluxe Edition", "Limited Edition", "Bonus Track", - "Instrumental", "Acapella" - ] - suffixes = sorted(tags, key=len, reverse=True) + suffixes = sorted(ALBUM_TAGS, key=len, reverse=True) transforms = [album_name] seen = {album_name.lower()} diff --git a/red_plex/infrastructure/rest/gazelle/gazelle_api.py b/red_plex/infrastructure/rest/gazelle/gazelle_api.py index 6db226e..622f2a3 100644 --- a/red_plex/infrastructure/rest/gazelle/gazelle_api.py +++ b/red_plex/infrastructure/rest/gazelle/gazelle_api.py @@ -1,21 +1,25 @@ """Module for interacting with Gazelle-based APIs.""" import asyncio +import re import time +import unicodedata from inspect import isawaitable -from typing import Dict, Any, Optional +from typing import Dict, Any, Optional, List import requests from pyrate_limiter import Limiter, Rate, Duration from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_fixed +from thefuzz import process, fuzz from red_plex.domain.models import Collection, TorrentGroup from red_plex.infrastructure.config.config import load_config +from red_plex.infrastructure.constants.constants import ALBUM_TAGS, VARIOUS_ARTISTS_TAGS from red_plex.infrastructure.logger.logger import logger from red_plex.infrastructure.rest.gazelle.mapper.gazelle_mapper import GazelleMapper -# pylint: disable=W0718 +# pylint: disable=W0718,R0914,R0913,R0917,R0912 class GazelleAPI: """Handles API interactions with Gazelle-based services.""" @@ -38,34 +42,60 @@ def __init__(self, site: str): self.rate_limit = rate_limit # Store rate_limit for calculations self.limiter = Limiter(rate_limit, raise_when_fail=False) + def _wait_for_rate_limit(self): + """Handle rate limiting by waiting if necessary.""" + while True: + did_acquire = self.limiter.try_acquire('api_call') + if did_acquire: + return + delay_ms = self.get_retry_after() + delay_seconds = delay_ms / 1000.0 + if delay_seconds > 0.001: + logger.debug('Rate limit exceeded. Sleeping for %.2f seconds.', delay_seconds) + time.sleep(delay_seconds) + else: + time.sleep(0.001) + @retry( retry=retry_if_exception_type(requests.exceptions.RequestException), stop=stop_after_attempt(5), wait=wait_fixed(4), reraise=True ) - def api_call(self, action: str, params: Dict[str, str]) -> Dict[str, Any]: + def get_call(self, action: str, params: Dict[str, str]) -> Dict[str, Any]: """ - Makes a rate-limited API call to the Gazelle-based service with retries. + Makes a rate-limited GET API call to the Gazelle-based service with retries. Rate limit is handled in a loop, while network/HTTP errors trigger a retry. """ formatted_params = '&' + '&'.join(f'{k}={v}' for k, v in params.items()) if params else '' formatted_url = f'{self.base_url}{action}{formatted_params}' - logger.debug('Calling API: %s', formatted_url) + logger.debug('Calling GET API: %s', formatted_url) - while True: - did_acquire = self.limiter.try_acquire('api_call') - if did_acquire: - response = requests.get(formatted_url, headers=self.headers, timeout=10) - response.raise_for_status() - return response.json() - delay_ms = self.get_retry_after() - delay_seconds = delay_ms / 1000.0 - if delay_seconds > 0.001: - logger.debug('Rate limit exceeded. Sleeping for %.2f seconds.', delay_seconds) - time.sleep(delay_seconds) - else: - time.sleep(0.001) + self._wait_for_rate_limit() + response = requests.get(formatted_url, headers=self.headers, timeout=10) + response.raise_for_status() + return response.json() + + @retry( + retry=retry_if_exception_type(requests.exceptions.RequestException), + stop=stop_after_attempt(5), + wait=wait_fixed(4), + reraise=True + ) + def post_call(self, action: str, + params: Dict[str, str] = None, + data: Dict[str, str] = None) -> Dict[str, Any]: + """ + Makes a rate-limited POST API call to the Gazelle-based service with retries. + Rate limit is handled in a loop, while network/HTTP errors trigger a retry. + """ + url = f'{self.base_url}{action}' + logger.debug('Calling POST API: %s', url) + + self._wait_for_rate_limit() + response = requests.post(url, headers=self.headers, params=params, data=data, timeout=10) + response.raise_for_status() + return response.json() def get_retry_after(self) -> int: """Calculates the time to wait until another request can be made.""" @@ -104,7 +134,7 @@ def get_collage(self, collage_id: str) -> Optional[Collection]: """Retrieves collage data as domain object""" params = {'id': str(collage_id), 'showonlygroups': 'true'} try: - json_data = self.api_call('collage', params) + json_data = self.get_call('collage', params) except Exception as e: # pylint: disable=W0703 logger.error('Error retrieving collage data for collage_id %s: %s', collage_id, e) return None @@ -115,7 +145,7 @@ def get_torrent_group(self, torrent_group_id: str) -> Optional[TorrentGroup]: """Retrieves torrent group data.""" params = {'id': str(torrent_group_id)} try: - json_data = self.api_call('torrentgroup', params) + json_data = self.get_call('torrentgroup', params) except Exception as e: # pylint: disable=W0703 logger.error('Error retrieving torrent group data ' 'for group_id %s: %s', torrent_group_id, e) @@ -129,9 +159,242 @@ def get_bookmarks(self, site: str) -> Optional[Collection]: """Retrieves user bookmarks.""" logger.debug('Retrieving user bookmarks...') try: - bookmarks_response = self.api_call('bookmarks', {}) + bookmarks_response = self.get_call('bookmarks', {}) except Exception as e: # pylint: disable=W0703 logger.error('Error retrieving user bookmarks: %s', e) return None logger.debug('Retrieved user bookmarks') return GazelleMapper.map_bookmarks(bookmarks_response, site) + + def _fetch_groups_from_api(self, album_name: str, artists: List[str]) -> List[TorrentGroup]: + """ + Helper method to fetch torrent groups from the API for a given album and artists. + """ + found_groups = [] + params = {} + for artist in artists: + # If the artist name is 'various artists', we skip it + if artist.lower() in VARIOUS_ARTISTS_TAGS: + params = {'groupname': album_name} + else: + params = {'groupname': album_name, 'artistname': artist} + try: + response = self.get_call('browse', params) + results = response.get('response', {}).get('results', []) + if results: + domain_tgs = [GazelleMapper.map_torrent_group(tg) for tg in results] + found_groups.extend(domain_tgs) + except Exception as e: + logger.error('Error during API call for [%s]-[%s]: %s', album_name, artist, e) + # We continue here instead of returning None to be more resilient + return found_groups + + def _get_fallback_album_name(self, album_name: str) -> str: + """ + Cleans an album name by removing tags and parenthesized content for a fallback search. + """ + # 1: Remove content in parentheses + clean_name = re.sub(r'\s*\([^)]*\)', '', album_name) + + # 2: Remove dots and commas from the result + clean_name = re.sub(r'[.,]', '', clean_name).strip() + + # 3. Remove common tags like EP, Single, etc. + # This regex looks for whole words at the end of the string + tag_pattern = r'\s+\b(?:' + '|'.join(re.escape(tag) for tag in ALBUM_TAGS) + r')\b$' + clean_name = re.sub(tag_pattern, '', clean_name, flags=re.IGNORECASE).strip() + + return clean_name + + def browse_by_album_and_artist_names(self, + album_name: str, + artists: List[str]) -> Optional[List[TorrentGroup]]: + """ + Searches for torrents by finding the best fuzzy match from all possible results, + including an initial, a fallback, and an album-only search. + """ + logger.debug('Initiating search for album [%s] and artists [%s]', album_name, artists) + + # --- PHASE 1: Comprehensive Data Fetching --- + + # 1. Initial search with the original album name and artists + initial_groups = self._fetch_groups_from_api(album_name, artists) + + # 2. Fallback search with the cleaned album name and artists + fallback_album_name = self._get_fallback_album_name(album_name) + fallback_groups = [] + if fallback_album_name.lower() != album_name.lower(): + logger.debug("Performing fallback search with cleaned name: [%s]", fallback_album_name) + fallback_groups = self._fetch_groups_from_api(fallback_album_name, artists) + + # 3. Combine initial and fallback results, removing duplicates + combined_groups = {group.id: group for group in initial_groups} + for group in fallback_groups: + combined_groups[group.id] = group + + all_possible_groups = list(combined_groups.values()) + + # --- PHASE 1.5: Final Fallback Search (Album Name Only) --- + # If we still haven't found anything, try one last time searching only by album name. + if not all_possible_groups: + logger.info("No results found. Trying a final search with album name only.") + try: + params = {'groupname': album_name} + response = self.get_call('browse', params) + results = response.get('response', {}).get('results', []) + if results: + all_possible_groups = [GazelleMapper.map_torrent_group(tg) for tg in results] + except Exception as e: + logger.error('Error during album-only fallback search for [%s]: %s', album_name, e) + return None # Fail on API error + + # If after all attempts there are still no results, exit. + if not all_possible_groups: + logger.debug("No potential matches found after all searches.") + return [] + + # --- PHASE 2: Single Fuzzy Matching Stage on All Collected Results --- + + choices: Dict[str, TorrentGroup] = {} + for group in all_possible_groups: + artist_str = ', '.join(group.artists) if group.artists else '' + choice_string = (f"{self._normalize_string(artist_str)} - " + f"{self._normalize_string(group.album_name)}") + choices[choice_string] = group + + overall_best_match = None + highest_score = 0 + + if len(artists) > 1: + artist_candidates = artists + ['Various Artists'] + else: + artist_candidates = artists + + for artist_variation in artist_candidates: + query_string = (f"{self._normalize_string(artist_variation)} - " + f"{self._normalize_string(album_name)}") + current_match = process.extractOne(query_string, + choices.keys(), + scorer=fuzz.token_set_ratio) + if current_match: + _, current_score = current_match + if current_score > highest_score: + highest_score = current_score + overall_best_match = current_match + + if not overall_best_match: + logger.info("Fuzzy matching could not determine a best match from all results.") + return all_possible_groups + + best_choice_str, score = overall_best_match + confidence_threshold = 90 + logger.info("Highest fuzzy match score found: '%s' with %d%%", best_choice_str, score) + + if score >= confidence_threshold: + logger.info("Confidence score is high. Returning the best match.") + best_group = choices[best_choice_str] + return [best_group] + + logger.info("Best match score is below threshold. Returning all potential results.") + return all_possible_groups + + def get_user_info(self) -> Optional[Dict[str, Any]]: + """Retrieves current user information including user ID.""" + try: + response = self.get_call('index', {}) + if response.get('status') == 'success': + return response.get('response', {}) + logger.error('Failed to get user info: %s', response) + return None + except Exception as e: + logger.error('Error retrieving user info: %s', e) + return None + + def get_user_collages(self, user_id: str) -> Optional[List[Collection]]: + """Retrieves collages created by the specified user.""" + if self.site.lower() != 'red': + logger.debug('get_user_collages is only supported for RED.') + return None + params = {'userid': str(user_id)} + try: + response = self.get_call('collages', params) + if response.get('status') == 'success': + collages_data = response.get('response', []) + # Map to Collection domain objects + collections = [] + for collage_dict in collages_data: + collection = Collection( + id="", # No local ID yet + external_id=str(collage_dict.get('id', '')), + name=collage_dict.get('name', ''), + torrent_groups=[], + site=self.site.lower() + ) + collections.append(collection) + return collections + logger.error('Failed to get user collages: %s', response) + return None + except Exception as e: + logger.error('Error retrieving user collages for user_id %s: %s', user_id, e) + return None + + def add_to_collage(self, collage_id: str, + group_ids: List[str]) -> Optional[Dict[str, Any]]: + """ + Adds group IDs to a collage. + + Args: + collage_id: The ID of the collage to add to + group_ids: List of group IDs to add + + Returns: + Response dict containing status and results, or None on error + """ + if not group_ids: + logger.warning('No group IDs provided to add to collage %s', collage_id) + return None + + # Format group IDs as comma-separated string + group_ids_str = ','.join(group_ids) + + params = {'collageid': str(collage_id)} + data = {'groupids': group_ids_str} + + logger.debug('Adding groups %s to collage %s', group_ids_str, collage_id) + + try: + result = self.post_call('addtocollage', params, data) + + if result.get('status') == 'success': + logger.info('Successfully added groups to collage ' + '%s: added=%s, rejected=%s, duplicated=%s', + collage_id, + result.get('response', {}).get('groupsadded', []), + result.get('response', {}).get('groupsrejected', []), + result.get('response', {}).get('groupsduplicated', [])) + return result + + except Exception as e: + logger.error('Error adding groups %s to collage %s: %s', group_ids_str, collage_id, e) + return None + + @staticmethod + def _normalize_string(text: str) -> str: + """ + Normalizes a string to a simple, comparable form. + """ + if not text: + return "" + + # Step 1: Replace dashes + text = text.replace('–', '-').replace('—', '-') + + # Step 2: Normalize unicode characters (accents, etc.) + text = unicodedata.normalize('NFKD', text).encode('ascii', 'ignore').decode('ascii') + + # Step 3: Normalize whitespace + # Replaces multiple spaces with a single space and strips ends + text = re.sub(r'\s+', ' ', text).strip() + + # Step 4: Convert to lowercase + return text.lower() diff --git a/red_plex/infrastructure/rest/gazelle/mapper/gazelle_mapper.py b/red_plex/infrastructure/rest/gazelle/mapper/gazelle_mapper.py index e8073bf..9ad40b7 100644 --- a/red_plex/infrastructure/rest/gazelle/mapper/gazelle_mapper.py +++ b/red_plex/infrastructure/rest/gazelle/mapper/gazelle_mapper.py @@ -45,23 +45,32 @@ def map_torrent_group(data: Dict[str, Any], torrents: List[Dict] = None) -> Torr person_roles = [ 'artists', 'composers', 'conductor', 'dj', 'producer', 'arranger', 'remixedBy' - 'with' + 'with' ] return TorrentGroup( - id=data.get('id') or group_data.get('id'), + id=data.get('id') + or data.get('groupId') + or group_data.get('id') + or group_data.get('groupId'), album_name=GazelleMapper._clean_text( - data.get('name') or group_data.get('name') or '' + data.get('name') + or data.get('groupName') + or group_data.get('name') + or group_data.get('groupName') + or '' ), artists=[ - GazelleMapper._clean_text(person.get('name', '')) - for role in person_roles - for person in music_info.get(role, []) - ], + GazelleMapper._clean_text(person.get('name', '')) + for role in person_roles + for person in music_info.get(role, []) + ] or ([data.get('artist')] if data.get('artist') else []), - file_paths=GazelleMapper._map_torrent_group_file_paths(torrents) + file_paths=GazelleMapper._map_torrent_group_file_paths(torrents), + + tags=data.get('tags') or group_data.get('tags') or [] ) @staticmethod diff --git a/red_plex/use_case/create_collection/query/query_sync_collection.py b/red_plex/use_case/create_collection/query/query_sync_collection.py index 5d5d098..790d5b2 100644 --- a/red_plex/use_case/create_collection/query/query_sync_collection.py +++ b/red_plex/use_case/create_collection/query/query_sync_collection.py @@ -57,6 +57,14 @@ def execute( # 4. Create or update if albums were found if not matched_albums: + # Create the entry in the database with no albums, for sync matching, for example. + if plex_collection: + found_collection = Collection(id=plex_collection.id, + name=plex_collection.name, + external_id=collage_id, + site=site) + self._save_to_db(found_collection, fetch_bookmarks) + # No new albums found or nothing to do. return CreateCollectionResponse(response_status=True, collection_data=source_collection, diff --git a/red_plex/use_case/create_collection/torrent_name/torrent_name_sync_collection.py b/red_plex/use_case/create_collection/torrent_name/torrent_name_sync_collection.py index 350bbb7..a16f472 100644 --- a/red_plex/use_case/create_collection/torrent_name/torrent_name_sync_collection.py +++ b/red_plex/use_case/create_collection/torrent_name/torrent_name_sync_collection.py @@ -59,6 +59,13 @@ def execute( # 4. Create or update if albums were found if not matched_rating_keys: + # Create the entry in the database with no albums, for sync matching, for example. + if plex_collection: + found_collection = Collection(id=plex_collection.id, + name=plex_collection.name, + external_id=collage_id, + site=site) + self._save_to_db(found_collection, fetch_bookmarks) # No new albums found or nothing to do. return CreateCollectionResponse(response_status=True, collection_data=source_collection, diff --git a/red_plex/use_case/site_tags/site_tags_use_case.py b/red_plex/use_case/site_tags/site_tags_use_case.py new file mode 100644 index 0000000..cd1c2fc --- /dev/null +++ b/red_plex/use_case/site_tags/site_tags_use_case.py @@ -0,0 +1,237 @@ +"""Use case for managing site tag mappings and collections.""" + +import re +from typing import List, Optional, Callable + +import click + +from red_plex.domain.models import Album, TorrentGroup +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.logger.logger import logger +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI + + +# pylint: disable=R0912,W0718,W0613,R0913,R0917 +class SiteTagsUseCase: + """Use case for managing site tag mappings and creating collections from tags.""" + + def __init__(self, local_database: LocalDatabase, + plex_manager: PlexManager, + gazelle_api: GazelleAPI = None): + self.local_database = local_database + self.plex_manager = plex_manager + self.gazelle_api = gazelle_api + + def scan_albums_for_site_tags(self, echo_func: Callable[[str], None], + confirm_func: Callable[[str], bool], + always_skip: bool = False): + """ + Scan albums and create site tag mappings by searching filenames on the site. + """ + site = self.gazelle_api.site + echo_func(f"Starting scan for site: {site}") + + # Get unscanned albums for this site + unscanned_rating_keys = self.local_database.get_unscanned_albums() + + if not unscanned_rating_keys: + echo_func("No unscanned albums found.") + return + + echo_func(f"Found {len(unscanned_rating_keys)} unscanned albums.") + + processed_count = 0 + success_count = 0 + + for rating_key in unscanned_rating_keys: + try: + processed_count += 1 + # Fetch album from Plex + domain_album = self.plex_manager.get_album_by_rating_key(int(rating_key)) + if not domain_album: + echo_func(f" Album with rating key {rating_key} not found in Plex.") + continue + echo_func( + f"Processing album {processed_count}/{len(unscanned_rating_keys)}: " + f"Rating Key: {rating_key}. " + f"{domain_album.artists[0] if domain_album.artists else ''}" + f" - {domain_album.name}" + ) + + match_data = self._search_by_album_and_artist_names(album_name=domain_album.name, + artists=domain_album.artists) + + if match_data: + # Process the match + if self._process_search_results(rating_key=rating_key, + torrent_groups=match_data, + domain_album=domain_album, + echo_func=echo_func, + confirm_func=confirm_func, + always_skip=always_skip): + success_count += 1 + + except Exception as e: + logger.exception("Error processing album %s: %s", rating_key, e) + echo_func(f" Error processing album {rating_key}: {e}") + + echo_func(f"Scan completed. Processed: {processed_count}, Successful: {success_count}") + + def _search_by_album_and_artist_names(self, + album_name: str, + artists: List[str]) -> Optional[List[TorrentGroup]]: + """Search torrent groups matching the album and artist names.""" + try: + if (album_name is None or album_name == '') or (artists is None or artists == []): + return None + artists = self._parse_plex_artists(artists[0]) + response = self.gazelle_api.browse_by_album_and_artist_names(album_name, artists) + return response + except Exception: + return None + + def _process_search_results(self, rating_key: str, torrent_groups: List[TorrentGroup], + domain_album: Album, + echo_func: Callable[[str], None], + confirm_func: Callable[[str], bool], + always_skip: bool = False) -> bool: + """Process search results and handle user confirmation for multiple matches.""" + if not torrent_groups: + return False + + if len(torrent_groups) == 1: + # Single match, process it + torrent_group = torrent_groups[0] + return self._create_site_tag_mapping(rating_key, torrent_group, echo_func) + + if always_skip: + echo_func(f" Skipping due to multiple matches ({len(torrent_groups)}) " + f"for album [{domain_album.name}] and --always-skip flag.") + return False + + # Multiple matches, ask user to choose + echo_func( + f" Found {len(torrent_groups)} matches for album name " + f"[{domain_album.name}] and artists {domain_album.artists}:") + for i, torrent_group in enumerate(torrent_groups): + artists = torrent_group.artists + if isinstance(artists, list): + artists = ', '.join(artists) + group_name = torrent_group.album_name + echo_func(f" {i + 1}. {artists} - {group_name}") + + while True: + try: + choice = click.prompt( + f" Choose a match (1-{len(torrent_groups)}) or 's' to skip", + type=str).strip().lower() + if choice == 's': + return False + choice_idx = int(choice) - 1 + if 0 <= choice_idx < len(torrent_groups): + return self._create_site_tag_mapping(rating_key, + torrent_groups[choice_idx], + echo_func) + echo_func(" Invalid choice. Please try again.") + except (ValueError, click.Abort): + echo_func(" Invalid input. Please try again.") + except KeyboardInterrupt: + echo_func(" Scan interrupted by user.") + return False + + def _create_site_tag_mapping(self, rating_key: str, torrent_group: TorrentGroup, + echo_func: Callable[[str], None]) -> bool: + """Create a site tag mapping from the search result.""" + try: + site = self.gazelle_api.site + + # Insert the mapping + self.local_database.insert_site_tag_mapping(rating_key, + torrent_group.id, + site, + torrent_group.tags) + + artists = torrent_group.artists + if isinstance(artists, list): + artists = ', '.join(artists) + group_name = torrent_group.album_name + echo_func(f" ✓ Mapped: {artists} - {group_name} " + f"(Group ID: {torrent_group.id}, " + f"Tags: {', '.join(torrent_group.tags)})") + + return True + + except Exception as e: + logger.exception("Error creating site tag mapping for rating_key %s: %s", rating_key, e) + echo_func(f" Error creating mapping: {e}") + return False + + def create_collection_from_tags(self, tags: List[str], collection_name: str, + echo_func: Callable[[str], None]) -> bool: + """Create a Plex collection from albums matching the specified tags.""" + echo_func(f"Creating collection '{collection_name}' with tags: {', '.join(tags)}") + + # Get rating keys that match all specified tags + matching_rating_keys = self.local_database.get_rating_keys_by_tags(tags) + + if not matching_rating_keys: + echo_func("No albums found matching the specified tags.") + return False + + echo_func(f"Found {len(matching_rating_keys)} matching albums.") + + try: + # Get or create the collection in Plex + collections = self.plex_manager.library_section.collections() + existing_collection = None + + for collection in collections: + if collection.title == collection_name: + existing_collection = collection + break + + if existing_collection: + echo_func(f"Collection {collection_name} already exists.") + return False + + # Fetch Plex albums + matching_albums = [Album(id=rating_key) for rating_key in matching_rating_keys] + + # Create the collection + self.plex_manager.create_collection(collection_name, matching_albums) + + echo_func(f"✓ Collection '{collection_name}' " + f"created/updated with {len(matching_rating_keys)} albums.") + return True + + except Exception as e: + logger.exception("Error creating collection: %s", e) + echo_func(f"Error creating collection: {e}") + return False + + @staticmethod + def _parse_plex_artists(artist_str: str) -> List[str]: + """ + Parses an artist string, returning a list of individual artists + and the original combined string if it contains separators. + """ + if not artist_str: + return [] + + # Store the original, clean string to add it later + original_artist = artist_str.strip() + + separators = r'\s*;\s*|\s+\bx\b\s+|\s*\+\s*|\s*&\s*|\s*,\s*|\s*/\s*|\s+\band\b\s+' + + # Get the split parts + split_artists = re.split(separators, original_artist, flags=re.IGNORECASE) + + # Use a set to easily combine and remove duplicates + final_artists = {art for art in split_artists if art} + + # If the name was split into more than one part, add the original as well + if len(final_artists) > 1: + final_artists.add(original_artist) + + return list(final_artists) diff --git a/red_plex/use_case/upstream_sync/response.py b/red_plex/use_case/upstream_sync/response.py new file mode 100644 index 0000000..5f798a8 --- /dev/null +++ b/red_plex/use_case/upstream_sync/response.py @@ -0,0 +1,40 @@ +"""Response models for upstream sync operations.""" +from dataclasses import dataclass +from typing import List, Dict, Any, Optional + + +@dataclass +class AlbumSyncInfo: + """Information about an album to be synced.""" + group_id: str + display_name: str + artists: Optional[List[str]] = None + album_name: Optional[str] = None + + +@dataclass +class CollagePreviewData: + """Preview data for a collage to be synced.""" + collage_id: str + collage_name: str + external_id: str + site: str + albums_to_add: List[AlbumSyncInfo] + + +@dataclass +class UpstreamSyncPreviewResponse: + """Response containing preview data for upstream sync.""" + preview_data: List[CollagePreviewData] + success: bool + error_message: Optional[str] = None + + +@dataclass +class UpstreamSyncResponse: + """Response from upstream sync operation.""" + success: bool + synced_collages: int + total_collages: int + errors: List[str] + sync_results: Dict[str, Dict[str, Any]] # collage_id -> sync results diff --git a/red_plex/use_case/upstream_sync/upstream_sync_use_case.py b/red_plex/use_case/upstream_sync/upstream_sync_use_case.py new file mode 100644 index 0000000..2de16c8 --- /dev/null +++ b/red_plex/use_case/upstream_sync/upstream_sync_use_case.py @@ -0,0 +1,272 @@ +"""Upstream sync use case for syncing collections to site collages.""" +import logging +from typing import List, Dict, Any, Optional + +from red_plex.domain.models import Collection +from red_plex.infrastructure.db.local_database import LocalDatabase +from red_plex.infrastructure.plex.plex_manager import PlexManager +from red_plex.infrastructure.rest.gazelle.gazelle_api import GazelleAPI +from red_plex.use_case.upstream_sync.response import ( + UpstreamSyncPreviewResponse, UpstreamSyncResponse, + CollagePreviewData, AlbumSyncInfo +) + +# pylint: disable=W0718,R1705 +class UpstreamSyncUseCase: + """Use case for syncing local Plex collections to upstream site collages.""" + + def __init__(self, local_database: LocalDatabase, plex_manager: PlexManager): + """Initialize the upstream sync use case. + + Args: + local_database: Database instance for local data + plex_manager: Plex manager for collection access + """ + self.db = local_database + self.plex_manager = plex_manager + self.logger = logging.getLogger('red_plex') + + def get_sync_preview(self, collages: List[Collection]) -> UpstreamSyncPreviewResponse: + """Get preview of what would be synced for the given collages. + + Args: + collages: List of collages to preview sync for + + Returns: + Preview response with data about what would be synced + """ + preview_data = [] + errors = [] + + for collage in collages: + try: + collage_preview = self._get_collage_preview(collage) + if collage_preview and collage_preview.albums_to_add: + preview_data.append(collage_preview) + except Exception as e: + error_msg = f'Error getting preview for collage "{collage.name}": {str(e)}' + self.logger.error(error_msg) + errors.append(error_msg) + + return UpstreamSyncPreviewResponse( + preview_data=preview_data, + success=len(errors) == 0, + error_message='; '.join(errors) if errors else None + ) + + def sync_collections_upstream(self, + collages: List[Collection], + selected_albums: + Optional[Dict[str, List[str]]] = None) -> ( + UpstreamSyncResponse): + """Sync the given collections to their upstream collages. + + Args: + collages: List of collages to sync + selected_albums: Optional dict mapping collage_id to list of selected group_ids + If None, all missing albums will be synced + + Returns: + Response with sync results + """ + sync_results = {} + errors = [] + success_count = 0 + + for collage in collages: + try: + result = self._sync_single_collage(collage, selected_albums) + sync_results[collage.id] = result + + if result.get('success', False): + success_count += 1 + else: + errors.append(f'Failed to sync collage "{collage.name}": ' + f'{result.get("error", "Unknown error")}') + + except Exception as e: + error_msg = f'Error syncing collage "{collage.name}": {str(e)}' + self.logger.error(error_msg) + errors.append(error_msg) + sync_results[collage.id] = {'success': False, 'error': str(e)} + + return UpstreamSyncResponse( + success=len(errors) == 0, + synced_collages=success_count, + total_collages=len(collages), + errors=errors, + sync_results=sync_results + ) + + def _get_collage_preview(self, collage: Collection) -> Optional[CollagePreviewData]: + """Get preview data for a single collage.""" + try: + gazelle_api = GazelleAPI(collage.site) + + # Verify user ownership + if not self._verify_collage_ownership(gazelle_api, collage): + return None + + # Get missing group IDs + missing_group_ids = self._get_missing_group_ids(gazelle_api, collage) + if not missing_group_ids: + return None + + # Get album details + albums_to_add = [] + for group_id in missing_group_ids: + album_info = self._get_album_info(gazelle_api, group_id) + albums_to_add.append(album_info) + + return CollagePreviewData( + collage_id=collage.id, + collage_name=collage.name, + external_id=collage.external_id, + site=collage.site, + albums_to_add=albums_to_add + ) + + except Exception as e: + self.logger.error('Error getting preview for collage "%s": %s', collage.name, e) + return None + + def _sync_single_collage(self, collage: Collection, + selected_albums: Optional[Dict[str, List[str]]]) -> Dict[str, Any]: + """Sync a single collage to upstream.""" + try: + gazelle_api = GazelleAPI(collage.site) + + # Verify ownership + if not self._verify_collage_ownership(gazelle_api, collage): + return {'success': False, 'error': 'User does not own this collage'} + + # Get group IDs to sync + if selected_albums and collage.id in selected_albums: + group_ids_to_sync = selected_albums[collage.id] + else: + # Sync all missing group IDs + group_ids_to_sync = self._get_missing_group_ids(gazelle_api, collage) + + if not group_ids_to_sync: + self.logger.info('Collage "%s" is already up to date', collage.name) + return {'success': True, 'message': 'Already up to date', + 'added': 0, 'rejected': 0, 'duplicated': 0} + + # Add groups to collage + self.logger.info('Adding %d items to collage "%s"', + len(group_ids_to_sync), collage.name) + add_result = gazelle_api.add_to_collage(collage.external_id, group_ids_to_sync) + + if add_result and add_result.get('status') == 'success': + # Update the local collection database table with new group IDs + self.db.merge_torrent_groups_for_collage_collection( + collage.id, + set(int(x) for x in group_ids_to_sync)) + + response_data = add_result.get('response', {}) + added_count = len(response_data.get('groupsadded', [])) + rejected_count = len(response_data.get('groupsrejected', [])) + duplicated_count = len(response_data.get('groupsduplicated', [])) + + self.logger.info('Successfully synced collage "%s": ' + '%d added, %d rejected, %d duplicated', + collage.name, added_count, + rejected_count, duplicated_count) + + return { + 'success': True, + 'added': added_count, + 'rejected': rejected_count, + 'duplicated': duplicated_count + } + else: + error_msg = f'API call failed: {add_result}' + return {'success': False, 'error': error_msg} + + except Exception as e: + return {'success': False, 'error': str(e)} + + def _verify_collage_ownership(self, gazelle_api: GazelleAPI, collage: Collection) -> bool: + """Verify that the user owns the given collage.""" + try: + # Get user info + user_info = gazelle_api.get_user_info() + if not user_info or not user_info.get('id'): + return False + + # Get user's collages + user_collages = gazelle_api.get_user_collages(str(user_info['id'])) + if not user_collages: + return False + + # Check ownership + return any(uc.external_id == collage.external_id for uc in user_collages) + + except Exception as e: + self.logger.error('Error verifying ownership for collage "%s": %s', collage.name, e) + return False + + def _get_missing_group_ids(self, gazelle_api: GazelleAPI, collage: Collection) -> List[str]: + """Get list of group IDs that are missing from the upstream collage.""" + try: + # Get Plex collection items + plex_collection = self.plex_manager.get_collection_by_rating_key(collage.id) + if not plex_collection: + self.logger.warning('Plex collection with rating key "%s" not found', + collage.id) + return [] + + # Get rating keys from collection + collection_items = plex_collection.items() + current_rating_keys = [item.ratingKey for item in collection_items] + + if not current_rating_keys: + return [] + + # Get group IDs for rating keys + group_ids = self.db.get_group_ids_by_rating_keys(current_rating_keys, + collage.site.upper()) + if not group_ids: + self.logger.warning('No group ID mappings found for collection' + ' "%s" on site %s', + collage.name, collage.site) + return [] + + # Get current collage content + current_collage_data = gazelle_api.get_collage(collage.external_id) + if not current_collage_data: + return [] + + # Find missing group IDs + current_group_ids = {str(tg.id) for tg in current_collage_data.torrent_groups} + missing_group_ids = [gid for gid in group_ids if gid not in current_group_ids] + + return missing_group_ids + + except Exception as e: + self.logger.error('Error getting missing group IDs for collage "%s": %s', + collage.name, e) + return [] + + def _get_album_info(self, gazelle_api: GazelleAPI, group_id: str) -> AlbumSyncInfo: + """Get album information for the given group ID.""" + try: + torrent_group = gazelle_api.get_torrent_group(group_id) + if torrent_group: + artists_str = ( + ', '.join(torrent_group.artists)) if torrent_group.artists else 'Unknown Artist' + display_name = f'{artists_str} - {torrent_group.album_name}' + return AlbumSyncInfo( + group_id=group_id, + display_name=display_name, + artists=torrent_group.artists, + album_name=torrent_group.album_name + ) + else: + display_name = f'Group ID {group_id} (unable to get details)' + return AlbumSyncInfo(group_id=group_id, display_name=display_name) + + except Exception as e: + self.logger.debug('Error getting details for group %s: %s', group_id, e) + display_name = f'Group ID {group_id} (unable to get details)' + return AlbumSyncInfo(group_id=group_id, display_name=display_name) diff --git a/requirements.txt b/requirements.txt index 33390b3..a543cd0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -7,4 +7,5 @@ pyyaml flask flask-socketio gunicorn -eventlet \ No newline at end of file +eventlet +thefuzz[speedup] diff --git a/setup.py b/setup.py index c2e62b1..cc1834d 100644 --- a/setup.py +++ b/setup.py @@ -26,7 +26,8 @@ setup( name='red_plex', version=version, - description='A CLI and web GUI tool for creating Plex collections from RED and OPS collages and bookmarks', + description='A CLI and web GUI tool for creating ' + 'Plex collections from RED and OPS collages and bookmarks', long_description=long_description, long_description_content_type="text/markdown", author='marceljungle', @@ -52,7 +53,8 @@ 'flask', 'flask-socketio', 'gunicorn', - 'eventlet' + 'eventlet', + 'thefuzz[speedup]' ], entry_points=''' [console_scripts]