Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 44 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,10 @@ A Docker-based web application for managing your tabletop RPG PDF collection. Br
- **Global Audio Player** - A persistent pop-out player that keeps playing while you navigate. Build a local queue by playing a whole folder, queueing tracks one at a time ("Play Next"), having a GM play a campaign resource group, or playing all the audio embedded in a wiki note. Expand it to see and reorder upcoming tracks, with a repeat-current-track toggle
- **Bookmarks** - Per-user page and text-selection bookmarks with inline highlights
- **Favorites** - Save systems, books, maps, tokens, and audio for quick access
- **Shared Tags** - One tag catalog across systems, books, maps, tokens, and audio. Tags match on a lowercased internal key with an editable display name, so "Draw Steel" and "draw steel" are the same tag. A dedicated Tags page lists every tag with usage counts, lets you rename/merge/delete, and browses all items carrying a tag; clicking a tag anywhere jumps there. Filter dropdowns show only tags used on the current page, and campaign resources can be bulk-added by tag
- **View Modes** - Toggle the systems, books, maps, tokens, and audio grids between card, compact, and list layouts; each content type remembers its own default (configurable in Account Settings) while the in-page toggle is a per-tab override. Cards and list rows include quick download and favorite buttons.
- **Metadata Editor** - Add descriptions, tags, genre, publisher links, and character builder URLs
- **Metadata Editor** - Rich metadata for systems (multiple genres, dice/materials, system family, parent system + edition, license, year, and multiple generic + character-builder links) and books (authors, artists, genres, ISBN, version, language, a per-book license override, a variable-precision publication date, and multiple links). Genres, system families, parent systems, licenses, and dice/materials are drawn from curated lists you manage in **Settings → Metadata** (each section collapsible; defaults plus your own custom values). A *parent system* groups related systems (e.g. D&D 5e and AD&D under "Dungeons & Dragons"), and an *edition* string combines with it for display ("Cyberpunk" + "Red" → "Cyberpunk Red")
- **Sort & Filter** - Sort systems by name, book count, total page count, or year, and books by title, page count, or year. A shared filter modal covers genre, system family, parent system, edition, dice/materials, tags, favourites, and explicit content. Named filter presets are saved to your account (server-side, so they follow you across devices), and one preset per view can be set as the default you land on
- **Bulk Actions** - Multi-select books, maps, tokens, and audio (click, shift-click for a range, ⌘/Ctrl-click to toggle) then bulk tag, add to a campaign, or edit metadata via a carousel
- **Campaigns** - Track GM-run and personal campaigns; a markdown notes wiki with deep linking, Markdown/JSON/LegendKeeper import & export, character art and sheets, linked resources, and scheduling
- **OPDS Catalog** - Each user can generate a personal OPDS feed URL to connect e-reader apps directly to their library
Expand Down Expand Up @@ -235,19 +237,22 @@ Archive files placed anywhere under `books/` are shown alongside your books in t

Archives are treated as opaque downloads - Grimoire does not extract or read their contents, so clicking one downloads the file rather than opening the reader. They're also included when you download a whole system, category, or subfolder as an archive. Comic-book archives (`.cbz`, `.cbr`, `.cb7`, `.cbt`) additionally get a cover thumbnail generated from the first image inside them.

#### System-agnostic collections
#### Special collections (system-agnostic & one-page)

Some books don't belong to a single game system - reference material, zines, art books, or rulesets like Ironsworn or Mothership that span multiple systems. Create a folder whose name is one of the recognized system-agnostic names and Grimoire will display its contents in a separate **System-Agnostic** section on the library page, outside the normal game-system grid.
Some books don't belong to a single game system - reference material, zines, art books, or rulesets like Ironsworn or Mothership that span multiple systems. And some "systems" are really a bucket of many tiny games: one-page and small RPGs. Create a folder whose name is one of the recognized names below and Grimoire will display its contents in a separate **Special Collections** section on the library page, outside the normal game-system grid.

**Recognized folder names** (case-insensitive):

| Folder name | Example |
|---|---|
| `System Agnostic` | `books/System Agnostic/` |
| `Generic` | `books/Generic/` |
| `Any` | `books/Any/` |
| Folder name | Collection | Example |
|---|---|---|
| `System Agnostic` | System-agnostic | `books/System Agnostic/` |
| `Generic` | System-agnostic | `books/Generic/` |
| `Any` | System-agnostic | `books/Any/` |
| `One-Page RPGs` | One-page / small RPGs | `books/One-Page RPGs/` |
| `Single-Page RPGs` | One-page / small RPGs | `books/Single-Page RPGs/` |
| `One-Shot RPGs` | One-page / small RPGs | `books/One-Shot RPGs/` |

Subfolders directly under the agnostic root become **custom category headings** - whatever you name them is what appears in the UI. There is no keyword matching; the folder name is used as-is (slugified).
Subfolders directly under one of these roots become **custom category headings** - whatever you name them is what appears in the UI. There is no keyword matching; the folder name is used as-is (slugified).

```
books/
Expand Down Expand Up @@ -276,6 +281,23 @@ books/

Users with explicit content disabled will not see this system or its books.

#### Sort-order prefixes

To pull a system to the top of an alphabetically-sorted file browser, you can
prefix its folder name with `!`, `$`, or `%`. Grimoire strips a leading run of
those characters when deriving the system name (only the leading run — internal
occurrences are kept):

```
books/
├── !!Dungeons & Dragons/ → "Dungeons & Dragons"
├── !system-agnostic/ → still the System-Agnostic collection
└── $%Pathfinder 2e/ → "Pathfinder 2e"
```

The prefix stacks with `(nsfw)`, so `!!Forbidden Lore (NSFW)` becomes the
explicit system "Forbidden Lore".

### Book metadata from OPF files

Grimoire reads [OPF](https://idpf.org/epub/20/spec/OPF_2.0.1_draft.htm) sidecar files to populate book metadata automatically on first scan. OPF files are the format used by [Calibre](https://calibre-ebook.com/) and many other library managers.
Expand Down Expand Up @@ -367,18 +389,25 @@ Drop a `tags.json` file into any `maps/`, `tokens/`, or `audio/` folder (or subf
| `"subfolder"` | A subfolder |
| `"subfolder/file.png"` | A file inside a subfolder |

Values are arrays of tag strings.
Values are arrays of tag strings. The casing you write is used as the tag's
display name the first time it's seen.

```json
{
".": ["dungeon", "fantasy"],
"cave-entrance.png": ["cave", "outdoors"],
"boss-arena": ["combat", "finale"],
"boss-arena/throne-room.png": ["throne", "indoor"]
".": ["Dungeon", "Fantasy"],
"cave-entrance.png": ["Cave", "Outdoors"],
"boss-arena": ["Combat", "Finale"],
"boss-arena/throne-room.png": ["Throne", "Indoor"]
}
```

Tags are applied (or updated) every time the library is rescanned. Tags set via the web UI are replaced by the values in `tags.json` on the next scan.
`tags.json` is **additive and read-only**: on every rescan it only *adds* the
tags it lists — it never removes tags you set (or removed) in the web UI, and it
never overwrites a tag's display name once the tag exists. A new tag is created
using the casing in the file; renaming a tag later in the web UI sticks, because
the display name lives in the app's tag catalog rather than in `tags.json` (which
the app treats as read-only and never rewrites). Tags are matched
case-insensitively, so `"dungeon"` and `"Dungeon"` are the same tag.

---

Expand Down
9 changes: 9 additions & 0 deletions backend/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
THUMB_DIR = os.path.join(DATA_PATH, "thumbnails")
PAGE_CACHE_DIR = os.path.join(DATA_PATH, "page_cache")
CAMPAIGN_UPLOAD_DIR = os.path.join(DATA_PATH, "campaign_uploads")
SYSTEM_COVER_DIR = os.path.join(DATA_PATH, "system_covers")
VALKEY_URL = os.environ.get("VALKEY_URL", "")

# OCR: image-only PDFs (scanned pages with no embedded text layer) can be run
Expand All @@ -44,6 +45,13 @@
OCR_ENABLED = os.environ.get("OCR_ENABLED", "true").lower() == "true"
OCR_LANGUAGES = os.environ.get("OCR_LANGUAGES", "eng").strip() or "eng"

# Set true to disable the "update available" check that proxies GitHub's
# releases API. When disabled, /api/latest-release always returns null and no
# outbound request to GitHub is ever made.
DISABLE_VERSION_CHECKING = (
os.environ.get("DISABLE_VERSION_CHECKING", "false").lower() == "true"
)


def _read_ocr_concurrency() -> int:
"""Parallel-OCR worker count. 0 = OCR disabled; negatives clamp to 0; bad
Expand Down Expand Up @@ -307,6 +315,7 @@ def clear(self) -> None:
os.makedirs(os.path.join(CAMPAIGN_UPLOAD_DIR, "art"), exist_ok=True)
os.makedirs(os.path.join(CAMPAIGN_UPLOAD_DIR, "sheets"), exist_ok=True)
os.makedirs(os.path.join(CAMPAIGN_UPLOAD_DIR, "files"), exist_ok=True)
os.makedirs(SYSTEM_COVER_DIR, exist_ok=True)

engine, SessionLocal = init_db(DB_PATH)

Expand Down
3 changes: 3 additions & 0 deletions backend/indexer/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,11 @@
agnostic_category,
folder_category_inference_disabled,
guess_category,
is_one_page_folder,
is_special_collection_folder,
is_system_agnostic_folder,
slugify,
strip_sort_prefix,
)

# --- Archive + thumbnail helpers -----------------------------------------------
Expand Down
32 changes: 32 additions & 0 deletions backend/indexer/categories.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,30 @@
CATEGORY_MAP,
NO_AUTO_CATEGORY_MARKER, # noqa: F401 (re-exported for callers)
UNCATEGORIZED,
_ONE_PAGE_SLUGS,
_SYSTEM_AGNOSTIC_SLUGS,
)

logger = logging.getLogger("grimoire.indexer")


# Leading characters people prepend to system folders purely to steer the
# alphabetical sort order of their file browser (e.g. "!!Dungeons & Dragons").
# Only these three are recognized, and only as a contiguous leading run — once a
# non-special character is read, the rest is the real name.
_SORT_PREFIX_CHARS = "!$%"


def strip_sort_prefix(name: str) -> str:
"""Strip leading sort-order prefix characters (``!$%``) from a folder name.

Only the contiguous run of these characters at the very start is removed;
everything from the first non-prefix character onward is kept verbatim
(including internal ``!``/``$``/``%``). Surrounding whitespace is trimmed.
"""
return name.lstrip(_SORT_PREFIX_CHARS).strip()


def slugify(name: str) -> str:
"""Create a URL-safe slug from a name."""
slug = name.lower().strip()
Expand All @@ -30,6 +48,20 @@ def is_system_agnostic_folder(folder_name: str) -> bool:
return slugify(folder_name) in _SYSTEM_AGNOSTIC_SLUGS


def is_one_page_folder(folder_name: str) -> bool:
"""Return True if this top-level books folder is the one-page / small-RPG collection."""
return slugify(folder_name) in _ONE_PAGE_SLUGS


def is_special_collection_folder(folder_name: str) -> bool:
"""Return True for any special collection folder (agnostic or one-page).

Both use their immediate subfolder name as the category label rather than
the normal CATEGORY_MAP inference.
"""
return is_system_agnostic_folder(folder_name) or is_one_page_folder(folder_name)


def _normalize_folder(name: str) -> str:
"""Collapse hyphens, underscores, and whitespace to a single space for category matching."""
return re.sub(r"[-_\s]+", " ", name.lower()).strip()
Expand Down
15 changes: 14 additions & 1 deletion backend/indexer/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,17 @@
}
)

# Normalized folder names treated as the "one-page / small RPG" collection — a
# special sibling of the system-agnostic collection (issue #202). Books here use
# their immediate subfolder name as the category label, exactly like agnostic.
_ONE_PAGE_SLUGS = frozenset(
{
"one-page-rpgs",
"single-page-rpgs",
"one-shot-rpgs",
}
)

IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff", ".svg"}
PDF_EXTS = {".pdf"}
DOC_EXTS = {".pdf", ".epub", ".djvu"}
Expand Down Expand Up @@ -120,4 +131,6 @@
METADATA_MODES = ("new", "missing", "replace")

# Book fields that can be sourced from an OPF sidecar.
_OPF_BOOK_FIELDS = ("title", "authors", "description", "publisher", "year", "tags")
# Note: OPF ``tags`` are applied separately via the shared-tag service (issue
# #235); they are intentionally NOT in this setattr list (no column to set).
_OPF_BOOK_FIELDS = ("title", "authors", "description", "publisher", "year")
56 changes: 48 additions & 8 deletions backend/indexer/scan.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,17 @@
GenericMap,
Token,
)
from ..services import tag_service
from ._subprocess import _run_with_timeout
from .categories import (
agnostic_category,
folder_category_inference_disabled,
guess_category,
is_one_page_folder,
is_special_collection_folder,
is_system_agnostic_folder,
slugify,
strip_sort_prefix,
)
from .constants import (
ARCHIVE_EXTS,
Expand Down Expand Up @@ -177,6 +181,10 @@ def _scan_books(ctx: _ScanContext, books_dir: Path) -> None:
raw_name = system_dir.name
is_nsfw = bool(re.search(r"\(nsfw\)", raw_name, re.IGNORECASE))
system_name = re.sub(r"\s*\(nsfw\)\s*", "", raw_name, flags=re.IGNORECASE).strip()
# Strip any leading sort-order prefix (!$%) people use to steer their file
# browser's alphabetical ordering — "!!Dungeons & Dragons" → "Dungeons & Dragons".
# This must happen before slug/name/special-collection derivation.
system_name = strip_sort_prefix(system_name)
system_slug = slugify(system_name)

logger.debug(f"DB: querying system '{system_slug}'")
Expand All @@ -191,6 +199,10 @@ def _scan_books(ctx: _ScanContext, books_dir: Path) -> None:
stats["errors"] += 1
continue
is_agnostic = is_system_agnostic_folder(system_name)
is_one_page = is_one_page_folder(system_name)
# Both special collections (agnostic + one-page) use immediate-subfolder
# names as category labels rather than CATEGORY_MAP inference.
is_special = is_special_collection_folder(system_name)
# Per-system opt-out: a marker file at the system root disables
# folder-name category inference for just this system.
system_category_off = category_inference_off or (
Expand All @@ -202,6 +214,7 @@ def _scan_books(ctx: _ScanContext, books_dir: Path) -> None:
slug=system_slug,
is_explicit=is_nsfw,
is_system_agnostic=is_agnostic,
is_one_page=is_one_page,
)
session.add(system)
logger.debug(f"DB: flushing new system '{system_name}'")
Expand All @@ -220,6 +233,16 @@ def _scan_books(ctx: _ScanContext, books_dir: Path) -> None:
system.is_explicit = True
if is_agnostic and not system.is_system_agnostic:
system.is_system_agnostic = True
if is_one_page and not system.is_one_page:
system.is_one_page = True

# Folder cover convention: a cover.*/folder.* image at the system root
# becomes the system's cover (precedence: folder > uploaded > book cover).
# Stored library-relative so it survives moves of the whole library dir.
artwork = _find_folder_artwork(str(system_dir))
new_folder_cover = os.path.relpath(artwork, ctx.library_path) if artwork else ""
if (system.folder_cover_path or "") != new_folder_cover:
system.folder_cover_path = new_folder_cover

# When scoped to a path deeper than the system dir, walk only that
# subtree; otherwise walk the whole system.
Expand All @@ -229,7 +252,7 @@ def _scan_books(ctx: _ScanContext, books_dir: Path) -> None:
else system_dir
)
stop = _scan_books_in_system(
ctx, system, system_name, system_category_off, is_agnostic, walk_root
ctx, system, system_name, system_category_off, is_special, walk_root
)
if stop:
return
Expand All @@ -240,10 +263,14 @@ def _scan_books_in_system(
system: GameSystem,
system_name: str,
system_category_off: bool,
is_agnostic: bool,
is_special_collection: bool,
walk_root: Path,
) -> bool:
"""Walk one system's tree and register its books. Returns True if stop requested."""
"""Walk one system's tree and register its books. Returns True if stop requested.

``is_special_collection`` is True for the system-agnostic and one-page
collections, which label categories by immediate subfolder name.
"""
session = ctx.session
ignore = ctx.ignore
stats = ctx.stats
Expand Down Expand Up @@ -309,7 +336,7 @@ def _scan_books_in_system(
system,
system_name,
system_category_off,
is_agnostic,
is_special_collection,
root,
filename,
filepath,
Expand All @@ -334,7 +361,7 @@ def _register_book(
system: GameSystem,
system_name: str,
system_category_off: bool,
is_agnostic: bool,
is_special_collection: bool,
root: str,
filename: str,
filepath: str,
Expand All @@ -354,7 +381,16 @@ def _register_book(
# requested (modes "missing"/"replace") — see _apply_opf_to_book.
if ctx.metadata_mode in ("missing", "replace"):
opf_meta = _find_opf_meta(root, filename)
if _apply_opf_to_book(existing, opf_meta, ctx.metadata_mode):
changed = _apply_opf_to_book(existing, opf_meta, ctx.metadata_mode)
# OPF ``tags`` are shared tags (issue #235), applied via the service.
# In "missing" mode only fill when the book has no tags yet.
opf_tags = opf_meta.get("tags")
if opf_tags:
current = tag_service.display_tags_for_resource(session, "book", existing.id)
if ctx.metadata_mode == "replace" or not current:
tag_service.set_resource_tags(session, "book", existing.id, opf_tags)
changed = True
if changed:
logger.debug(f"Refreshing metadata for '{filename}' (mode={ctx.metadata_mode})")
try:
_run_with_timeout(
Expand Down Expand Up @@ -384,7 +420,7 @@ def _register_book(

if system_category_off:
category = UNCATEGORIZED
elif is_agnostic:
elif is_special_collection:
category = agnostic_category(relative_path)
else:
category = guess_category(relative_path)
Expand Down Expand Up @@ -420,7 +456,6 @@ def _register_book(
description=opf_meta.get("description"),
publisher=opf_meta.get("publisher"),
year=opf_meta.get("year"),
tags=opf_meta.get("tags"),
)

# Commit the book record first so that if a subsequent
Expand All @@ -430,6 +465,11 @@ def _register_book(
logger.debug(f"DB: committing new book '{filename}'")
try:
_run_with_timeout(session.commit, _DB_TIMEOUT, f"commit book '{filepath}'")
# OPF ``subjects`` become shared tags on the book (issue #235).
opf_tags = opf_meta.get("tags")
if opf_tags:
tag_service.set_resource_tags(session, "book", book.id, opf_tags)
session.commit()
stats["new_books"] += 1
logger.info(f"Added book: {title} ({category}) in {system_name}")
except TimeoutError as e:
Expand Down
Loading
Loading