Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/client-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ Abandoned uploads that a client cannot clean remain eligible for `media prune`.
CLI: `moment media delete <id> --dry-run --json`, then `--yes` to delete.
Both direct and API modes return the same result and retain confirmation rules.

`PUT /links/drafts/:key/media` protects the captured images of one browser-local draft. It requires an active account, CSRF for cookie authentication, a 64-character hexadecimal key, and `{"media_ids":[...]}` containing at most two media IDs granted to that account. An empty array releases the draft's references. The response is `{"saved":true}` in the standard envelope. References expire after 30 days, matching browser draft retention; publication establishes content references before releasing draft references. This endpoint manages browser-local draft resources only and intentionally has no standalone CLI draft command. Media maintenance remains available through the CLI.

The server automatically checks captured link resources at startup and hourly. Only captured resources continuously unreferenced for at least 24 hours are eligible; saved draft references and content references prevent deletion. Ordinary uploads are excluded from automatic link cleanup. `moment media prune --link-previews-only --dry-run` inspects this subset offline; actual deletion requires `--yes` or confirmation.

## Request source

Send `X-Moment-Surface: ios` on native requests, including login and public reads.
Expand All @@ -148,6 +152,8 @@ inside a sentence intentionally remains visible.

The web editor restores a post's saved link card even when the body has no URL. Caption-only edits preserve the stored snapshot and its managed cover without resolving the external URL again. Removing the card is an explicit edit that can be undone or saved independently of the caption.

Link cards accept optional `display_mode: "cover" | "compact"`, defaulting to `cover` for older snapshots and clients. Public cards include the selected mode. `PATCH /posts/:id` also accepts `link_card_display_mode` to change presentation without resubmitting captured metadata. It requires an existing card and retains its title, description, original URL, capture time, and managed images. CLI: `moment post update <id> --link-display compact --json`; new snapshots may set `display_mode` in `--link-json` input. Browser drafts preserve the selection, while their editing card stays compact.

Post and search DTOs add nullable `edited_at` (RFC 3339 UTC strings with millisecond
precision). Actual changes
to Markdown, ordered media/alt text, location, map card, or link cards update this field.
Expand Down
2 changes: 2 additions & 0 deletions docs/export-format-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ Media members follow in SHA order. The checksum file lists every other ZIP membe

`data/entries.jsonl` preserves IDs, `author_user_id`, publication and edit timestamps, visibility, publication state, original/repost state, locations, link snapshots, media order and descriptions, and quoted entry IDs. Private and unpublished content remains associated with its real author.

Link snapshots preserve `display_mode` as `cover` or `compact`. Older records that omit it use `cover`. Import, personal restore, and merge retain the presentation choice together with the captured metadata and managed image references.

An article entry has `kind: "article"` and a null Markdown body. Its structured document and cover are stored in `data/entry_articles.jsonl`. Ordinary posts carry Markdown instead. Derived HTML and search indexes are rebuilt with the current implementation.

`data/article_drafts.jsonl` preserves the real author, title, structured body, cover, publication relationship and revision timestamps. Published working copies, withdrawn documents and unpublished drafts remain distinct. The full graph is validated, including inline media and references to published entries.
Expand Down
8 changes: 8 additions & 0 deletions docs/media-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,14 @@ avoids trusting a cache hit that a concurrent prune could remove. A failed
processor or commit never publishes an authoritative media row or original;
an orphaned derived cache entry is harmless and can be rebuilt or pruned.

## Link preview retention

Link thumbnails and site icons are captured into managed media while a preview is resolved. Newly captured media carry a link-resource marker; existing ordinary uploads keep their original lifecycle when deduplicated. Closing a preview does not delete shared bytes immediately. Startup and hourly maintenance remove marked resources only after at least 24 continuous hours without a content or saved browser-draft reference. The central reference resolver protects saved cards from explicit media deletion and manual pruning as well as automatic cleanup.

Browser drafts retain their card snapshot locally and register account-scoped media references before confirming a save. Each browser uses a separate resource key namespace. Removing or publishing a loaded draft releases its temporary references; successful publication owns independent content references. Draft references expire after the browser's existing 30-day retention period, then start a fresh 24-hour cleanup grace period. A failed release can therefore delay cleanup until expiry. Unmarked historical media are not classified for automatic deletion by filename.

The schema migration adds resource markers and draft references without rewriting content or originals. Older executables reject the newer schema. Rollback requires a compatible build that keeps schema support, or a pre-upgrade database backup when discarding subsequent writes is acceptable.

## Video quality strategy

Moment optimizes for a useful display version, not for making every result
Expand Down
27 changes: 22 additions & 5 deletions internal/app/posts.go
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,8 @@ type UpdatePostInput struct {
// follows the media-usage list convention rather than the single-value
// double-pointer convention Location uses.
LinkCards *[]domain.LinkCard
// LinkCardDisplayMode changes presentation while preserving captured metadata.
LinkCardDisplayMode *domain.LinkCardDisplayMode
// Visibility nil preserves the current audience. Changing it alone is a
// complete update: nothing else about the post has to move.
Visibility *domain.Visibility
Expand Down Expand Up @@ -1013,7 +1015,7 @@ func (service *PostService) Update(ctx context.Context, input UpdatePostInput) (
if err := validatePostID(input.ID); err != nil {
return Post{}, err
}
if input.PreserveContent && input.Media == nil && input.Location == nil && input.LinkCards == nil && input.MapCard == nil && input.Visibility == nil {
if input.PreserveContent && input.Media == nil && input.Location == nil && input.LinkCards == nil && input.LinkCardDisplayMode == nil && input.MapCard == nil && input.Visibility == nil {
return Post{}, domain.NewError(domain.CodeInvalidArgument, "post update must change content_md, media, map_card, location, link cards, or visibility", map[string]any{"field": "body"})
}
if input.MapCard != nil && *input.MapCard != nil {
Expand All @@ -1024,6 +1026,9 @@ func (service *PostService) Update(ctx context.Context, input UpdatePostInput) (
if input.Visibility != nil && !input.Visibility.Valid() {
return Post{}, domain.NewError(domain.CodeInvalidArgument, "visibility must be everyone or only_me", map[string]any{"field": "visibility"})
}
if input.LinkCardDisplayMode != nil && !input.LinkCardDisplayMode.Valid() {
return Post{}, domain.NewError(domain.CodeInvalidArgument, "link card display mode must be cover or compact", map[string]any{"field": "link_card_display_mode"})
}
if input.Media != nil {
if err := domain.ValidateEntryMediaUsages(*input.Media); err != nil {
return Post{}, err
Expand Down Expand Up @@ -1102,13 +1107,22 @@ func (service *PostService) Update(ctx context.Context, input UpdatePostInput) (
// The replacement set is authoritative for validation and for the
// derived search text. Projections are reloaded after the write so the
// response carries each thumbnail's stored geometry.
nextLinkCards := LinkCardsOf(currentLinkCards)
nextLinkCards := domain.CloneLinkCards(LinkCardsOf(currentLinkCards))
if input.LinkCards != nil {
if mediaErr := domain.ValidateLinkCards(*input.LinkCards); mediaErr != nil {
return mediaErr
}
nextLinkCards = *input.LinkCards
}
if input.LinkCardDisplayMode != nil {
if len(nextLinkCards) == 0 {
return domain.NewError(domain.CodeInvalidArgument, "a link card is required to change its display mode", nil)
}
nextLinkCards = domain.CloneLinkCards(nextLinkCards)
for index := range nextLinkCards {
nextLinkCards[index].DisplayMode = *input.LinkCardDisplayMode
}
}
nextMap := entry.MapCard
if input.MapCard != nil {
nextMap = domain.CloneMapCard(*input.MapCard)
Expand Down Expand Up @@ -1167,13 +1181,13 @@ func (service *PostService) Update(ctx context.Context, input UpdatePostInput) (
return updateErr
}
}
if input.LinkCards != nil {
if updateErr := store.ReplaceEntryLinkCards(ctx, input.ID, *input.LinkCards); updateErr != nil {
if input.LinkCards != nil || input.LinkCardDisplayMode != nil {
if updateErr := store.ReplaceEntryLinkCards(ctx, input.ID, nextLinkCards); updateErr != nil {
return updateErr
}
// Old and new thumbnails are reconciled together in this same
// transaction so a replaced card never leaves a stale clock.
affectedCards := append(linkCardMediaIDs(LinkCardsOf(currentLinkCards)), linkCardMediaIDs(*input.LinkCards)...)
affectedCards := append(linkCardMediaIDs(LinkCardsOf(currentLinkCards)), linkCardMediaIDs(nextLinkCards)...)
if updateErr := store.ReconcileMediaReferences(ctx, affectedCards, now); updateErr != nil {
return updateErr
}
Expand Down Expand Up @@ -1349,6 +1363,9 @@ func hashCreatePost(content string, media []domain.EntryMediaUsage, location *do
_, _ = digest.Write([]byte("\x00link-card-v1\x00"))
writeHashLength(digest, len(cards))
for _, card := range cards {
if card.EffectiveDisplayMode() != domain.LinkDisplayCover {
_, _ = digest.Write([]byte("\x00link-display:" + string(card.EffectiveDisplayMode()) + "\x00"))
}
writeHashLength(digest, card.Position)
for _, value := range []string{
card.SourceURL, card.CanonicalURL, string(card.Provider), string(card.Kind),
Expand Down
8 changes: 6 additions & 2 deletions internal/archive/export.go
Original file line number Diff line number Diff line change
Expand Up @@ -716,6 +716,7 @@ type articleDraftRecord struct {
// content hash rather than by media ID because media identifiers are reassigned
// on import while the hash is the stable anchor across instances.
type linkCardRecord struct {
DisplayMode string `json:"display_mode,omitempty"`
Position int `json:"position"`
SourceURL string `json:"source_url"`
CanonicalURL string `json:"canonical_url"`
Expand Down Expand Up @@ -1383,7 +1384,8 @@ func portableLinkCards(values []sqlitestore.ArchiveLinkCard) []linkCardRecord {
records := make([]linkCardRecord, 0, len(values))
for _, value := range values {
record := linkCardRecord{
Position: value.Card.Position, SourceURL: value.Card.SourceURL,
DisplayMode: string(value.Card.EffectiveDisplayMode()),
Position: value.Card.Position, SourceURL: value.Card.SourceURL,
CanonicalURL: value.Card.CanonicalURL, Provider: string(value.Card.Provider),
CardKind: string(value.Card.Kind), Title: value.Card.Title,
Description: value.Card.Description, SiteName: value.Card.SiteName,
Expand Down Expand Up @@ -1422,11 +1424,13 @@ func domainLinkCards(values []linkCardRecord) ([]sqlitestore.ArchiveLinkCard, er
}
capturedAt := instant.UnixMilli()
card := domain.LinkCard{
Position: index, SourceURL: value.SourceURL, CanonicalURL: value.CanonicalURL,
DisplayMode: domain.LinkCardDisplayMode(value.DisplayMode),
Position: index, SourceURL: value.SourceURL, CanonicalURL: value.CanonicalURL,
Provider: domain.LinkCardProvider(value.Provider), Kind: domain.LinkCardKind(value.CardKind),
Title: value.Title, Description: value.Description, SiteName: value.SiteName,
AuthorName: value.AuthorName, EmbedURL: value.EmbedURL, CapturedAt: capturedAt,
}
card.DisplayMode = card.EffectiveDisplayMode()
if value.DurationSeconds != nil {
seconds := *value.DurationSeconds
card.DurationSeconds = &seconds
Expand Down
7 changes: 7 additions & 0 deletions internal/archive/import_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1296,6 +1296,10 @@ INSERT INTO entry_link_card (
'A captured title', 'A captured description', 'YouTube', 'A channel',
212, ?, 'https://www.youtube-nocookie.com/embed/dQw4w9WgXcQ', 1700000000000)`,
cardEntryID, mediaOneID)
if err != nil {
return err
}
_, err = queries.ExecContext(context.Background(), `UPDATE entry_link_card SET display_mode='compact' WHERE entry_id=?`, cardEntryID)
return err
}); err != nil {
t.Fatal(err)
Expand Down Expand Up @@ -1340,6 +1344,9 @@ INSERT INTO entry_link_card (
t.Fatalf("restored entry = %#v", restored)
}
card := restored.LinkCards[0]
if card.Card.DisplayMode != domain.LinkDisplayCompact {
t.Fatalf("display mode was not restored: %q", card.Card.DisplayMode)
}
if card.Card.Title != "A captured title" || card.Card.Description != "A captured description" {
t.Fatalf("card text did not survive: %#v", card.Card)
}
Expand Down
4 changes: 3 additions & 1 deletion internal/archive/merge_review.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ type MergeReviewLocation struct {
// difference between two versions of a post, so a review that omitted it would
// show two identical-looking sides beside the words "link_cards changed".
type MergeReviewLinkCard struct {
DisplayMode string `json:"display_mode"`
Position int `json:"position"`
SourceURL string `json:"source_url"`
CanonicalURL string `json:"canonical_url"`
Expand Down Expand Up @@ -407,7 +408,8 @@ func reviewLinkCards(values []sqlitestore.ArchiveLinkCard) []MergeReviewLinkCard
result := make([]MergeReviewLinkCard, 0, len(values))
for _, value := range values {
result = append(result, MergeReviewLinkCard{
Position: value.Card.Position, SourceURL: value.Card.SourceURL,
DisplayMode: string(value.Card.EffectiveDisplayMode()),
Position: value.Card.Position, SourceURL: value.Card.SourceURL,
CanonicalURL: value.Card.CanonicalURL, Provider: string(value.Card.Provider),
CardKind: string(value.Card.Kind), Title: value.Card.Title, SiteName: value.Card.SiteName,
})
Expand Down
26 changes: 26 additions & 0 deletions internal/domain/link_card.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,20 @@ const (
// LinkCardKind separates a plain link snapshot from an embeddable video.
type LinkCardKind string

// LinkCardDisplayMode controls presentation independently of the captured page.
type LinkCardDisplayMode string

// Supported link card display modes.
const (
LinkDisplayCover LinkCardDisplayMode = "cover"
LinkDisplayCompact LinkCardDisplayMode = "compact"
)

// Valid reports whether mode is an explicit supported presentation choice.
func (mode LinkCardDisplayMode) Valid() bool {
return mode == LinkDisplayCover || mode == LinkDisplayCompact
}

// Recognized link card kinds.
const (
// LinkKindLink is a static snapshot with no inline player.
Expand All @@ -55,6 +69,7 @@ const (

// LinkCard is an immutable URL preview snapshot attached to content.
type LinkCard struct {
DisplayMode LinkCardDisplayMode
Position int
SourceURL string
CanonicalURL string
Expand Down Expand Up @@ -82,8 +97,19 @@ type LinkCard struct {
LikeCount *int64
}

// EffectiveDisplayMode treats an omitted preference as the cover presentation.
func (card LinkCard) EffectiveDisplayMode() LinkCardDisplayMode {
if card.DisplayMode == "" {
return LinkDisplayCover
}
return card.DisplayMode
}

// Validate checks preview fields and managed-media references.
func (card LinkCard) Validate() error {
if !card.EffectiveDisplayMode().Valid() {
return NewError(CodeInvalidArgument, "link card display mode must be cover or compact", map[string]any{"field": "display_mode"})
}
if card.Position < 0 || card.Position >= MaxLinkCardsPerEntry {
return NewError(CodeInvalidArgument, "link card position is out of range", map[string]any{"field": "position", "limit": MaxLinkCardsPerEntry})
}
Expand Down
13 changes: 12 additions & 1 deletion internal/media/account_access.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,16 @@ func grantIngestedMedia(ctx context.Context, queries sqlite.Queries, id domain.I
}

func (store *Store) grantExistingMedia(ctx context.Context, id domain.ID) error {
return store.db.WriteTx(ctx, func(queries sqlite.Queries) error { return grantIngestedMedia(ctx, queries, id, store.options.Clock()) })
return store.db.WriteTx(ctx, func(queries sqlite.Queries) error {
if err := grantIngestedMedia(ctx, queries, id, store.options.Clock()); err != nil {
return err
}
if capture, _ := ctx.Value(linkPreviewCaptureKey{}).(bool); capture {
return nil
}
// An explicit upload can share bytes with a captured link image. It must
// then follow ordinary upload retention, regardless of which arrived first.
_, err := queries.ExecContext(ctx, `DELETE FROM link_preview_media WHERE media_id=?`, id.String())
return err
})
}
45 changes: 45 additions & 0 deletions internal/media/link_retention.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
package media

import (
"context"
"encoding/hex"
"time"

"github.com/moment/moment/internal/app"
"github.com/moment/moment/internal/domain"
"github.com/moment/moment/internal/runtimefs"
"github.com/moment/moment/internal/storage/sqlite"
)

// RetainLinkDraft replaces one browser draft's thumbnail and site-mark references.
// An empty list releases the draft; publication owns its own references first.
func (store *Store) RetainLinkDraft(ctx context.Context, key string, rawIDs []string) error {
user, ok := app.ContentActor(ctx)
if !ok {
return domain.NewError(domain.CodeUnauthorized, "an account is required", nil)
}
decoded, err := hex.DecodeString(key)
if err != nil || len(decoded) != 32 || len(rawIDs) > 2 {
return domain.NewError(domain.CodeInvalidArgument, "invalid draft resources", nil)
}
ids := make([]domain.ID, 0, len(rawIDs))
for _, raw := range rawIDs {
id, err := domain.ParseID(raw)
if err != nil {
return err
}
ids = append(ids, id)
}
lease, err := store.acquireMaintenanceAndMedia(ctx, runtimefs.Shared)
if err != nil {
return err
}
defer func() { _ = lease.Release() }()
return sqlite.ReplaceLinkDraftMedia(ctx, store.db, user.ID, key, ids, store.options.Clock())
}

// PruneLinkPreviews is the automatic subset of media maintenance: unrelated
// uploads are excluded even when they have never been attached to content.
func (store *Store) PruneLinkPreviews(ctx context.Context, now time.Time, dryRun bool) (PruneResult, error) {
return store.prune(ctx, now, dryRun, true)
}
Loading
Loading