Mismatched Library Content

[Chireven]

Good Morning!

I'm doing a hardware migration and moving my home services to proxmox from a few different platforms. In doing so, I found a need to do a staged migration of my media libraries. This is causing me to have multiple servers with different content. In order to preserve my watch history, I started looking around and found WatchState.

After getting it up and running in an lxc container, everything looked like it was working great -- but I quickly found that the way I wanted things to work and reality didn't quite line up! I ran into an issue where moving media from one server to another would cause the media to become marked unwatched. After digging in, I identified the code was doing this intentionally. After some more exploration, I believe it's done to support the ability to mark something unwatched.

You've shared procedures to overcome this, but I was bored this weekend and my project is going to span a couple of months. I didn't really want to manage every piece of media I migrated to the new media server. I dug in and thought I'd discuss a potential change to see if you were open to it.

Right now, the latest date wins. My proposal is an exception that fires when new media is found. A check would be made to see if this media is already in the database. If it is, we use the watch state from the database. If not, we proceed as normal. In order to achieve this, a change would need to be made to the normal import process as well as incoming webhooks (this one tripped me up for a while as I forgot I configured them!)

I've found some references to other people having this same problem, so I know you've put some thought into it. I wanted to run this by you and see if you'd be open to this sort of change, or if there is a scenario that I'm not considering that this might be breaking.

I have a current copy of the code that I've modified and it's a small patch. Most of the changes are actually logging to make sure things worked as expected. I implemented it by adding an 'advanced option' to the backend server configuration that defaults to false. When I turn it on and move media, the known watch state from the database is used. When I turn it off and move media, the unwatched value is synchronized. It's actually working really well for me, but I've only done very limited testing.

If you're interested, I'm happy to work on formalizing this and sharing it.

WatchState is an very powerful tool, and I think this ability (even if it were off by default) would be a great addition. It might benefit a smaller subset of your users (those doing swing migrations with their media libraries), but I'm willing to share the work I've done and, and do more to make it align with your vision, if you're interested.

I really appreciate the work you've done -- it's a very solid project, and I love the console. It's going to finally give me the opportunity to test some of the other media servers out there, since watch history has been one of the reasons I haven't moved away from Plex.

Thanks again!
-joe

[arabcoders]

Thanks for the feature request,

I’ll consider it. The use case makes sense, especially for migrations "thanks plex" where libraries temporarily exist in different places and newly added copies should not necessarily override known watched history.

That said, I need to think through it carefully. WatchState already has quite a few conditional paths around state conflict resolution, mark unplayed behavior, imports, exports, webhooks, tainted events, timestamps, and backend metadata. Adding another rule is not just toggle, it touches a lot of the logic brain and can create surprising edge cases if it is not scoped very precisely.

The main concern is making sure this would not break legitimate mark unwatched workflows or create situations where WatchState refuses to accept a backend real current state. If this moves forward, it would almost certainly need to be opt-in, per-backend, clearly logged, and covered by regression tests for both normal imports and webhooks.

I appreciate you offering to share the patch. I’m interested in looking at the approach, but I cannot promise acceptance until I’ve reviewed the behavior from the different sync/conflict angles.

[arabcoders]

Hi Joe,

Thanks again for taking the time to write this up and dig into the behavior.

After looking through the mapper logic more carefully, I don’t think I want to add this as a separate feature right now. The main reason is that WatchState already has several layers of conditional state handling: stale-date checks, tainted events, metadata-only imports, mark-unplayed protection, force-full imports, webhook handling, and per-backend options. Adding another rule here would complicate an already delicate part of the state engine for what now looks like limited gain.

The behavior you’re describing is mostly already covered during normal incremental syncs. When WatchState has a valid import.lastSync, an incoming unwatched item that conflicts with an existing watched item is handled as a conflict/metadata update rather than blindly clearing the watched state. The webhook bug you found meant that protection was not being applied correctly for webhook events, but that has now been fixed.

For the remaining cases where there is no lastSync, such as first imports or --force-full, I’m hesitant to change the behavior. In those modes WatchState intentionally treats the backend as a fuller source of truth. If we add a rule that preserves local state even there, it weakens the meaning of force-full and makes the sync model harder to reason about.

For users who want stronger protection during migrations, the existing DISABLE_MARK_UNPLAYED option should be sufficient, especially with the webhook lastSync fix in place. It is broader than your proposed behavior, but it is explicit, already supported, and avoids adding another branch to the core state-resolution logic.

So for now I’m going to pass on implementing this as a new option. I do appreciate the investigation and the offer to share the patch. If we still see watched history being erased incorrectly after the webhook fix and with DISABLE_MARK_UNPLAYED available, I’d rather revisit it from a concrete failing case than add another state-management mode preemptively.