28 Commits

Author SHA1 Message Date
Phlogi
8c6adf7157
feat(server): resolve duplicates (#25316)
* feat(web): Synchronize information from deduplicated images

* Added new settings menu to the the deduplication tab.
* The toggable options in the settings are synchronization of: albums, favorites, ratings, description, visibility and location.
* When synchronizing the albums, the resolved images will be added to all albums of the duplicates.
* When synchronizing the favorite status, the resolved images will be marked as favorite, if at least one selectable image is marked as favorite.
* When synchronizing the ratings, the highest rating from the selectable images will be applied to the resolved image.
* When synchronizing the description, all descriptions from the selectable images will be merged into one description for the resolved image.
* When synchronizing the visibility, the most restrictive visibility setting from the selectable images will be applied to the resolved image.
* When synchronizing the location, if exactly one unique location exists among the selectable images, this location will be applied to the resolved image.
* There is no additional UI for these settings to keep the visual clutter minimal. The settings are applied automatically based on the user's preferences.

* Replace addAssetToAlbums with copyAsset

* fix linter

* feat(web): add duplicate sync fields and fix typo

* feat(web): add tag sync and enhance duplicate resolution

This update introduces tag synchronization for duplicate resolution,
ensuring all unique tag IDs from duplicates are applied to kept assets.
The visibility sync logic is updated to use a simplified ordering, as the hidden status items will never show up in a duplicate set.
Album synchronization now merges albums directly via addAssetsToAlbums; as the approach with copyAsset API endpoint was ineffiecient.
Description, rating, and location sync logic is improved for correctness.
and deduplication. i18n strings were added / updated.

* feat(server): move duplicate resolution to backend with sync and stacking

Moves duplicate metadata synchronization from frontend to backend, enabling robust
batch operations and proper validation. This is an improved refactor of PR #13851.

New endpoints:
- POST /duplicates/resolve - batch resolve with configurable metadata sync
- POST /duplicates/stack - create stacks from duplicate groups
- GET /duplicates - now includes suggestedKeepAssetIds based on file size and EXIF

Key changes:
- Move sync logic (albums, tags, favorites, ratings, descriptions, location, visibility) to server
- Add server-side metadata merge policies with proper conflict resolution
- Replace client-side resolution logic with new backend endpoints
- Add comprehensive E2E tests (70+ test cases) and unit tests
- Update OpenAPI specs and TypeScript SDK

No breaking changes - only additions to existing API.

* feat(preferences): enable all duplicate sync settings by default

* chore: clean up

* chore: clean up

* refactor: rename & clean up

* fix: preference upgrade

* chore: linting

* refactor(e2e): use updateAssets API for setAssetDuplicateId

* fix: visibility sync logic in duplicate resolution

* fix(duplicate): write description to exifUpdate

Previously the duplicate resolution populated assetUpdate.description even
though description belongs to exif info.

* fix(duplicate): remove redundant updateLockedColumns wrapper

updateAllExif already computes lockedProperties via distinctLocked
using Object.keys(options). The wrapper added a lockedProperties key
to the options object, causing the spurious string 'lockedProperties'
to be stored in the lockedProperties array.

* fix(duplicate): write merged tags to asset_exif to survive metadata re-extraction

During duplicate resolution, replaceAssetTags correctly wrote merged tag
IDs to the tag_asset table, but never updated asset_exif.tags or locked
the tags property. The subsequent SidecarWrite → AssetExtractMetadata
chain calls applyTagList, which destructively replaces tag_asset rows
with whatever is in asset_exif.tags — still the original per-asset tags,
not the merged set.

Write merged tag values to asset_exif.tags via updateAllExif (which also
locks the property via distinctLocked), and queue SidecarWrite when tags
change so they persist to the sidecar file.

* docs(duplicates): clarify location and tag sync behavior

* refactor(duplicate): remove sync settings, always sync all metadata on resolve

Remove DuplicateSyncSettingsDto and the per-field sync toggles
(albums, favorites, rating, description, visibility, location, tags).
Duplicate resolution now unconditionally syncs all metadata from
trashed assets to kept assets.

- Remove DuplicateSyncSettingsDto and settings field from DuplicateResolveDto
- Update DuplicateService to always run all sync logic without conditionals
- Delete DuplicateSettingsModal.svelte and settings gear button from UI
- Remove DuplicateSettings type and duplicateSettings persisted store
- Update unit and e2e tests to remove settings from resolve requests

* docs: update duplicates utility to reflect automatic metadata sync

* docs(web): replace duplicates info modal with link to documentation

* chore: clean up

* fix: add missing type cast to jsonAgg in duplicate repository getAll

* fix: skip persisting rating=0 in duplicate merge to avoid unnecessary sidecar write

---------

Co-authored-by: Toni <51962051+EinToni@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jason@rasm.me>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2026-03-26 18:33:55 +00:00
Jason Rasmussen
bcb968e3d1
refactor: job names (#19949) 2025-07-15 18:39:00 -04:00
Jason Rasmussen
e73abe0762
refactor: enum casing (#19946) 2025-07-15 14:50:13 -04:00
Michael Anderson
de756d9497
fix: skip locked photos during duplicate scan. (#19105)
Skip locked photos during duplicate scan.
2025-06-13 10:50:56 -04:00
Jason Rasmussen
c9bcae813b
feat: duplicate delete groups api (#19142) 2025-06-12 17:48:43 -04:00
Jason Rasmussen
5cd186d3d4
refactor: duplicate queries (#19136) 2025-06-12 14:23:02 -04:00
Mert
a6a4dfcfd3
fix(server): queueing for duplicate detection (#18380)
* fix queueing

* update tests
2025-05-20 08:44:39 -05:00
Alex
d33ce13561
feat(server): visibility column (#17939)
* feat: private view

* pr feedback

* sql generation

* feat: visibility column

* fix: set visibility value as the same as the still part after unlinked live photos

* fix: test

* pr feedback
2025-05-06 12:12:48 -05:00
Jason Rasmussen
8c5116bc1d
refactor: stream search duplicates (#17991) 2025-04-30 10:40:32 -04:00
Daniel Dietzler
52ae06c119
refactor: remove album entity, update types (#17450) 2025-04-18 23:10:34 +02:00
Jason Rasmussen
17e720440d
refactor: new asset-job repository (#17622)
* refactor: new asset-job repository

* fix: broken medium tests on main
2025-04-15 10:24:51 -04:00
Jason Rasmussen
fdbe6d649f
refactor: remove smart search entity (#17447)
refactor: smart search entity
2025-04-08 09:56:45 -04:00
Mert
59e5c82569
fix(server): full-size images not migrated or deleted correctly (#17308)
* fix file path logic

* update tests

* add empty array fallback just in case for now
2025-04-01 18:11:46 -04:00
Mert
a708649504
fix(server): skip stacked assets in duplicate detection (#16380)
* skip stacked assets in duplicate detection

* update sql

* handle stacking after duplicate detection runs
2025-02-27 10:16:13 -06:00
Jason Rasmussen
fa5aeaf539
refactor: last repository (#16042) 2025-02-11 22:15:56 +00:00
Jason Rasmussen
9d85272c2b
refactor: repositories (#16036) 2025-02-11 14:08:13 -05:00
Mert
2e12c46980
refactor(server): use kysely (#12857) 2025-01-09 11:15:41 -05:00
Pranay Pandey
380fc06979
fix: remove duplicateIds on unique assets (#13752) 2024-11-04 10:03:03 -05:00
Jason Rasmussen
5b2cd704d3
refactor(server): job discovery (#13838)
refactor(server): job discorvery
2024-10-31 13:42:58 -04:00
Jason Rasmussen
4ea281f854
refactor: service dependencies (#13108)
refactor(server): simplify service dependency management
2024-10-02 10:54:35 -04:00
Jason Rasmussen
63437529e1
refactor(server): config file env (#13100) 2024-10-01 16:03:55 -04:00
Jason Rasmussen
a019fb670e
refactor(server): config service (#13066)
* refactor(server): config service

* fix: function renaming

---------

Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
2024-09-30 17:31:21 -04:00
Jason Rasmussen
7af6733665
refactor(server): move files to separate table (#11861) 2024-08-19 20:03:33 -04:00
Jason Rasmussen
8338657eaa
refactor(server): stacks (#11453)
* refactor: stacks

* mobile: get it built

* chore: feedback

* fix: sync and duplicates

* mobile: remove old stack reference

* chore: add primary asset id

* revert change to asset entity

* mobile: refactor mobile api

* mobile: sync stack info after creating stack

* mobile: update timeline after deleting stack

* server: update asset updatedAt when stack is deleted

* mobile: simplify action

* mobile: rename to match dto property

* fix: web test

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2024-08-19 12:37:15 -05:00
Jason Rasmussen
e84657192c
refactor: config caching (#10168) 2024-06-12 11:07:35 +00:00
Mert
e7c8501930
fix(server): search duplicates of the same asset type (#9747)
* search by type

* make sql

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2024-05-26 22:04:23 +00:00
Alex
60427f18ce
chore(server): return duplicate assets as group (#9576)
* chore(server): return duplicate assets as group

* file name
2024-05-18 13:15:56 -05:00
Mert
ff52300624
refactor(server): duplicate controller and service (#9542)
* duplicate controller and service

* change endpoint name

* fix search tests

* remove unused import

* add to index
2024-05-16 19:39:33 -04:00