1
0
forked from Cutlery/immich
Commit Graph

298 Commits

Author SHA1 Message Date
Phlogi 8c6adf7157 feat(server): resolve duplicates (#25316)
* feat(web): Synchronize information from deduplicated images

* Added new settings menu to the the deduplication tab.
* The toggable options in the settings are synchronization of: albums, favorites, ratings, description, visibility and location.
* When synchronizing the albums, the resolved images will be added to all albums of the duplicates.
* When synchronizing the favorite status, the resolved images will be marked as favorite, if at least one selectable image is marked as favorite.
* When synchronizing the ratings, the highest rating from the selectable images will be applied to the resolved image.
* When synchronizing the description, all descriptions from the selectable images will be merged into one description for the resolved image.
* When synchronizing the visibility, the most restrictive visibility setting from the selectable images will be applied to the resolved image.
* When synchronizing the location, if exactly one unique location exists among the selectable images, this location will be applied to the resolved image.
* There is no additional UI for these settings to keep the visual clutter minimal. The settings are applied automatically based on the user's preferences.

* Replace addAssetToAlbums with copyAsset

* fix linter

* feat(web): add duplicate sync fields and fix typo

* feat(web): add tag sync and enhance duplicate resolution

This update introduces tag synchronization for duplicate resolution,
ensuring all unique tag IDs from duplicates are applied to kept assets.
The visibility sync logic is updated to use a simplified ordering, as the hidden status items will never show up in a duplicate set.
Album synchronization now merges albums directly via addAssetsToAlbums; as the approach with copyAsset API endpoint was ineffiecient.
Description, rating, and location sync logic is improved for correctness.
and deduplication. i18n strings were added / updated.

* feat(server): move duplicate resolution to backend with sync and stacking

Moves duplicate metadata synchronization from frontend to backend, enabling robust
batch operations and proper validation. This is an improved refactor of PR #13851.

New endpoints:
- POST /duplicates/resolve - batch resolve with configurable metadata sync
- POST /duplicates/stack - create stacks from duplicate groups
- GET /duplicates - now includes suggestedKeepAssetIds based on file size and EXIF

Key changes:
- Move sync logic (albums, tags, favorites, ratings, descriptions, location, visibility) to server
- Add server-side metadata merge policies with proper conflict resolution
- Replace client-side resolution logic with new backend endpoints
- Add comprehensive E2E tests (70+ test cases) and unit tests
- Update OpenAPI specs and TypeScript SDK

No breaking changes - only additions to existing API.

* feat(preferences): enable all duplicate sync settings by default

* chore: clean up

* chore: clean up

* refactor: rename & clean up

* fix: preference upgrade

* chore: linting

* refactor(e2e): use updateAssets API for setAssetDuplicateId

* fix: visibility sync logic in duplicate resolution

* fix(duplicate): write description to exifUpdate

Previously the duplicate resolution populated assetUpdate.description even
though description belongs to exif info.

* fix(duplicate): remove redundant updateLockedColumns wrapper

updateAllExif already computes lockedProperties via distinctLocked
using Object.keys(options). The wrapper added a lockedProperties key
to the options object, causing the spurious string 'lockedProperties'
to be stored in the lockedProperties array.

* fix(duplicate): write merged tags to asset_exif to survive metadata re-extraction

During duplicate resolution, replaceAssetTags correctly wrote merged tag
IDs to the tag_asset table, but never updated asset_exif.tags or locked
the tags property. The subsequent SidecarWrite → AssetExtractMetadata
chain calls applyTagList, which destructively replaces tag_asset rows
with whatever is in asset_exif.tags — still the original per-asset tags,
not the merged set.

Write merged tag values to asset_exif.tags via updateAllExif (which also
locks the property via distinctLocked), and queue SidecarWrite when tags
change so they persist to the sidecar file.

* docs(duplicates): clarify location and tag sync behavior

* refactor(duplicate): remove sync settings, always sync all metadata on resolve

Remove DuplicateSyncSettingsDto and the per-field sync toggles
(albums, favorites, rating, description, visibility, location, tags).
Duplicate resolution now unconditionally syncs all metadata from
trashed assets to kept assets.

- Remove DuplicateSyncSettingsDto and settings field from DuplicateResolveDto
- Update DuplicateService to always run all sync logic without conditionals
- Delete DuplicateSettingsModal.svelte and settings gear button from UI
- Remove DuplicateSettings type and duplicateSettings persisted store
- Update unit and e2e tests to remove settings from resolve requests

* docs: update duplicates utility to reflect automatic metadata sync

* docs(web): replace duplicates info modal with link to documentation

* chore: clean up

* fix: add missing type cast to jsonAgg in duplicate repository getAll

* fix: skip persisting rating=0 in duplicate merge to avoid unnecessary sidecar write

---------

Co-authored-by: Toni <51962051+EinToni@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jason@rasm.me>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2026-03-26 18:33:55 +00:00
Jonathan Jogenfors 22bf7c2005 feat(server): add checksum algorithm field (#26573)
* feat: add checksum algorithm field

* fix comments

* chore: rename migration

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2026-03-26 18:20:25 +00:00
Michel Heusschen 144a57ddff refactor(server): use helpers for shared link queries (#27088)
* fix(server): prevent album shared link from breaking after uploads

* update test

* add withSharedAssets helper

* remove options

* add more helpers

* update selects
2026-03-26 13:51:00 -04:00
Michel Heusschen 4812a2e2d8 fix(server): refresh unedited asset dimensions on metadata extraction (#27220) 2026-03-26 18:17:32 +01:00
Brandon Wees d8b39906f9 fix: incorrect asset face sync (#27243)
* fix: incorrect asset face sync

* chore: sync sql
2026-03-26 09:39:02 -05:00
Michel Heusschen b36911a16b fix(server): filter out empty search suggestions (#27292)
* fix(server): filter out empty search suggestions

* make sql
2026-03-26 09:36:04 -05:00
Jason Rasmussen 813d684aaa fix: shared link add to album (#27063) 2026-03-20 13:14:07 -05:00
Brandon Wees f3b7cd6198 refactor: move encoded video to asset files table (#26863)
* refactor: move encoded video to asset files table

* chore: update
2026-03-12 16:15:21 -04:00
Daniel Dietzler 34ce68095d chore: upgrade to kysely 0.28.11 (#26744) 2026-03-11 16:17:31 +01:00
Michel Heusschen 27f69b39b2 fix(server): use correct day ordering in timeline buckets (#26821)
* fix(web): sort timeline day groups received from server

* fix(server): use correct day ordering in timeline buckets
2026-03-11 08:49:35 -04:00
Min Idzelis 84abad564e fix(server): deduplicate shared links in getAll query (#26395) 2026-03-01 14:41:15 -05:00
Hao Xi aa87d1b9a3 fix: tune up the performance of the getByDayOfYear query. (#26495) 2026-02-27 16:51:19 -05:00
Nikhil Alapati 334fc250d3 fix(server): Live Photo migration bug when album is in template (#25329)
Co-authored-by: Nikhil Alapati <nikhilalapati@meta.com>
2026-02-27 12:46:55 +01:00
Brandon Wees 3c9fb651d0 feat(server): SyncAssetEditV1 (#26446)
* feat: SyncAssetEditV1

* fix: audit table import

* fix: sql tools table fetch

* fix: medium tests (wip)

* fix: circ dependency

* chore: finalize tests

* chore: codegen/lint

* fix: code review
2026-02-25 18:12:41 +00:00
Min Idzelis d14d0a9b9b feat: add isTransparent to db (#26413) 2026-02-23 21:33:52 +00:00
Brandon Wees e5722c525b feat: getAssetEdits respond with edit IDs (#26445)
* feat: getAssetEdits respond with edit IDs

* chore: cleanup typings for edit API

* chore: cleanup types with jason

* fix: openapi sync

* fix: factory
2026-02-23 20:57:57 +00:00
Brandon Wees e633bc3f24 fix: missing deletedAt and isVisible columns on mobile (#26414)
* feat: SyncAssetV2

* feat: mobile sync handling

* feat: request correct sync object based on server version

* fix: mobile queries

* chore: sync sql

* fix: test

* chore: switch to mapper

* fix: sql sync
2026-02-23 09:50:54 -05:00
Daniel Dietzler db4e7abf6d chore: refactor more queries (#25572)
* refactor: asset service queries

* chore: refactor more queries
2026-02-19 10:48:30 -05:00
Daniel Dietzler fd0338f89c refactor: asset service queries (#25535) 2026-02-19 08:54:28 -05:00
Daniel Dietzler d0ed76dc37 refactor: small face tests (#26340) 2026-02-19 08:51:18 -05:00
Jorge Montejo 1f8359ead4 fix: Download the edited version when downloading multiple photos (#26259)
* fix: download the edited version when downloading multiple photos

* test: update tests

* chore: clean up

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2026-02-18 21:47:45 +00:00
Brandon Wees 8c8b11f80c fix: person thumbnail generation on edited assets (#26112)
* fix: person thumbnail generation on edited assets

* chore: sql sync
2026-02-10 11:38:43 -06:00
Mert 7fa6f617f5 fix(server): thumbnail queueing (#26077)
* fix thumbnail queueing

* add bmp

* other isEdited column
2026-02-10 09:04:03 -06:00
Jason Rasmussen b3820c259e refactor: test factories (#25977) 2026-02-06 16:32:50 -05:00
Daniel Dietzler a356497d96 refactor: album service small tests (#25640) 2026-02-06 08:56:46 -05:00
Min Idzelis 092ebe01a5 fix: queue assets missing fullsize files for thumbnail regeneration (#25794)
* fix: queue assets missing fullsize files for thumbnail regeneration

* refactor: query

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2026-02-05 19:31:13 +00:00
Jason Rasmussen 6cdebdd3b3 fix(server): deleting stacked assets (#25874)
* fix(server): deleting stacked assets

* fix: log a warning when removing an empty directory fails
2026-02-04 17:33:37 +00:00
Jason Rasmussen 141be5cbc9 fix: memory generation (#25650) 2026-01-28 15:51:24 -06:00
Mert 9506398153 refactor(server): add isProgressive column (#25537)
* add isProgressive column

* don't select isProgressive by default

* linting

* exclude sidecars
2026-01-26 17:05:25 -05:00
Jason Rasmussen 4fedae4150 refactor: event manager (#25481)
* refactor: event manager

* fix: broken downloadFile endpoint
2026-01-23 18:02:23 -05:00
Daniel Dietzler 984fb12ada refactor: asset media service queries (#25477) 2026-01-23 16:07:57 -05:00
Brandon Wees 4bd01b70ff fix: asset edit sequence (#25457) 2026-01-22 12:41:01 -06:00
Daniel Dietzler 184f1a6d32 fix: tag update race condition (#25371) 2026-01-21 16:30:45 +01:00
Daniel Dietzler 1649d87360 refactor: extract isEdited into its own column in asset_file (#25358) 2026-01-21 16:08:21 +01:00
Brandon Wees 1b56bb84f9 fix: mobile edit handling (#25315)
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2026-01-19 12:22:53 -06:00
Daniel Dietzler 07675a2de4 feat: download original asset (#25302)
Co-authored-by: bwees <brandonwees@gmail.com>
2026-01-16 19:05:13 +00:00
Daniel Dietzler 5e3f5f2b55 fix: unlock properties after successful sidecar write (#25168) 2026-01-12 14:01:38 +01:00
Brandon Wees e8c80d88a5 feat: image editing (#24155) 2026-01-09 17:59:52 -05:00
Jason Rasmussen a2ba36c16d feat: bulk asset metadata endpoints (#25133) 2026-01-08 14:52:16 -05:00
Rahul Kumar Saini a16c5955d7 feat(server): Support camera make, model, and lensModel in Storage Template (#24650)
* add support for make, model, lensModel in storage template

* no pkg lock

* Apply suggestion from @danieldietzler

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* query and formatting

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2025-12-29 21:55:06 +00:00
Alex 28f6064240 feat: workflow ui (#24190)
* feat: workflow ui

* wip

* wip

* wip

* pr feedback

* refactor: picker field

* use showDialog directly

* better test

* refactor step selection modal

* move enable button to info form

* use  for Props

* pr feedback

* refactor ActionItem

* refactor ActionItem

* more refactor

* fix: new schemaformfield has value of the same type

* chore: clean up
2025-12-20 21:07:07 -06:00
Daniel Dietzler c15998e805 fix: asset update race condition (#24384)
* fix: asset update race condition

* fix: asset update race condition

* single statement

* update sql

* missed one

* fix `none` handling

* fix: tests

* chore: simplify update all assets

* fix: updating lockable properties

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-12-17 09:23:13 -06:00
Mert cffb68d1c4 fix(server): do not delete offline assets (#24355)
* do not delete isOffline assets

* update sql

* add medium test

* add normal delete test

* formatting
2025-12-03 14:19:26 -06:00
Jonathan Jogenfors 1bcf28c062 chore(server): sidecars in asset_files (#21199)
* fix: sidecar check job

* feat: move sidecars to asset_files

* feat: combine with handleSidecarCheck

* fix(server): improved method signatures for stack and sidecar copying

* fix(server): improved method signatures for stack and sidecar copying

* chore: clean up

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2025-12-02 13:31:43 -05:00
Alex 4dcc049465 feat: workflow foundation (#23621)
* feat: plugins

* feat: table definition

* feat: type and migration

* feat: add repositories

* feat: validate manifest with class-validator and load manifest info to database

* feat: workflow/plugin controller/service layer

* feat: implement workflow logic

* feat: make trigger static

* feat: dynamical instantiate plugin instances

* fix: access control and helper script

* feat: it works

* chore: simplify

* refactor: refactor and use queue for workflow execution

* refactor: remove unsused property in plugin-schema

* build wasm in prod

* feat: plugin loader in transaction

* fix: docker build arm64

* generated files

* shell check

* fix tests

* fix: waiting for migration to finish before loading plugin

* remove context reassignment

* feat: use mise to manage extism tools (#23760)

* pr feedback

* refactor: create workflow now including create filters and actions

* feat: workflow medium tests

* fix: broken medium test

* feat: medium tests

* chore: unify workflow job

* sign user id with jwt

* chore: query plugin with filters and action

* chore: read manifest in repository

* chore: load manifest from server configs

* merge main

* feat: endpoint documentation

* pr feedback

* load plugin from absolute path

* refactor:handle trigger

* throw error and return early

* pr feedback

* unify plugin services

* fix: plugins code

* clean up

* remove triggerConfig

* clean up

* displayName and methodName

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
Co-authored-by: bo0tzz <git@bo0tzz.me>
2025-11-14 20:05:05 +00:00
Sergey Katsubo 2c54b506b3 fix(server): include the previous year in memories for January 1, 2, 3 (#23832)
* Test memory creation in advance

Use year 2035 to make sure it's in the future of current time of a test run

* Use target year instead of current year when fetching assets during memory creation

This fixes an edge case of creating memories in advance when target year is
different from current year.
Example: job runs on 2025-12-31 (current year is 2025) and creates memories
to be shown on 2026-01-01 (target year is 2026). If using _current_ year in
calculation then range of years is capped at (2025 - 1 = 2024) thus excluding
2025-01-01 from created memories. With _target_ year it is (2026 - 1 = 2025),
so 2025-01-01 will be included in memories.

* Update sql queries
2025-11-12 15:38:03 -06:00
Jason Rasmussen aa69d89b9f fix: bad merge (#23610) 2025-11-04 16:22:45 -05:00
Jason Rasmussen 29c14a3f58 refactor: database column names (#23356) 2025-11-04 16:03:21 -05:00
Daniel Dietzler 4ae7cadeae feat: asset copy (#23172) 2025-10-29 08:43:47 -05:00
Kang 02b29046b3 feat: ocr (#18836)
* feat: add OCR functionality and related configurations

* chore: update labeler configuration for machine learning files

* feat(i18n): enhance OCR model descriptions and add orientation classification and unwarping features

* chore: update Dockerfile to include ccache for improved build performance

* feat(ocr): enhance OCR model configuration with orientation classification and unwarping options, update PaddleOCR integration, and improve response structure

* refactor(ocr): remove OCR_CLEANUP job from enum and type definitions

* refactor(ocr): remove obsolete OCR entity and migration files, and update asset job status and schema to accommodate new OCR table structure

* refactor(ocr): update OCR schema and response structure to use individual coordinates instead of bounding box, and adjust related service and repository files

* feat: enhance OCR configuration and functionality

- Updated OCR settings to include minimum detection box score, minimum detection score, and minimum recognition score.
- Refactored PaddleOCRecognizer to utilize new scoring parameters.
- Introduced new database tables for asset OCR data and search functionality.
- Modified related services and repositories to support the new OCR features.
- Updated translations for improved clarity in settings UI.

* sql changes

* use rapidocr

* change dto

* update web

* update lock

* update api

* store positions as normalized floats

* match column order in db

* update admin ui settings descriptions

fix max resolution key

set min threshold to 0.1

fix bind

* apply config correctly, adjust defaults

* unnecessary model type

* unnecessary sources

* fix(ocr): switch RapidOCR lang type from LangDet to LangRec

* fix(ocr): expose lang_type (LangRec.CH) and font_path on OcrOptions for RapidOCR

* fix(ocr): make OCR text search case- and accent-insensitive using ILIKE + unaccent

* fix(ocr): add OCR search fields

* fix: Add OCR database migration and update ML prediction logic.

* trigrams are already case insensitive

* add tests

* format

* update migrations

* wrong uuid function

* linting

* maybe fix medium tests

* formatting

* fix weblate check

* openapi

* sql

* minor fixes

* maybe fix medium tests part 2

* passing medium tests

* format web

* readd sql

* format dart

* disabled in e2e

* chore: translation ordering

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2025-10-27 14:09:55 +00:00