70 Commits

Author SHA1 Message Date
Michel Heusschen
f9b1d1edaf
fix(server): better metadata extraction for images (#2653) 2023-06-03 21:55:30 -05:00
Michel Heusschen
1b301984dd
fix(server): handle invalid coordinates (#2648) 2023-06-02 11:29:12 -05:00
Michel Heusschen
94d0705607
refactor(server): change asset entity to date type (#2599)
* refactor(server): change asset entity to date type

* lower coverage threshold
2023-05-29 09:05:14 -05:00
Michel Heusschen
fd4357cf23
fix(server): exif time extraction (#2583) 2023-05-27 16:24:07 -05:00
Jason Rasmussen
e41e0df27e
fix(server): invalid exif date string (#2580) 2023-05-26 21:13:09 -05:00
Jason Rasmussen
1c2d83e2c7
refactor(server): job handlers (#2572)
* refactor(server): job handlers

* chore: remove comment

* chore: add comments for
2023-05-26 14:43:24 -05:00
Jason Rasmussen
065fb166c2
refactor(server): bull jobs (#2569)
* refactor(server): bull jobs

* chore: add comment

* chore: metadata test coverage

* fix typo

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-05-26 07:52:52 -05:00
Alex Phillips
7c1dae918d
feat(server): xmp sidecar metadata (#2466)
* initial commit for XMP sidecar support

* Added support for 'missing' metadata files to include those without sidecar files, now detects sidecar files in the filesystem for media already ingested but the sidecar was created afterwards

* didn't mean to commit default log level during testing

* new sidecar logic for video metadata as well

* Added xml mimetype for sidecars only

* don't need capture group for this regex

* wrong default value reverted

* simplified the move here - keep it in the same try catch since the outcome is to move the media back anyway

* simplified setter logic

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* simplified logic per suggestions

* sidecar is now its own queue with a discover and sync, updated UI for the new job queueing

* queue a sidecar job for every asset based on discovery or sync, though the logic is almost identical aside from linking the sidecar

* now queue sidecar jobs for each assset, though logic is mostly the same between discovery and sync

* simplified logic of filename extraction and asset instantiation

* not sure how that got deleted..

* updated code per suggestions and comments in the PR

* stat was not being used, removed the variable set

* better type checking, using in-scope variables for exif getter instead of passing in every time

* removed commented out test

* ran and resolved all lints, formats, checks, and tests

* resolved suggested change in PR

* made getExifProperty more dynamic with multiple possible args for fallbacks, fixed typo, used generic in function  for better type checking

* better error handling and moving files back to positions on move or save failure

* regenerated api

* format fixes

* Added XMP documentation

* documentation typo

* Merged in main

* missed merge conflict

* more changes due to a merge

* Resolving conflicts

* added icon for sidecar jobs

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-05-24 20:59:30 -05:00
Jason Rasmussen
a1f1e5bc37
fix(server): reverse geocoding delete dump logic (#2551) 2023-05-23 20:36:36 -05:00
Michel Heusschen
f1384fea58
feat(server): pagination for asset queries in jobs (#2516)
* feat(server): pagination for asset queries in jobs

* default mock value for getAll

* remove live photo name correction

* order paginated results by createdAt

* change log level

* move usePagination to domain
2023-05-22 13:05:06 -05:00
Jason Rasmussen
eefe5266a8
chore(server): remove unused filename (#2517) 2023-05-22 10:26:56 -05:00
Jason Rasmussen
e028cf9002
fix(server): reverse geocoding crash loop (#2489) 2023-05-20 21:39:12 -05:00
Alex
a9859bc029
feat: Add description (#2237)
* Added dto, logic to insert description and web implementation

* create text field and update on remote database

* Update description and save changes

* styling

* fix web test

* fix server test

* preserve description on metadata extraction job run

* handle exif info is null situation

* pr feedback

* format openapi spec

* update createAssetDto

* refactor logic to service

* move files

* only owner can update description

* Render description correctly in shared album

* Render description correctly in shared link

* disable description edit for not owner of asset on mobile

* localization and clean up

* fix test

* Uses providers for description text (#2244)

* uses providers for description text

* comments

* fixes initial data setting

* fixes notifier

---------

Co-authored-by: martyfuhry <martyfuhry@gmail.com>
2023-04-13 10:22:06 -05:00
Alex
258bc328e0
chore(server): better logging for error message (#2230)
* chore(server): better logging for error message

* pr feedback
2023-04-11 08:53:42 -05:00
Alex
a1a62b00a0
feat(server): add originalFileName to asset table (#2231) 2023-04-11 05:23:39 -05:00
Jason Rasmussen
4cb74f0fe4
refactor(server): reverse geocoding (#2167)
* refactor(server): reverse geocoding

* fix: nullable results
2023-04-04 17:23:07 -05:00
Jason Rasmussen
ec6a7ae97c
fix(server): do not link live photos across users (#2162) 2023-04-03 23:48:05 -05:00
AndreAle94
94b2ea9b5f
Add timezone to exif entity (#1894)
* Add timezone to exif entity

* Refactor logging

---------

Co-authored-by: Andrea Alemani <andrea.alemani94@gmail.com>
2023-04-02 14:11:24 -05:00
Jason Rasmussen
34d300d1da
refactor(server): flatten infra folders (#2120)
* refactor: flatten infra folders

* fix: database migrations

* fix: test related import

* fix: github actions workflow

* chore: rename schemas to typesense-schemas
2023-03-30 14:38:55 -05:00
Jason Rasmussen
b0d5c7035b
feat(server): apply storage migration after exif completes (#2093)
* feat(server): apply storage migraiton after exif completes

* feat: same for videos

* fix: migration for live photos
2023-03-28 15:04:11 -05:00
Jason Rasmussen
da5a6d2272
fix(cli): missing dep in immich cli (#2094)
* fix: missing dep in immich cli

* fix: imports
2023-03-28 11:29:20 -05:00
Michel Heusschen
4dafc74223
fix(server): invalid video duration format (#2058) 2023-03-23 09:27:29 -05:00
Jason Rasmussen
386eef046d
refactor(server): jobs (#2023)
* refactor: job to domain

* chore: regenerate open api

* chore: tests

* fix: missing breaks

* fix: get asset with missing exif data

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-03-20 10:55:28 -05:00
Jason Rasmussen
2ca560ebf8
feat(web,server): explore (#1926)
* feat: explore

* chore: generate open api

* styling explore page

* styling no result page

* style overlay

* style: bluring text on thumbnail card for readability

* explore page tweaks

* fix(web): search urls

* feat(web): use objects for things

* feat(server): filter by motion, sort by createdAt

* More styling

* better navigation

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
2023-03-05 14:44:31 -06:00
Jason Rasmussen
a5f49b065c
fix: duration string parsing (#1923) 2023-03-03 16:49:22 -06:00
Jason Rasmussen
0aaeab124d
feat(server)!: search via typesense (#1778)
* build: add typesense to docker

* feat(server): typesense search

* feat(web): search

* fix(web): show api error response message

* chore: search tests

* chore: regenerate open api

* fix: disable typesense on e2e

* fix: number properties for open api (dart)

* fix: e2e test

* fix: change lat/lng from floats to typesense geopoint

* dev: Add smartInfo relation to findAssetById to be able to query against it

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-03-02 20:47:08 -06:00
Michel Heusschen
830f4268c3
fix(server): exif extraction swapped params (#1914) 2023-03-01 10:55:24 -06:00
Jason Rasmussen
6c7679714b
refactor(server): jobs and processors (#1787)
* refactor: jobs and processors

* refactor: storage migration processor

* fix: tests

* fix: code warning

* chore: ignore coverage from infra

* fix: sync move asset logic between job core and asset core

* refactor: move error handling inside of catch

* refactor(server): job core into dedicated service calls

* refactor: smart info

* fix: tests

* chore: smart info tests

* refactor: use asset repository

* refactor: thumbnail processor

* chore: coverage reqs
2023-02-25 08:12:03 -06:00
Zack Pollard
d1ea6a897e
chore(server): typeorm definitions fix part 3 (#1796)
* chore(server): tidy up exif typeorm entity definition

* chore(server): tidy up shared link typeorm entity definition

* chore(server): tidy up smart info typeorm entity definition

* chore(server): tidy up tag typeorm entity definition

* ci: add job that checks typeorm migrations are correct and up-to-date
2023-02-20 01:50:27 +00:00
Zack Pollard
5ad4e5b614
infra(server)!: fix typeorm asset entity relations (#1782)
* fix: add correct relations to asset typeorm entity

* fix: add missing createdAt column to asset entity

* ci: run check to make sure generated API is up-to-date

* ci: cancel workflows that aren't for the latest commit in a branch

* chore: add fvm config for flutter
2023-02-19 16:44:53 +00:00
Jason Rasmussen
36197cca98
feat(server): auto-link live photos (#1761)
* feat(server): auto-link live photos

* fix: video extraction and linking
2023-02-16 01:41:51 -06:00
Jason Rasmussen
9428b2576b
refactor(server): asset service - upload asset (#1438)
* refactor: asset upload

* refactor: background service

* chore: tests

* Regenerate api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-01-30 10:14:13 -06:00
Alex
788b435f9b
feat(web/server): Add options to rerun job on all assets (#1422) 2023-01-26 22:50:22 -06:00
Alex
8b73c2bf8a
fix(server): Handle exposure time correctly (#1432) 2023-01-26 13:14:05 -06:00
Alex
b07891089f
feat(web/server) Add more options to public shared link (#1348)
* Added migration files

* Added logic for shared album level

* Added permission for EXIF

* Update shared link response dto

* Added condition to show download button

* Create and edit link with new parameter:

* Remove deadcode

* PR feedback

* More refactor

* Move logic of allow original file to service

* Simplify

* Wording
2023-01-21 22:15:16 -06:00
Jason Rasmussen
c0a6b3d5a3
refactor(server): system config (#1353)
* refactor(server): system config

* fix: jest circular import

* chore: ignore migrations in coverage report

* chore: tests

* chore: tests

* chore: todo note

* chore: remove vite config backup

* chore: fix redis hostname
2023-01-21 10:11:55 -06:00
Alex
3e4a14b299
chore(server) harden EXIF extraction (#1347)
* chore(server) Harden EXIF extraction

* Remove unused function in timeutil

* Remove deadcode
2023-01-17 13:41:00 -06:00
Skyler Mäntysaari
dff10e89fe
feat(server): Fix exif data parsing (#1326)
* Trying to get exifdata working with different lib.

* Got the new library working.

* Addressing PR comments.

* Removed not used vars and proper place for the eslint disable.

* Fix time-utils to use the exiftool-vendored lib.

Fixed also one test, as that would be valid.

* Using filename for timestamp as well if possible.

* Add new tests for time-utils.

* Remember to gracefully terminate the exiftool instance when not needed.

* eslint ignore...

* Apperantly Dockerfile changes were not pushed.

* feat(dockerfile): Tweak the Server Dockerfile

* feat(server): getTimestampFromFilename should return string or undefined.

* feat(server): If we don't have exifData or timestamp from filename, raise an error.

* Apparently test was already right, but my local system disagrees.

* More utilities for parsing and fix the timestampFromFilename.

It was returning an incorrect date as the regex doesn't seem to be the best for this as files named `IMG_0115.HEIC` will want to get parsed incorrectly due to it.

* feat(server/docker): Install perl as it seems to be required.

* feat(server): remember to include exposureTime and focalLength in new exif data.

* feat(server): Remove the parsing from filename as requested.

* feat(server): Import exiftool differently in time-utils.

* feat(server): Error handling when there is no exifData.

* feat(server): Fixes for the error handling when there is no exifData.

* feat(server): Remember to include modifyDate despite no exif.

* feat(server): Remember to include model of Camera.

* feat(server): Fixing up Exiftool usage.

Including proper logging for it, which had to be done in wrapped fashion due to it expecting all the logging levels which NextJS logger doesn't implement.

* feat(server): Do not use a wrapper for ExifTool logging.

* fix merge conflicts in metadata-extractor
2023-01-17 09:29:49 -06:00
Jason Rasmussen
693adf8488
refactor: job names (#1343)
* refactor: job names

* refactor: remove jobId
2023-01-17 08:43:45 -06:00
Jason Rasmussen
ba04b753de
refactor: logging (#1318) 2023-01-13 08:23:12 -06:00
Jason Rasmussen
131caa20eb
refactor(server): domain/infra (#1298)
* refactor: user repository

* refactor: user module

* refactor: move database into infra

* refactor(cli): use user core

* chore: import path

* chore: tests
2023-01-11 21:34:36 -05:00
Jason Rasmussen
d9fa6619e7
refactor(server): imports and repository tokens (#1220)
* refactor: entity imports

* refactor: rename user repository token

* chore: merge imports

* refactor: rename album repository token

* refactor: rename asset repository token

* refactor: rename tag repository token
2022-12-30 07:22:06 -06:00
Alex Tran
82b8313da0
Fix test 2022-10-07 09:16:45 -05:00
Alex Tran
4f7e764fa0
Fix typing 2022-10-07 09:15:05 -05:00
Alex
7587f858ae
feat(server/web) Add manual job trigger mechanism to the web (#767) 2022-10-06 11:25:54 -05:00
bo0tzz
854c214bc0
Fix: Use boolean comparison for DISABLE_REVERSE_GEOCODING config (#787) 2022-10-05 15:18:57 -05:00
Johannes Zellner
848781aef5
Provide a sensible dumpDirectory for the local-reverse-geocoder module (#759)
Fixes #758
2022-09-28 15:43:34 -05:00
Zack Pollard
c33775b944
feat(server): missing exif extract nightly task (#754)
* fix: nightly reverse geocoding task checking for mapbox

* refactor: remove file size from image processor and queue data

* feat: add missing exif nightly job

* Remove filesize requirement in assetUploadedProcessorName queue insertion

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2022-09-28 05:41:50 -05:00
Zack Pollard
040e02cfc5
fix(server): handle missing reverse geocoding admin zones (#742) 2022-09-23 10:14:42 -05:00
Zack Pollard
f377b64065
feat(server) Remove mapbox and use local reverse geocoding (#738)
* feat: local reverse geocoding implementation, removes mapbox

* Disable non-null tslintrule

* Disable non-null tslintrule

* Remove tsignore

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2022-09-22 21:50:05 -05:00