1
0
forked from Cutlery/immich

45 Commits

Author SHA1 Message Date
Jason Rasmussen
0aaeab124d
feat(server)!: search via typesense (#1778)
* build: add typesense to docker

* feat(server): typesense search

* feat(web): search

* fix(web): show api error response message

* chore: search tests

* chore: regenerate open api

* fix: disable typesense on e2e

* fix: number properties for open api (dart)

* fix: e2e test

* fix: change lat/lng from floats to typesense geopoint

* dev: Add smartInfo relation to findAssetById to be able to query against it

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-03-02 20:47:08 -06:00
Michel Heusschen
830f4268c3
fix(server): exif extraction swapped params (#1914) 2023-03-01 10:55:24 -06:00
Jason Rasmussen
6c7679714b
refactor(server): jobs and processors (#1787)
* refactor: jobs and processors

* refactor: storage migration processor

* fix: tests

* fix: code warning

* chore: ignore coverage from infra

* fix: sync move asset logic between job core and asset core

* refactor: move error handling inside of catch

* refactor(server): job core into dedicated service calls

* refactor: smart info

* fix: tests

* chore: smart info tests

* refactor: use asset repository

* refactor: thumbnail processor

* chore: coverage reqs
2023-02-25 08:12:03 -06:00
Zack Pollard
d1ea6a897e
chore(server): typeorm definitions fix part 3 (#1796)
* chore(server): tidy up exif typeorm entity definition

* chore(server): tidy up shared link typeorm entity definition

* chore(server): tidy up smart info typeorm entity definition

* chore(server): tidy up tag typeorm entity definition

* ci: add job that checks typeorm migrations are correct and up-to-date
2023-02-20 01:50:27 +00:00
Zack Pollard
5ad4e5b614
infra(server)!: fix typeorm asset entity relations (#1782)
* fix: add correct relations to asset typeorm entity

* fix: add missing createdAt column to asset entity

* ci: run check to make sure generated API is up-to-date

* ci: cancel workflows that aren't for the latest commit in a branch

* chore: add fvm config for flutter
2023-02-19 16:44:53 +00:00
Jason Rasmussen
36197cca98
feat(server): auto-link live photos (#1761)
* feat(server): auto-link live photos

* fix: video extraction and linking
2023-02-16 01:41:51 -06:00
Jason Rasmussen
9428b2576b
refactor(server): asset service - upload asset (#1438)
* refactor: asset upload

* refactor: background service

* chore: tests

* Regenerate api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-01-30 10:14:13 -06:00
Alex
788b435f9b
feat(web/server): Add options to rerun job on all assets (#1422) 2023-01-26 22:50:22 -06:00
Alex
8b73c2bf8a
fix(server): Handle exposure time correctly (#1432) 2023-01-26 13:14:05 -06:00
Alex
b07891089f
feat(web/server) Add more options to public shared link (#1348)
* Added migration files

* Added logic for shared album level

* Added permission for EXIF

* Update shared link response dto

* Added condition to show download button

* Create and edit link with new parameter:

* Remove deadcode

* PR feedback

* More refactor

* Move logic of allow original file to service

* Simplify

* Wording
2023-01-21 22:15:16 -06:00
Jason Rasmussen
c0a6b3d5a3
refactor(server): system config (#1353)
* refactor(server): system config

* fix: jest circular import

* chore: ignore migrations in coverage report

* chore: tests

* chore: tests

* chore: todo note

* chore: remove vite config backup

* chore: fix redis hostname
2023-01-21 10:11:55 -06:00
Alex
3e4a14b299
chore(server) harden EXIF extraction (#1347)
* chore(server) Harden EXIF extraction

* Remove unused function in timeutil

* Remove deadcode
2023-01-17 13:41:00 -06:00
Skyler Mäntysaari
dff10e89fe
feat(server): Fix exif data parsing (#1326)
* Trying to get exifdata working with different lib.

* Got the new library working.

* Addressing PR comments.

* Removed not used vars and proper place for the eslint disable.

* Fix time-utils to use the exiftool-vendored lib.

Fixed also one test, as that would be valid.

* Using filename for timestamp as well if possible.

* Add new tests for time-utils.

* Remember to gracefully terminate the exiftool instance when not needed.

* eslint ignore...

* Apperantly Dockerfile changes were not pushed.

* feat(dockerfile): Tweak the Server Dockerfile

* feat(server): getTimestampFromFilename should return string or undefined.

* feat(server): If we don't have exifData or timestamp from filename, raise an error.

* Apparently test was already right, but my local system disagrees.

* More utilities for parsing and fix the timestampFromFilename.

It was returning an incorrect date as the regex doesn't seem to be the best for this as files named `IMG_0115.HEIC` will want to get parsed incorrectly due to it.

* feat(server/docker): Install perl as it seems to be required.

* feat(server): remember to include exposureTime and focalLength in new exif data.

* feat(server): Remove the parsing from filename as requested.

* feat(server): Import exiftool differently in time-utils.

* feat(server): Error handling when there is no exifData.

* feat(server): Fixes for the error handling when there is no exifData.

* feat(server): Remember to include modifyDate despite no exif.

* feat(server): Remember to include model of Camera.

* feat(server): Fixing up Exiftool usage.

Including proper logging for it, which had to be done in wrapped fashion due to it expecting all the logging levels which NextJS logger doesn't implement.

* feat(server): Do not use a wrapper for ExifTool logging.

* fix merge conflicts in metadata-extractor
2023-01-17 09:29:49 -06:00
Jason Rasmussen
693adf8488
refactor: job names (#1343)
* refactor: job names

* refactor: remove jobId
2023-01-17 08:43:45 -06:00
Jason Rasmussen
ba04b753de
refactor: logging (#1318) 2023-01-13 08:23:12 -06:00
Jason Rasmussen
131caa20eb
refactor(server): domain/infra (#1298)
* refactor: user repository

* refactor: user module

* refactor: move database into infra

* refactor(cli): use user core

* chore: import path

* chore: tests
2023-01-11 21:34:36 -05:00
Jason Rasmussen
d9fa6619e7
refactor(server): imports and repository tokens (#1220)
* refactor: entity imports

* refactor: rename user repository token

* chore: merge imports

* refactor: rename album repository token

* refactor: rename asset repository token

* refactor: rename tag repository token
2022-12-30 07:22:06 -06:00
Alex Tran
82b8313da0
Fix test 2022-10-07 09:16:45 -05:00
Alex Tran
4f7e764fa0
Fix typing 2022-10-07 09:15:05 -05:00
Alex
7587f858ae
feat(server/web) Add manual job trigger mechanism to the web (#767) 2022-10-06 11:25:54 -05:00
bo0tzz
854c214bc0
Fix: Use boolean comparison for DISABLE_REVERSE_GEOCODING config (#787) 2022-10-05 15:18:57 -05:00
Johannes Zellner
848781aef5
Provide a sensible dumpDirectory for the local-reverse-geocoder module (#759)
Fixes #758
2022-09-28 15:43:34 -05:00
Zack Pollard
c33775b944
feat(server): missing exif extract nightly task (#754)
* fix: nightly reverse geocoding task checking for mapbox

* refactor: remove file size from image processor and queue data

* feat: add missing exif nightly job

* Remove filesize requirement in assetUploadedProcessorName queue insertion

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2022-09-28 05:41:50 -05:00
Zack Pollard
040e02cfc5
fix(server): handle missing reverse geocoding admin zones (#742) 2022-09-23 10:14:42 -05:00
Zack Pollard
f377b64065
feat(server) Remove mapbox and use local reverse geocoding (#738)
* feat: local reverse geocoding implementation, removes mapbox

* Disable non-null tslintrule

* Disable non-null tslintrule

* Remove tsignore

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2022-09-22 21:50:05 -05:00
Alex
e5459b68ff
fix(server,web,mobile): Incorrectly record and show timestamp and time zone of the asset (#706)
Implemented a mechanism to extract the correct time zone from the GPS coordinate if presented in the file's EXIF, and to convert the timestamp to the correct UTC time so that the time will show correctly based on the mobile/web local time zone.
2022-09-22 15:58:17 -05:00
Alex
ccf792f9d3
fix(server): mismatch createdAt value in table and table (#688) 2022-09-13 20:12:42 -05:00
Alex
e79e92c60f
Added Log level to background service (#685) 2022-09-13 12:09:57 -05:00
Jaime Baez
1f4ba73da7 Add all server checks to CI - fix lint issues
CI will now run linter, type-checks and tests for the server.

All the lint issues have been fixed.
2022-09-08 11:07:27 +02:00
Thanh Pham
e745cb5e4b
fix(server): parse all img formats and enrich metadata (#547)
* fix(server): use file path instead buffer to reduce memory usage

fix undefined exif data

* fix(server): parse all img formats

* feat(server): enrich metadata

* Format oneliner condition

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2022-08-28 15:43:31 -05:00
Thanh Pham
dfaa4969da
Server - Fixed - Use file path instead buffer to reduce memory usage on EXIF extraction (#545)
fix undefined exif data
2022-08-28 11:09:24 -05:00
Thanh Pham
3b55cdc0be
refactor(server): move constant into common package (#522)
* refactor(server): move constant into common package

* refactor(server): re-arrange import statement in microservice module

* refactor(server): move app.config into common package

* fix(server): e2e testing
2022-08-23 07:34:21 -07:00
Thanh Pham
ba2cda8955
feat(server): support tiff uploading (#513)
* feat(server): suport tiff uploading

* remove unused variable

Co-authored-by: Alex <alex.tran1502@gmail.com>
2022-08-22 12:49:17 -07:00
Thanh Pham
7f9f825589
fix(server): correct media info (#508)
* fix(server): correct media info

* fix(server): video metadata
2022-08-20 22:58:47 -07:00
Thanh Pham
9c30d58b10
feat(server): preserve caption fields and extract mediainfo for video (#505)
* feat(server): preserve caption fields and extract mediainfo for video

* Fixed Geocoding missing info leads to fail EXIF extraction for the whole file

Co-authored-by: Alex <alex.tran1502@gmail.com>
2022-08-20 16:31:37 -07:00
Alex
c60e852226
fix 331 (#335)
* fix #331 - Video with no date information in EXIF upload via web caused mobile client not able to render other assets
2022-07-12 16:34:43 -05:00
Alex
357f7d1c31
Added schedule job to perform reverse geocoding if key is added after backing up assets (#305) 2022-07-04 15:16:39 -05:00
Alex
97d9b80baa
Added creation date for video from ffmpeg.prob (#303) 2022-07-04 13:44:43 -05:00
Alex
76bf1c0379
Remove thumbnail generation on mobile app (#292)
* Remove thumbnail generation on mobile

* Remove tconditions for missing thumbnail on the backend

* Remove console.log

* Refactor queue systems

* Convert queue and processor name to constant

* Added corresponding interface to job queue
2022-07-02 21:06:36 -05:00
Zack Pollard
c071e64a7e
infra: switch port to 3003 for machine learning container (#290)
* infra: switch port to 3003 for machine learning container

fixes #289

* Changed port of machine-learning-endpoint to match with new port

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2022-07-01 10:20:04 -05:00
Alex
663f12851e
Fixed filename duplication when upload from web (#288)
* Fixed filename duplication when upload from web

* Fixed cosmetic of detail panel view
2022-06-30 20:43:33 -05:00
Jaime Baez
c918f5b001
Set TypeScript to strict mode and fix issues related to server types (#261)
* Fix lint issues and some other TS issues

- set TypeScript in strict mode
- add npm commands to lint / check code
- fix all lint issues
- fix some TS issues
- rename User reponse DTO to make it consistent with the other ones
- override Express/User interface to use UserResponseDto interface
 This is for when the accessing the `user` from a Express Request,
 like in `asset-upload-config`

* Fix the rest of TS issues

- fix all the remaining TypeScript errors
- add missing `@types/mapbox__mapbox-sdk` package

* Move global.d.ts to server `src` folder

* Update AssetReponseDto duration type

This is now of type `string` that defaults to '0:00:00.00000' if not set
which is what the mobile app currently expects

* Set context when logging error in asset.service

Use `ServeFile` as the context for logging an error when
asset.resizePath is not set

* Fix wrong AppController merge conflict resolution

`redirectToWebpage` was removed in main as is no longer used.
2022-06-25 12:53:06 -05:00
Alex
1e3464fe47
Feature - Add upload functionality on Web (#231)
* Added file selector

* Extract metadata to upload files to the web

* Added request for uploading

* Generate jpeg/Webp thumbnail for asset uploaded without thumbnail data

* Added generating thumbnail for video and WebSocket broadcast after thumbnail is generated

* Added video length extraction

* Added Uploading Panel

* Added upload progress store and styling the uploaded asset

* Added condition to only show upload panel when there is upload in progress

* Remove asset from the upload list after successfully uploading

* Added WebSocket to listen to upload event on the web

* Added mechanism to check for existing assets before uploading on the web

* Added test workflow

* Update readme
2022-06-19 08:16:35 -05:00
Alex
a8e723d722
Fixed ENABLE_MAPBOX value is ignored (#223) 2022-06-12 18:19:53 -05:00
Alex
a8220172f8
WIP refactor container and queuing system (#206)
* refactor microservices to machine-learning

* Update tGithub issue template with correct task syntax

* Added microservices container

* Communicate between service based on queue system

* added dependency

* Fixed problem with having to import BullQueue into the individual service

* Added todo

* refactor server into monorepo with microservices

* refactor database and entity to library

* added simple migration

* Move migrations and database config to library

* Migration works in library

* Cosmetic change in logging message

* added user dto

* Fixed issue with testing not able to find the shared library

* Clean up library mapping path

* Added webp generator to microservices

* Update Github Action build latest

* Fixed issue NPM cannot install due to conflict witl Bull Queue

* format project with prettier

* Modified docker-compose file

* Add GH Action for Staging build:

* Fixed GH action job name

* Modified GH Action to only build & push latest when pushing to main

* Added Test 2e2 Github Action

* Added Test 2e2 Github Action

* Implemented microservice to extract exif

* Added cronjob to scan and generate webp thumbnail  at midnight

* Refactor to ireduce hit time to database when running microservices

* Added error handling to asset services that handle read file from disk

* Added video transcoding queue to process one video at a time

* Fixed loading spinner on web while loading covering the info panel

* Add mechanism to show new release announcement to web and mobile app (#209)

* Added changelog page

* Fixed issues based on PR comments

* Fixed issue with video transcoding run on the server

* Change entry point content for backward combatibility when starting up server

* Added announcement box

* Added error handling to failed silently when the app version checking is not able to make the request to GITHUB

* Added new version announcement overlay

* Update message

* Added messages

* Added logic to check and show announcement

* Add method to handle saving new version

* Added button to dimiss the acknowledge message

* Up version for deployment to the app store
2022-06-11 16:12:06 -05:00