7 Commits

Author SHA1 Message Date
Joseph Milazzo
e7619e6b0a
Local Metadata Integration Part 1 (#817)
* Started with some basic plumbing with comic info parsing updating Series/Volume.

* We can now get chapter title from comicInfo.xml

* Hooked in the ability to store people into the chapter metadata.

* Removed no longer used imports, fixed up some foreign key constraints on deleting series with person linked.

* Refactored Summary out of the UI for Series into SeriesMetadata. Updated application to .net 6. There is a bug in metadata code for updating.

* Removed the parallel.ForEach with a normal foreach which lets us use async. For I/O heavy code, shouldn't change much.

* Refactored scan code to only check extensions with comic info, fixed a bug on scan events not using correct method name, removed summary field (still buggy)

* Fixed a bug where on cancelling a metadata request in modal, underlying button would get stuck in a disabled state.

* Changed how metadata selects the first volume to read summary info from. It will now select the first non-special volume rather than Volume 1.

* More debugging and found more bugs to fix

* Redid all the migrations as one single one. Fixed a bug with GetChapterInfo returning null when ChapterMetadata didn't exist for that Chapter.

Fixed an issue with mapper failing on GetChapterMetadata. Started work on adding people and a design for people.

* Fixed a bug where checking if file modified now takes into account if file has been processed at least once. Introduced a bug in saving people to series.

* Just made code compilable again

* Fixed up code. Now people for series and chapters add correctly without any db issues.

* Things are working, but I'm not happy with how the management of Person is. I need to take into account that 1 person needs to map to an image and role is arbitrary.

* Started adding UI code to showcase chapter metadata

* Updated workflow to be .NET 6

* WIP of updating card detail to show the information more clearly and without so many if statements

* Removed ChatperMetadata and store on the Chapter itself. Much easier to use and less joins.

* Implemented Genre on SeriesMetadata level

* Genres and People are now removed from Series level if they are no longer on comicInfo

* PeopleHelper is done with unit tests. Everything is working.

* Unit tests in place for Genre Helper

* Starting on CacheHelper

* Finished tests for ShouldUpdateCoverImage. Fixed and added tests in ArchiveService/ScannerService.

* CacheHelper is fully tested

* Some DI cleanup

* Scanner Service now calls GetComicInfo for books. Added ability to update Series Sort name from metadata files (mainly epub as comicinfo doesn't have a field)

* Forgot to move a line of code

* SortName now populates from metadata (epub only, ComicInfo has no tags)

* Cards now show the chapter title name if it's set on hover, else will default back to title.

* Fixed a major issue with how MangaFiles were being updated with LastModified, which messed up our logic for avoiding refreshes.

* Woohoo, more tests and some refactors to be able to test more services wtih mock filesystem. Fixed an issue where SortName was getting set as first chapter, but the Series was in a group.

* Refactored the MangaFile creation code into the DbFactory where we also setup the first LastModified update.

* Has file changed bug is now finally fixed

* Remove dead genres, refactor genre to use title instead of name.

* Refactored out a directory from ShouldUpdateCoverImage() to keep the code clean

* Unit tests for ComicInfo on BookService.

* Refactored series detail into it's own component

* Series-detail now received refresh metadata events to refresh what's on screen

* Removed references to Artist on PersonRole as it has no metadata mapping

* Security audit

* Fixed a benchmark

* Updated JWT Token generator to use new methods in .NET 6

* Updated all the docker and build commands to use net6.0

* Commented out sonar scan since it's not setup for net6.0 yet.
2021-12-02 09:02:34 -08:00
Joseph Milazzo
82b5b599e0
Foundational Cover Image Rework (#584)
* Updating wording on card item when total pages is 0, to be just "Cannot Read" since it could be a non-archive file

* Refactored cover images to be stored on disk. This first commit has the extraction to disk and the metadata service to handle updating when applicable.

* Refactored code to have the actual save to cover image directory done by ImageService.

* Implemented the ability to override cover images.

* Some cleanup on Image service

* Implemented the ability to cleanup old covers nightly

* Added a migration to migrate existing covers to new cover image format (files).

* Refactored all CoverImages to just be the filename, leaving the Join with Cover directory to higher level code.

* Ensure when getting user progress, we pick the first.

* Added cleanup cover images for deleted tags. Don't pull any cover images that are blank.

* After series update, clear out cover image. No change on UI, but just keeps things clear before metadata refresh hits

* Refactored image formats for covers to ImageService.

* Fixed an issue where after refactoring how images were stored, the cleanup service was deleting them after each scan.

* Changed how ShouldUpdateCoverImage works to check if file exists or not even if cover image is locked.

* Fixed unit tests

* Added caching back to cover images.

* Caching on series as well

* Code Cleanup items

* Ensure when checking if a file exists in MetadataService, that we join for cover image directory. After we scan library, do one last filter to delete any series that have 0 pages total.

* Catch exceptions so we don't run cover migration if this is first time run.

* After a scan, only clear out the cache directory and not do a deep clean.

* Implemented the ability to backup custom locked covers only.

* Fixed unit tests

* Trying to figure out why GA crashes when running MetadataServiceTests.cs

* Some debugging on GA tests not running

* Commented out tests that were causing issues in GA.

* Fixed an issue where series cover images wouldn't migrate

* Fixed the updating of links to actually do all series and not just locked
2021-09-21 19:15:29 -05:00
Joseph Milazzo
0d2d73e8ae
Bugfix/release cleanup (#512)
* Lots of cleanup on the warnings in the solution. Deprecated IsLastWriteLessThan and made a new method HasFileBeenModifiedSince.

* Added some tests for the new extension method.

* Changed filter import to use correct import

* Scan Series now uses Refresh Metadata for Series, rather than library one.

* Fixed an issue where cover generation wasn't properly taking forced update into consideration. Removed a case of cover generation for no reason.

* Fixed series downloads not triggering backend call
2021-08-21 10:03:47 -07:00
Joseph Milazzo
68bb5ed5a8
Feature/bookmark feedback (#508)
* ImageService had a stream reset before writting out to array. Added logging statment for updating series metadata. Removed ConcurencyCheck due to bad update issue for CollectionTag.

* Added a new screen which lets you quickly see all your bookmarks for a given user.

* Built user bookmark page in user settings. Moved user settings to it's own lazy loaded module. Removed unneded debouncing from downloader and just used throttleTime instead.

* Removed a not-yet implemented tab from series modal

* Fixed a bug in clear bookmarks and adjusted icons within anchors to have proper styling
2021-08-18 17:16:05 -07:00
Joseph Milazzo
3ed99afd32
Misc Bugfixes and Enhancements (#507)
* Removed some extra spam for the console

* Implemented the code to update RowVersion, which is our concurrency check

* Fixed a critical issue where more than one bookmark could occur for a given chapter due to a race condition. Now we use concurrency checks and we also gracefully allow more than one row, by only grabbing first.

* Cleaned up the logic for IHasConcurencyToken and updated the setters to be private.

* Lots of comments and when deleting a library, remove any user progress items for which chapters don't exist.

* When deleting a Series, cleanup user progress rows.

* Now after a scan of library, if a series is removed, collection tags are pruned as well if there are no longer any series bound to it.

* Updated the image on the Readme to show a better picture

* Small code cleanup to remove null check modifier as I check for null just before then

* Fixed images loading multiple times due to using function in binding with random. You can now click chapter images to read that chapter specifically.

* Fixed cards being different sizes when read vs unread

* Moved over Robbie's workflow changes from notifier. Commented out activity indicators as that is not shipping with this release.

* Remove code that isn't needed

* Reverted GA

* Changed GA to trigger only when HEAD is updated
2021-08-17 14:15:32 -07:00
Joseph Milazzo
2fd02f0d2b
Custom Cover Images (#499)
* Added some documentation. Removed Require Admin Role from Search Tags. Added Summary to be updated on UpdateTag.

* Added Swagger xml doc generation to beef up the documentation. Started adding xml comments to the APIs. This is a needed, slow task for upcoming Plugins system.

* Implemented the ability to upload a custom series image to override the existing cover image.

Refactored some code out to use ImageService and added more documentation

* When a page cache fails, delete cache directory so user can try to reload.

* Implemented the ability to lock a series cover image such that after user uploads something, it wont get refreshed by Kavita.

* Implemented the ability to reset cover image for series by unlocking

* Kick off a series refresh after a cover is unlocked.

* Ability to press enter to load a url

* Ability to reset selection

* Cleaned up cover chooser such that reset is nicer, errors inform user to use file upload, series edit modal now doesn't use scrollable body. Mobile tweaks. CoverImageLocked is now sent to the UI.

* More css changes to look better

* When no bookmarks, don't show both markups

* Fixed issues where images wouldn't refresh after cover image was changed.

* Implemented the ability to change the cover images for collection tags.

* Added property and API for chapter cover image update

* Added UI code to prepare for updating cover image for chapters. need to rearrange components

* Moved a ton of code around to separate card related screens into their own module.

* Implemented the ability to update a chapter/volume cover image

* Refactored action for volume to say edit to reflect modal action

* Fixed issue where after editing chapter cover image, the underlying card wouldn't update

* Fixed an issue where we were passing volumeId to the reset chapter lock. Changed some logic in volume cover image generation.

* Automatically apply when you hit reset cover image
2021-08-15 10:36:47 -07:00
Joseph Milazzo
d7d7f9b529
Collection Support (#234)
* Readme refactored to be more clean and clear, taking inspiration from wiki.js's readme.

* Initial backend for Collections and basic metadata implemented.

* More build flavors for Raspberry Pi users and updated Install since we don't need users to set their own JWT Token Key. Update a typo in appsettings.json file for prod.

* Fixed #224. Sort before getting a First?Last() chatper

* The rough ability to add and get series metadata and tags.

* Fix a bug on getting metadata for when it doesn't exist.

* Fixed a bug where flattening directories with some unique filenames could cause reading order of images to be out of order.

* Added a seed code to ensure all series have SeriesMetdata

* Ensure all instances of opening an epub is using "using" so we don't lock the file. When we have a malformed html file, log the issues and inform the user we can't open the file.

* Book reader now handles @Import "" statements in CSS and inlines the css into css file that references them. This allows for them to be scoped. In addition, if the html or body tag had classes, we now send back a single div with those classes.

* Fixed GetSeriesDtoForCollectionAsync which was not properly returning series

* Implemented cover image for collection tag. Fixed an issue in metadata update call.

* Add check for user access when resolving series for a collection tag. When asking for all tags, if the user is not an admin, only give promotoed tags back.

* Implemented updateTag api

* Implemented the ability to update series the tags have access to.

* Cleanup, sorting, and null check

* More sorting changes

* Ensure we can delete tags when editing a series tags

* Fix order of update to make sure a tag is properly deleted

* Code smells
2021-05-30 17:24:23 -05:00