20 Commits

Author SHA1 Message Date
Joseph Milazzo
b6a38bbd86
Lots of Bugfixes (#1426)
* Fixed bookmarks not being able to load due to missing [AllowAnonymous]

* Downgraded Docnet to 2.4.0-alpha2 which is the version we added our patches to. This might fix reports of broken PDF reading on ARM

* Updated all but one api in collections to admin only policy

* Ensure all config folders are created or exist on first load

* Ensure plugins can authenticate

* Updated some headers we use on Kavita to tighten security.

* Tightened up cover upload flow to restrict more APIs to only the admin

* Enhanced the reset password flow to ensure that the user passes their existing password in (if already authenticated). Admins can still change other users without having existing password.

* Removed an additional copy during build and copied over the prod appsettings and not Development.

* Fixed up the caching mechanism for cover resets and migrated to profiles. Left an etag filter for reference.

* Fixed up manual jump key calculation to include period in #

* Added jumpbar to reading lists page

* Fixed a double scrollbar on library detail page

* Fixed weird scroll issues with want to read

* Fixed a bug where remove from want to read list wasn't hooked up on series card

* Cleaned up Clear bookmarks to use a dedicated api for bulk clearing. Converted Bookmark page to OnPush.

* Fixed jump bar being offset when clicking a jump key

* Ensure we don't overflow on add to reading list

* Fixed a bad name format on reading list items
2022-08-11 18:16:31 -07:00
Joseph Milazzo
1ee6c3e506
Fixed access issues around new Authorize scheme. (#1417) 2022-08-08 15:00:10 -07:00
Joseph Milazzo
88b5ebeb69
Security Hotfix (#1415)
* Updated ngx-extended-pdf-viewer to 14.5.2 + misc security vuln

* Hooked up remove from want to read AND fixed a bug in the logic that was removing everything BUT what was passed.

Allow for bookmarks to have date info for better ordering.

* Implemented a quick way to set darkneses level on manga reader for when nightlight just isn't dark enough

* Added Japanese Series name support in the Parser

* Updated our security file with our Huntr.

* Fixed a security vulnerability where through the API, an unauthorized user could delete/modify reading lists that did not belong to them.

Fixed a bug where when creating a reading list with the name of another users, the API would throw an exception (but reading list would still get created)

* Ensure all reading list apis are authorized

* Ensured all APIs require authentication, except those that explicitly don't. All APIs are default requiring Authentication.

Fixed a security vulnerability which would allow a user to take over an admin account.

* Fixed a bug where cover-upload would accept filenames that were not expected.

* Explicitly check that a user has access to the pdf file before we serve it back.

* Enabled lock out when invalid user auth occurs. After 5 invalid auths, the user account will be locked out for 10 mins.
2022-08-08 13:47:37 -07:00
Joseph Milazzo
af4f35da5b
Better Caching & Global Downloads (#1372)
* Fixed a bug where cache TTL was using a field which always was 0.

* Updated Scan Series task (from UI) to always re-calculate what's on file and not rely on last update. This leads to more reliable results, despite extra overhead.

* Added image range processing on images for the reader, for slower networks or large files

* On manga (single) try to use prefetched image, rather than re-requesting an image on pagination

* Reduced some more latency when rendering first page of next chapter via continuous reading mode

* Fixed a bug where metadata filter, after updating a typeahead, collapsing filter area then re-opening, the filter would still be applied, but the typeahead wouldn't show the modification.

* Coded an idea around download reporting, commiting for history, might not go with it.

* Refactored the download indicator into it's own component. Cleaning up some code for download within card component

* Another throw away commit. Put in some temp code, not working but not sure if I'm ditching entirely.

* Updated download service to enable range processing (so downloads can resume) and to reduce re-zipping if we've just downloaded something.

* Refactored events widget download indicator to the correct design. I will be moving forward with this new functionality.

* Added Required fields to ProgressDTO

* Cleaned up the event widget and updated existing download progress to indicate preparing the download, rather than the download itself.

* Updated dependencies for security alerts

* Refactored all download code to be streamlined and globally handled

* Updated ScanSeries to find the highest folder path before library, not just within the files. This could lead to scan series missing files due to nested folders on same parent level.

* Updated the caching code to use a builtin annotation. Images are now caching correctly.

* Fixed a bad redirect on an auth guard

* Tweaked how long we allow cache for, as the cover update now doesn't work well.

* Fixed a bug on downloading bookmarks from multiple series, where it would just choose the first series id for the temp file.

* Added an extra check for downloading bookmarks

* UI Security updates, Fixed a bug on bookmark reader, the reader on last page would throw some errors and not show No Next Chapter toast.

* After scan, clear temp

* Code smells
2022-07-13 07:45:14 -07:00
Joseph Milazzo
2ab0aedd22
Swagger, Tachiyomi, and some new settings (#1331)
* Fixed up swagger generation

* Updated Tachiyomi's latest-chapter to hopefully solve some sync issues.

* Fixed #1279 with table of contents due to new EPubReader

* When errors occur, show the event widget icon in red

* Lots of documentation added and tweaked some wording around backups and swagger

* For promidius

* Return proper ChapterDTO

* Hacks for Promidius

* Cleanup code

* No loose leaf, send max chapter

* One more encode change

* Implemented code per promiduius' requirements

* Fixed a bug in the epub parsing where even if you had a series index and series group, but didn't have the series in the title, Kavita wouldn't group them properly.

* Removed some extra comment

* Implemented the ability to change a library's type after it's been setup. This displays a warning explaining the dangers of it.

* Removed some whitespace

* Blur descriptions based on read status for list item view to avoid spoilers

* Tweaked placement of a tooltip due to new series detail styles

* Hooked up a user preference for bluring unread summaries. Fixed a bug in refresh token where we would cause re-authentication when it shouldn't be needed.
2022-06-25 15:52:21 -07:00
Joseph Milazzo
3bbb02f574
On Deck + Misc Fixes and Changes (#1215)
* Added playwright and started writing e2e tests.

* To make things easy, disabled other browsers while I get confortable. Added a login flow (assumes my dev env)

* More tests on login page

* Lots more testing code, trying to figure out auth code.

* Ensure we don't track DBs inside config

* Added a new date property for when chapters are added to a series which helps with OnDeck calculations. Changed a lot of heavy api calls to use IEnumerable to stream repsonse to UI.

* Fixed OnDeck with a new field for when last chapter was added on Series. This is a streamlined way to query.

Updated Reading List with NormalizedTitle, CoverImage, CoverImageLocked.

* Implemented the ability to read a random item in the reading list and for the reading list to be intact for order.

* Tweaked the style for webtoon to not span the whole width, but use max width

* When we update a cover image just send an event so we don't need to have logic for when updates occur

* Fixed a bad name for entity type on cover updates

* Aligned the edit collection tag modal to align with new tab design

* Rewrote code for picking the first file for metadata to ensure it always picks the correct file, esp if the first chapter of a series starts with a float (1.1)

* Refactored setting LastChapterAdded to ensure we do it on the Series.

* Updated Chapter updating in scan loop to avoid nested for loop and an additional loop.

* Fixed a bug where locked person fields wouldn't persist between scans.

* Updated Contributing to reflect how to view the swagger api
2022-04-11 15:43:40 -07:00
Joseph Milazzo
e41b455d09
Bugfixes and Cover Chooser Upgrades (#1146)
* Fixed a bug where GetNextChapter would return a loose leaf chapter from a special when it should return nothing.

* Fixed a bug in events widget when an update comes in after a user refreshes, the active event counter could get out of sync, thus showing "Nothing going on here"

Refactored the events widget to be named appropriately.

* Refactored code to have errors during threaded tasks propagate to the UI via events widget (css still needed).

Removed ScanLibraryError in favor of generic Error event.

* Fixed up some code and added ability to remove the event from events widget

* Fixed a bug where modifiying certain fields, like summary, wouldn't lock the field

* Fixed a few bugs where lock state was not being set in the DB correctly nor were certain combinations of locking fields and editing fields.

* Removed debug code

* Updated the discord alert to tag new group

* Refactored cover upload to actually handle uploading a temp file via url on the backend so that users can user change cover by url. Fixed up some bugs that occured when chaning the image container in a previous PR.

* Code cleanup

* Cleaned up the css on the error items

* Code cleanup
2022-03-16 15:02:24 -07:00
Joseph Milazzo
ba77954d5c
Metadata Editing from the UI! (#1135)
* Added the skeleton code for layout, hooked up Age Rating, Publication Status, and Tags

* Tweaked message of Scan service to Finished scan of to better indicate the total scan time

* Hooked in foundation for person typeaheads

* Fixed people not populating typeaheads on load

* For manga/comics, when parsing, set the SeriesSort from ComicInfo if it exists.

* Implemented the ability to override and create new genre tags. Code is ready to flush out the rest.

* Ability to update metadata from the UI is hooked up. Next is locking.

* Updated typeahead to allow for non-multiple usage. Implemented ability to update Language tag in Series Metadata.

* Fixed a bug in GetContinuePoint for a case where we have Volumes, Loose Leaf chapters and no read progress.

* Added ETag headers on Images to allow for better caching (bookmarks and images in manga reader)

* Built out UI code to show locked indication to user

* Implemented Series locking and refactored a lot of styles in typeahead to make the lock setting work, plus misc cleanup.

* Added locked properties to dtos. Updated typeahead loading indicator to not interfere with close button if present

* Hooked up locking flags in UI

* Integrated regular field locking/unlocking

* Removed some old code

* Prevent input group from wrapping

* Implemented some basic layout for metadata on volume/chapter card modal. Refactored out all metadata from Chapter object in terms of UI and put into a separate call to ensure speedy delivery and simplicity of code.

* Refactored code to hide covers section if not an admin

* Implemented ability to modify a chapter/volume cover from the detail modal

* Removed a few variables and change cover image modal

* Added bookmark to single chapter view

* Put a temp fix in for a ngb v12 z-index bug (reported). Bumped ngb to 12.0 stable and fixed some small rendering bugs

* loading buttons ftw

* Lots of cleanup, looks like the story is finished

* Changed action name from Info to Details

* Style tweaks

* Fixed an issue where Summary would assume it's locked due to a subscription firing on setting the model

* Fixed some misc bugs

* Code smells

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2022-03-04 13:04:15 -08:00
Joseph Milazzo
a1a6333f09
Bookmark Refactor (#893)
* Fixed a bug which didn't take sort direction when not changing sort field

* Added foundation for Bookmark refactor

* Code broken, need to take a break. Issue is Getting bookmark image needs authentication but UI doesn't send.

* Implemented the ability to send bookmarked files to the web. Implemented ability to clear bookmarks on disk on a re-occuring basis.

* Updated the bookmark design to have it's own card that is self contained. View bookmarks modal has been updated to better lay out the cards.

* Refactored download bookmark codes to select files from bookmark directory directly rather than open underlying files.

* Wrote the basic logic to kick start the bookmark migration.

Added Installed Version into the DB to allow us to know more accurately when to run migrations

* Implemented the ability to change the bookmarks directory

* Updated all references to BookmarkDirectory to use setting from the DB.

Updated Server Settings page to use 2 col for some rows.

* Refactored some code to DirectoryService (hasWriteAccess) and fixed up some unit tests from a previous PR.

* Treat folders that start with ._ as blacklisted.

* Implemented Reset User preferences. Some extra code to prep for the migration.

* Implemented a migration for existing bookmarks to using new filesystem based bookmarks
2022-01-05 09:56:49 -08:00
Joseph Milazzo
bbe8f800f6
.NET 6 Coding Patterns + Unit Tests (#823)
* Refactored all files to have Interfaces within the same file. Started moving over to file-scoped namespaces.

* Refactored common methods for getting underlying file's cover, pages, and extracting into 1 interface.

* More refactoring around removing dependence on explicit filetype testing for getting information.

* Code is buildable, tests are broken. Huge refactor (not completed) which makes most of DirectoryService testable with a mock filesystem (and thus the services that utilize it).

* Finished porting DirectoryService to use mocked filesystem implementation.

* Added a null check

* Added a null check

* Finished all unit tests for DirectoryService.

* Some misc cleanup on the code

* Fixed up some bugs from refactoring scan loop.

* Implemented CleanupService testing and refactored more of DirectoryService to be non-static.

Fixed a bug where cover file cleanup wasn't properly finding files due to a regex bug.

* Fixed an issue in CleanupBackup() where we weren't properly selecting database files older than 30 days. Finished CleanupService Tests.

* Refactored Flatten and RemoveNonImages to directory service to allow CacheService to be testable.

* Finally have CacheService tested. Rewrote GetCachedPagePath() to be much more straightforward & performant.

* Updated DefaultParserTests.cs to contain all existing tests and follow new test layout format.

* All tests fixed up
2021-12-05 08:58:53 -08:00
Joseph Milazzo
f17d89ea47
ComicInfo Refactor (#636)
* Implemented methods to parse out the whole ComicInfo file and a mock ComicInfo from epub files.

* Removed unused imports. ScanSeries needs to cleanup deleted chapters and call RefreshMetadata. Ensure after scan we cleanup tags without any series.

* Random cleanup

* Added some comments about data getting stale and not updating.

* Removed old Summary methods in favor of the ComicInfo versions

* Added a missing property

* Fixed unit test
2021-10-04 16:10:48 -07:00
Joseph Milazzo
a872165747
Feature/image rework cleanup (#589)
* Added volume migrations. Added parser case for "Chapter 63 - The Promise Made for 520 Cenz.cbr"

* Added some info statements for when full library scans occur. For image apis, return the name of the file to aid in caching.

* When managing users, show the current logged in user at the top of the list. Added a message when no libraries have been setup but you are trying to add a user to a library.
2021-09-22 06:00:14 -07:00
Joseph Milazzo
82b5b599e0
Foundational Cover Image Rework (#584)
* Updating wording on card item when total pages is 0, to be just "Cannot Read" since it could be a non-archive file

* Refactored cover images to be stored on disk. This first commit has the extraction to disk and the metadata service to handle updating when applicable.

* Refactored code to have the actual save to cover image directory done by ImageService.

* Implemented the ability to override cover images.

* Some cleanup on Image service

* Implemented the ability to cleanup old covers nightly

* Added a migration to migrate existing covers to new cover image format (files).

* Refactored all CoverImages to just be the filename, leaving the Join with Cover directory to higher level code.

* Ensure when getting user progress, we pick the first.

* Added cleanup cover images for deleted tags. Don't pull any cover images that are blank.

* After series update, clear out cover image. No change on UI, but just keeps things clear before metadata refresh hits

* Refactored image formats for covers to ImageService.

* Fixed an issue where after refactoring how images were stored, the cleanup service was deleting them after each scan.

* Changed how ShouldUpdateCoverImage works to check if file exists or not even if cover image is locked.

* Fixed unit tests

* Added caching back to cover images.

* Caching on series as well

* Code Cleanup items

* Ensure when checking if a file exists in MetadataService, that we join for cover image directory. After we scan library, do one last filter to delete any series that have 0 pages total.

* Catch exceptions so we don't run cover migration if this is first time run.

* After a scan, only clear out the cache directory and not do a deep clean.

* Implemented the ability to backup custom locked covers only.

* Fixed unit tests

* Trying to figure out why GA crashes when running MetadataServiceTests.cs

* Some debugging on GA tests not running

* Commented out tests that were causing issues in GA.

* Fixed an issue where series cover images wouldn't migrate

* Fixed the updating of links to actually do all series and not just locked
2021-09-21 19:15:29 -05:00
Joseph Milazzo
83f8e25478
Continuous Reading for Webtoons & I Just Couldn't Stop Coding (#574)
* Fixed an issue from perf tuning where I forgot to send Pages to frontend, breaking reader.

* Built out continuous reading for webtoon reader. Still has some issues with triggering.

* Refactored GetUserByUsernameAsync to have a new flavor and allow the caller to pass in bitwise flags for what to include. This has a get by username or id variant. Code is much cleaner and snappier as we avoid many extra joins when not needed.

* Cleanup old code from UserRepository.cs

* Refactored OPDS to use faster API lookups for User

* Refactored more code to be cleaner and faster.

* Refactored GetNext/Prev ChapterIds to ReaderService.

* Refactored Repository methods to their correct entity repos.

* Refactored DTOs and overall cleanup of the code.

* Added ability to press 'b' to bookmark a page

* On hitting last page, save progress forcing last page to be read. Adjusted logic for the top and bottom spacers for triggering next/prev chapter load

* When at top or moving between chapters, scrolling down then up will now trigger page load. Show a toastr to inform the user of a change in chapter (it can be really fast to switch)

* Cleaned up scroll code

* Fixed an issue where loading a chapter with last page bookmarked, we'd load lastpage - 1

* Fixed last page of webtoon reader not being resumed on loading said chapter due to a difference in how max page is handled between infinite scroller and manga reader.

* Removed some comments

* Book reader shouldn't look at left/right tap to paginate elems for position bookmarking. Missed a few areas for saving while in incognito mode

* Added a benchmark to test out a sort code

* Updated the read status on reading list to use same style as other places

* Refactored GetNextChapterId to bring the average response time from 1.2 seconds to 400ms.

* Added a filter to add to list when there are more than 5 reading lists

* Added download reading list (will be removed, just saving for later). Fixes around styling on reading lists

* Removed ability to download reading lists

* Tweaked the logic for infinite scroller to be much smoother loading next/prev chapter. Added a bug marker for a concurrency bug.

* Updated the top spacer so that when you hit the top, you stay at the page height and can now just scroll up.

* Got the logic for scrolling up. Now just need the CSS then cont infinite scroller will be working

* More polishing on infinite scroller

* Removed IsSpecial on volumeDto, which is not used anywhere.

* Cont Reading inf scroller edition is done.

* Code smells and fixed package.json explore script
2021-09-11 11:47:12 -07:00
Joseph Milazzo
2fd02f0d2b
Custom Cover Images (#499)
* Added some documentation. Removed Require Admin Role from Search Tags. Added Summary to be updated on UpdateTag.

* Added Swagger xml doc generation to beef up the documentation. Started adding xml comments to the APIs. This is a needed, slow task for upcoming Plugins system.

* Implemented the ability to upload a custom series image to override the existing cover image.

Refactored some code out to use ImageService and added more documentation

* When a page cache fails, delete cache directory so user can try to reload.

* Implemented the ability to lock a series cover image such that after user uploads something, it wont get refreshed by Kavita.

* Implemented the ability to reset cover image for series by unlocking

* Kick off a series refresh after a cover is unlocked.

* Ability to press enter to load a url

* Ability to reset selection

* Cleaned up cover chooser such that reset is nicer, errors inform user to use file upload, series edit modal now doesn't use scrollable body. Mobile tweaks. CoverImageLocked is now sent to the UI.

* More css changes to look better

* When no bookmarks, don't show both markups

* Fixed issues where images wouldn't refresh after cover image was changed.

* Implemented the ability to change the cover images for collection tags.

* Added property and API for chapter cover image update

* Added UI code to prepare for updating cover image for chapters. need to rearrange components

* Moved a ton of code around to separate card related screens into their own module.

* Implemented the ability to update a chapter/volume cover image

* Refactored action for volume to say edit to reflect modal action

* Fixed issue where after editing chapter cover image, the underlying card wouldn't update

* Fixed an issue where we were passing volumeId to the reset chapter lock. Changed some logic in volume cover image generation.

* Automatically apply when you hit reset cover image
2021-08-15 10:36:47 -07:00
Joseph Milazzo
d7d7f9b529
Collection Support (#234)
* Readme refactored to be more clean and clear, taking inspiration from wiki.js's readme.

* Initial backend for Collections and basic metadata implemented.

* More build flavors for Raspberry Pi users and updated Install since we don't need users to set their own JWT Token Key. Update a typo in appsettings.json file for prod.

* Fixed #224. Sort before getting a First?Last() chatper

* The rough ability to add and get series metadata and tags.

* Fix a bug on getting metadata for when it doesn't exist.

* Fixed a bug where flattening directories with some unique filenames could cause reading order of images to be out of order.

* Added a seed code to ensure all series have SeriesMetdata

* Ensure all instances of opening an epub is using "using" so we don't lock the file. When we have a malformed html file, log the issues and inform the user we can't open the file.

* Book reader now handles @Import "" statements in CSS and inlines the css into css file that references them. This allows for them to be scoped. In addition, if the html or body tag had classes, we now send back a single div with those classes.

* Fixed GetSeriesDtoForCollectionAsync which was not properly returning series

* Implemented cover image for collection tag. Fixed an issue in metadata update call.

* Add check for user access when resolving series for a collection tag. When asking for all tags, if the user is not an admin, only give promotoed tags back.

* Implemented updateTag api

* Implemented the ability to update series the tags have access to.

* Cleanup, sorting, and null check

* More sorting changes

* Ensure we can delete tags when editing a series tags

* Fix order of update to make sure a tag is properly deleted

* Code smells
2021-05-30 17:24:23 -05:00
Joseph Milazzo
71d3af5f52 LastModified on Volumes wasn't updating, validated it does update when data is changed. 2021-04-01 15:59:07 -05:00
Joseph Milazzo
d73bd22db2 Some code cleanup 2021-03-23 12:22:47 -05:00
Joseph Milazzo
af35d8aad5 Cleanup of lazy loading code. Made some DTOs use init rather than set to keep it clean. 2021-03-12 13:20:08 -06:00
Joseph Milazzo
33515ad865 First pass for supporting lazy loading and not using base64 strings for loading images in UI. 2021-03-12 12:22:04 -06:00