68 Commits

Author SHA1 Message Date
Joseph Milazzo
0eac193248
New Scan Loop (#1447)
* Staging the code for the new scan loop.

* Implemented a basic idea of changes on drives triggering scan loop. Issues: 1. Scan by folder does not work, 2. Queuing system is very hacky and needs a separate thread, 3. Performance degregation could be very real.

* Started writing unit test for new loop code

* Implemented a basic method to scan a folder path with ignore support (not implemented, code in place)

* Added some code to the parser to build out the idea of processing series in batches based on some top level folder.

* Scan Series now uses the new code (folder based parsing) and now handles the LocalizedSeries issue.

* Got library scan working with the new folder-based scan loop. Updated code to set FolderPath (for improved scan times and partial scan support).

* Wrote some notes on update library scan loop.

* Removed migration for merge

* Reapplied the SeriesFolder migration after merge

* Refactored a check that used multiple db calls into one.

* Made lots of progress on ignore support, but some confusion on underlying library. Ticket created. On hold till then.

* Updated Scan Library and Scan Series to exit early if no changes are on the underlying folders that need to be scanned.

* Implemented the ability to have .kavitaignore files within your directories and Kavita will parse them and ignore files and directories based on rules within them.

* Fixed an issue where ignore files nested wouldn't stack with higher level ignores

* Wrote out some basic code that showcases how we can scan series or library based on file events on the underlying system. Very buggy, needs lots of edge case testing and logging and dupplication checking.

* Things are working kinda. I'm getting lost in my own code and complexity. I'm not sure it's worth it.

* Refactored ScanFiles out to Directory Service.

* Refactored more code out to keep the code clean.

* More unit tests

* Refactored the signature of ParsedSeries to use IList. Started writing unit tests and reworked the UpdateLibrary to work how it used to with new scan loop code (note: using async update library/series does not work).

* Fixed the bug where processSeriesInfos was being invoked twice per series and made the code work very similar to old code (except loose leaf files dont work) but with folder based scanning.

* Prep for unit tests (updating broken ones with new implementations)

* Just some notes. Not sure I want to finish this work.

* Refactored the LibraryWatcher with some comments and state variables.

* Undid the migrations in case I don't move forward with this branch

* Started to clean the code and prepare for finishing this work.

* Fixed a bad merge

* Updated signatures to cleanup the code and commit to the new strategy for scanning.

* Swapped out the code with async processing of series on a small library

* The new scan loop is working in both Sync and Async methods. The code is slow and not optimized. This represents a good point to start polling and applying optimizations.

* Refactored UpdateSeries out of Scanner and into a dedicated file.

* Refactored how ProcessTasks are awaited to allow more async

* Fixed an issue where side nav item wouldn't show correct highlight and migrated to OnPush

* Moved where we start to stopwatch to encapsulate the full scan

* Cleaned up SignalR events to report correctly (still needs a redesign)

* Remove the "remove" code until I figure it out

* Put in extremely expensive series deletion code for library scan.

* Have Genre and Tag update the DB immediately to avoid dup issues

* Taking a break

* Moving to a lock with People was successful. Need to apply to others.

* Refactored code for series level and tag and genre with new locking strategy.

* New scan loop works. Next up optimization

* Swapped out the Kavita log with svg for faster load

* Refactored metadata updates to occur when the series are being updated.

* Code cleanup

* Added a new type of generic message (Info) to inform the user.

* Code cleanup

* Implemented an optimization which prevents any I/O (other than an attribute lookup) for Library/Series Scan. This can bring a recently updated library on network storage (650 series) to fully process in 2 seconds.

Fixed a bug where File Analysis was running everytime for each non-epub file.

* Fixed ARM x64 builds not being able to view PDF cover images due to a bad update in DocNet.

* Some code cleanup

* Added experimental signalr update code to have a more natural refresh of library-detail page

* Hooked in ability to send new series events to UI

* Moved all scan (file scan only) tasks into Scan Queue. Made it so scheduled ScanLibraries will now check if any existing task is being run and reschedule for 3 hours, and 10 mins for scan series.

* Implemented the info event in the events widget and added a clear all button to dismiss all infos and errors.  Added --event-widget-info-bg-color

* Remove --drawer-background-color since it's not used

* When new series added, inject directly into the view.

* Some debug code cleanup

* Fixed up the unit tests

* Ensure all config directories exist on startup

* Disabled Library Watching (that will go in next build)

* Ensure update for series is admin only

* Lots of code changes, scan series kinda works, specials are splitting, optimizations are failing. Demotivated on this work again.

* Removed SeriesFolder migration

* Added the SeriesFolder migration

* Added a new pipe for dates so we can provide some nicer defaults. Added folder path to the series detail.

* The scan optimizations now work for NTFS systems.

* Removed a TODO

* Migrated all the times to use DateTime.Now and not Utc.

* Refactored some repo calls to use the includes flag pattern

* Implemented a check for the library scan optimization check to validate if the library was updated (type change, library rename, folder change, or series deleted) and let the optimization be bypassed.

* Added another optimization which will use just folder attribute of last write time if the drive is not NTFS.

* Fixed a unit test

* Some code cleanup
2022-08-19 05:42:38 -07:00
Joseph Milazzo
e9c5cf3c92
Pre-Shakeout (#1429)
* Fixed zindex on some buttons due to a bug being fixed by bootstrap. Tweaked some styles from new bootstrap design.

* Fixed a bug where scan series, when generating cover images wouldn't emit cover updates to the UI.

* Fixed an issue where opening directory picker to change bookmark directory then clicking Share without chaning directory, would show the field as empty.

* Updating workflow

* Reverting version

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2022-08-13 17:28:04 -07:00
Joseph Milazzo
b6a38bbd86
Lots of Bugfixes (#1426)
* Fixed bookmarks not being able to load due to missing [AllowAnonymous]

* Downgraded Docnet to 2.4.0-alpha2 which is the version we added our patches to. This might fix reports of broken PDF reading on ARM

* Updated all but one api in collections to admin only policy

* Ensure all config folders are created or exist on first load

* Ensure plugins can authenticate

* Updated some headers we use on Kavita to tighten security.

* Tightened up cover upload flow to restrict more APIs to only the admin

* Enhanced the reset password flow to ensure that the user passes their existing password in (if already authenticated). Admins can still change other users without having existing password.

* Removed an additional copy during build and copied over the prod appsettings and not Development.

* Fixed up the caching mechanism for cover resets and migrated to profiles. Left an etag filter for reference.

* Fixed up manual jump key calculation to include period in #

* Added jumpbar to reading lists page

* Fixed a double scrollbar on library detail page

* Fixed weird scroll issues with want to read

* Fixed a bug where remove from want to read list wasn't hooked up on series card

* Cleaned up Clear bookmarks to use a dedicated api for bulk clearing. Converted Bookmark page to OnPush.

* Fixed jump bar being offset when clicking a jump key

* Ensure we don't overflow on add to reading list

* Fixed a bad name format on reading list items
2022-08-11 18:16:31 -07:00
Joseph Milazzo
5812588fe5
misc stuff to avoid scan loop (#1389)
* Implemented a workaround for nginx users with BlockCommonExploits enabled, which would interfere with book image escaping done by Kavita when images had ../ in their path.

* Added back to top support on all pages but those that untilize virtual scrolling without a parent scroll.

* Hide jumpbar on pages where there is no scroll

* Refactored jumbar code into a dedicated service

* Stash some jumpkey resume code as I can't get it working with the virtual scroller.

* Don't allow non-admins to see File locations on card detail drawer.

* Some cleanup on GetServerInfo

* When an error occurs in register, delete the user on exception.

* Fixed a NPE in Stat collection for brand new users

* When we catch an exception on registering a new user, delete the user as rolling back doesn't do anything.

* Don't close typeahead when we are selecting options from it

* Added shortcut key H to open shortcut modal on manga reader

* When processing progress updates on cards, for volumes, properly find the chapter to update pages read.

* Hide cover image on reading list if it's not set and fixed a missing closing div tag

* Hide collection poster when nothing is set on collection detail

* Small fix around updating state

* Sped up the bookmark image call by removing one DB call

* Fixed broken test from change in bookmark code

* Fixed an oversight where if there is no tag in ComicInfo after a chapter was updated with People or Genres, then the People/Genres would never be removed.

* Added test with TagHelper

* Fixed a bug where 2 clear buttons would show on search bar due to browser injecting their own. Search bar wont show clear button until text is typed.

* Fixed a bug where InstallID wasn't being selected correctly in converter
2022-07-27 08:16:45 -07:00
Joseph Milazzo
141d10e6da
Misc Bugfixes (#1373)
* Fixed an issue where signalr cover update events would fire before the covers were updated in db and hence UI would show as if no cover for quite some time.

* Refactored MetadataService to GenerateCovers, as that is what this service does.

* Fixed a bug where list item for books that have 0.X series index wouldn't render on series detail. Added Name updates on volume on scan

* Removed some debug code
2022-07-13 12:19:00 -07:00
Joseph Milazzo
4e49aa47ce
Change Detection: On Push aka UI Smoothness (#1369)
* Updated Series Info Cards to use OnPush and hooked in progress events when we do a mark as read/unread on entities. These events update progress bars but also will now trigger a re-calculation on Read Time Left.

* Removed Library Card Component

* Refactored manga reader title and subtitle calculation to the backend.

* Coverted card actionables to onPush

* Series Card on push cleanup

* Updated edit collection tags for on push

* Update cover image chooser for on push

* Cleaned up carsouel reel

* Updated cover image to allow for uploading gif and webp files

* Bulk add to collection on push

* Updated bulk operation to use on push. Updated bulk operation to have mark as unread and read buttons explicitly. Updated so add to collection is visible and delete.

Fixed a bug where manage library component wasn't invoking the trackBy function

* Updating entity title for on push

* Removed file info component

* Updated Mange Library for on push

* Entity info cards on push

* List item on push

* Updated icon and title for on push and fixed some missing change detection on series detail

* Restricted the typeahead interface to simplify the design

* Edit Series Relation now shows a value in the dropdown for Parent relationships and disables the field.

* Updated edit series relation to focus on new typeahead when adding a new relationship

* Added some documentation and when Scanning a library, don't allow the user to enqueue the same job multiple times.

* Applied the No-enqueue if already enqueued logic to other tasks

* Library detail on push

* Updated events widget to onpush

* Card detail drawer on push. Card detail cover chooser now will show all chapter's covers for selection in cover chooser.

* Chapter metadata detail on push

* Removed Card Detail modal

* All collections on push

* Removed some comments

* Updated bulk selection to use an observable rather than function calls so new on push strategy works

* collection detail now uses on push and scroller is placed on correct element

* Updated library recommended to on push. Ensure that when mark as read occurs, the appropriate streams are refreshed.

* Updated library detail to on push

* Update metadata fiter to onpush. Bugs found and reported to Project

* person badge on push

* Read more on push

* Updated tag badge to on push

* User login on push

* When initing side nav, don't call an authenticated api until we are sure a user is logged in

* Updated splash container to on push

* Dashboard on push

* Side nav slight refactor around some api calls

* Cleaned up series card on push to use same cdRef naming convention

* Updated Static Files to use caching

* Added width and height to logo image

* shortcuts modal on push

* reading lists on push

* Reading list detail on push

* draggable ordered list on push

* Refactored reading-list-detail to use a new item which drastically reduces renders on operations

* series format on push

* circular loader on push

* Badge Expander on push

* update notification modal on push

* drawer on push

* Edit Series Modal on push

* reset password on push

* review series modal on push

* series metadata detail on push

* theme manager on push

* confirm reset password on push

* register on push

* confirm migration email on push

* confirm email on push

* add email to account migration on push

* user preferences on push. Made global settings default open

* edit series relation on push

* Fixed an edge case bug for next chapter where if the current volume had a single chapter of 1 and the next volume had a chapter number of 0, it would say there are no more chapters.

* Updated infinite scroller with on push support

* Moved some animations over to typeahead, not integrated yet.

* Manga reader is now on push

* Reader settings on push

* refactored how we close the book

* Updated table of contents for on push

* Updated book reader for on push. Fixed a bug where table of contents wasn't showing current page anchor due to a scroll calulation bug

* Small code tweak

* Icon and title on push

* nav header on push

* grouped typeahead on push

* typeahead on push and added a new trackby identity function to allow even faster rendering of big lists

* pdf reader on push

* code cleanup
2022-07-11 08:57:07 -07:00
Joseph Milazzo
f54eb5865b
All Around Polish (#1328)
* Added --card-list-item-bg-color for the card list items

* Updated the card list item progress to match how cards render

* Implemented the ability to configure how many backups are retained.

* Fixed a bug where odd jump keys could cause a bad index error for jump bar

* Commented out more code for the pagination route if we go with that.

* Reverted a move of DisableConcurrentExecution to interface, as it seems to not work there.

* Updated manga format utility code to pipes

* Fixed bulk selection on series detail page

* Fixed bulk selection on all other pages

* Changed card item to OnPush

* Updated image component to OnPush

* Updated Series Card to OnPush

* Updated Series Detail to OnPush

* Lots of changes here. Integrated parentscroll support on card detail layout. Added jump bar (custom js implementation) on collection, reading list and all series pages. Updated UserParams to default to no pagination. Lots of cleanup all around

* Updated some notes on a module use

* Some code cleanup

* Fixed up a broken test due to the mapper not being configured in the test.

* Applied TabID pattern to edit collection tags

* Applied css from series detail to collection detail page to remove double scrollbar

* Implemented the ability to sort by Time To Read.

* Throw an error to the UI when we extract an archive and it contains invalid characters in the filename for the Server OS.

* Tweaked how the page scrolls for jumpbar on collection detail. We will have to polish another release

* Cleaned up the styling on directory picker

* Put some code in but it doesn't work for scroll to top on virtual scrolling. I'll do it later.

* Fixed a container bug
2022-06-22 10:25:52 -07:00
Joseph Milazzo
bbc48a5f5b
Infinite Scroll + List View + Cover Upload Redesign (#1319)
* Started with the redesign of the cover image chooser redesign to be less click intensive for volume/chapter images.

Made some headings bold in card detail drawer.

* Tweaked the styles

* Moved where the info cards show

* Added an ability to open a page settings drawer

* Cleaned up some old code that isn't needed anymore.

* Started implementing a list view. Refactored some title code to a dedicated component

* List view implemented but way too many API calls. Either need caching or adjusting the SeriesDetail api.

* Fixed a bug where if the progress bar didn't render on a card item while a download was in progress, the download indicator would be removed.

* Large refactor to move a lot of the needed fields to the chapter and volume dtos for series detail. All fields are noted when only used in series detail.

* Implemented cards for other tabs (except related)

* Fixed the unit test which needed a mocked reader service call.

* More cleanup around age rating and removing old code from the refactor. Commented out sorting till i feel motivated to work on that.

* Some cleanup and restored cards as initial layout. Time to test this out and see if there is value add.

* Added ability for Chapters tab to show the volume chapters belong to (if applicable)

* Adding style fixes

* Cover image updates, don't allow the first image (which is what is currently set) to respond to cover changes.

Hide the ID field on list item for series detail.

* Refactored the title for list item to be injectable

* Cleaned up the selection code to make it less finicky on mobile when tap scrolling.

* Refactored chapter tab to show volume as well on list view.

* Ensure word count shows for Volumes

* Started adding virtual scrolling, pushing up so Robbie can mess around

* Started adding virtual scrolling, pushing up so Robbie can mess around

* Fixed a bug where all chapters would come under specials

* Show title data as accent if set.

* Style fixes for virtual scroller

* Restyling scroll

* Implemented a way to show storyline with virtual scrolling

* Show Word Count for chapters and cleaned up some logics.

* I might have card layout working with virtual scroll code.

* Some cleanup to hide more system like properties from info bar on series detail page. Fixed some missing time estimate info on storyline chapters.

* Fixed a regression on series service when I integrated VolumeTitle.

* Refactored read time to the backend. Added WordCount to the volume itself so we don't need to calculate on frontend. When asking to analyze files from a series, force the calculation.

* Fixed SeriesDetail api code

* Fixed up the code in the drawer to better update list/card mode

* Basic infinite scroll implemented, however due to how we are updating the list to render, we are re-rending cards that haven't been touched.

* Updated how we render and layout data for infinite scroll on library detail. It's almost there.

* Started laying foundation for loading pages backwards.

Removed lazy loading of images since we are now using virtual paging.

* Hooked in some basic code to allow user to load a prev page with infinite scroll.

* Fixed up series detail api and undid the non-lazy loaded images.

Changed the router to help with this infinite loading on Firefox issue.

* Fixed up some naming issues with Series Detail and added a new test.

* This is an infinite scroll without pagination implementation. It is not fully done, but off to a good start. Virtual scroller with jump bar is working pretty well, def needs more polishing and tweaking. There are hacks in this implementation that need to be revisited.

* Refactored code so that we don't use any pagination and load all results by default.

* Misc code cleanup from build warnings.

* Cleaned up some logic for how to display titles in list view.

* More title cleanup for specials

* Hooked up page layout to user preferences and renamed an existing user pref name to match the dto.

* Swapped out everything but storyline with virtual-scroller over CDK

* Removed CDK from series detail.

* Default value for migration on page layout

* Updating card layout for library detail page

* fixing height for mobile

* Moved scrollbar

* Tweaked some styling for layouts when there is no data

* Refactored the series cards into their own component to make it re-usable.

* More tweaks on series info cards layout and enhanced a few pages with trackby functions.

* Removed some dead code

* Added download on series detail to actionables to fit in with new scroll strategy.

* Fixed language not being updated and sent to the backend for series update.

* Fixed a bad migration (if you ran any prior migration in this branch, you need to undo before you use this commit)

* Adding sticky tabs

* fixed mobile gap on sticky tab

* Enhanced the card title for books to show number up front.

* Adjusted the gutters on admin dashboard

* Removed debug code

* Removing duplicate book title

* Cleaned up old references to cdk scroller

* Implemented a basic jump bar scaling algorithm. Not perfect, but works pretty well.

* Code smells

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2022-06-13 14:37:49 -07:00
Joseph Milazzo
8e69b6cfc0
Time Estimation Cleanup (#1301)
* Moved the calculation for time to read to the backend. Tweaked some logic around showing est time to complete.

* Added debug logging to help pinpoint a duplicate issue in Kavita.

* More combination logic is error checked in a special way for Robbie to reproduce an issue.

* Migrated chapter detail card to use backend for time calculation. Ensure we take all chapters into account for volume time calcs

* Tweaked messaging for some critical logs to include file

* Ensure pages count uses comma separated number

* Moved Hangfire annotations to interface level. Adjusted word count service to always recalculate when user requests via analyze series files.
2022-05-29 09:50:59 -05:00
Joseph Milazzo
63475722ea
Drawers, Estimated Reading Time, Korean Parsing Support (#1297)
* Started building out idea around detail drawer. Need code from word count to continue

* Fixed the logic for caluclating time to read on comics

* Adding styles

* more styling fixes

* Cleaned up the styles a bit more so it's at least functional. Not sure on the feature, might abandon.

* Pulled Robbie's changes in and partially migrated them to the drawer.

* Add offset overrides for offcanvas so it takes our header into account

* Implemented a basic time left to finish the series (or at least what's in Kavita). Rough around the edges.

* Cleaned up the drawer code.

* Added Quick Catch ups to recommended page. Updated the timeout for scan tasks to ensure we don't run 2 at the same time.

* Quick catchups implemented

* Added preliminary support for Korean filename parsing. Reduced an array alloc that is called many thousands of times per scan.

* Fixing drawer overflow

* Fixed a calculation bug with average reading time.

* Small spacing changes to drawer

* Don't show estimated reading time if the user hasn't read anything

* Bump eventsource from 1.1.1 to 2.0.2 in /UI/Web

Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.1.1 to 2.0.2.
- [Release notes](https://github.com/EventSource/eventsource/releases)
- [Changelog](https://github.com/EventSource/eventsource/blob/master/HISTORY.md)
- [Commits](https://github.com/EventSource/eventsource/compare/v1.1.1...v2.0.2)

---
updated-dependencies:
- dependency-name: eventsource
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Added image to series detail drawer

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-27 07:08:54 -07:00
Joseph Milazzo
c1490d6e86
Word Count (#1286)
* Adding some code for Robbie

* See more on series detail metadata area is now at the bottom on the section

* Cleaned up subtitle headings to use a single class for offset with actionables

* Added some markup for the new design, waiting for Robbie to finish it off

* styling age-rating badge

* Started hooking up basic analyze file service and hooks in the UI. Basic code to implement the count is implemented and in benchmarks.

* Hooked up analyze ui to backend

* Refactored Series Detail metadata area to use a new icon/title design

* Cleaned up the new design

* Pushing for robbie to do css

* Massive performance improvement to scan series where we only need to scan folders reported that have series in them, rather than the whole library.

* Removed theme page as we no longer need it. Added WordCount to DTOs so the UI can show them. Added new pipe to format numbers in compact mode.

* Hooked up actual reading time based on user's words per hour

* Refactor some magic numbers to consts

* Hooked in progress reporting for series word count

* Hooked up analyze files

* Re-implemented time to read on comics

* Removed the word Last Read

* Show proper language name instead of iso tag on series detail page. Added some error handling on word count code.

* Reworked error handling

* Fixed some security vulnerabilities in npm.

* Handle a case where there are no text nodes and instead of returning an empty list, htmlagilitypack returns null.

* Tweaked the styles a bit on the icon-and-title

* Code cleanup

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2022-05-25 14:53:39 -07:00
Joseph Milazzo
2723a6cd10
Book Reader Redesign with e-ink focus (#1246)
* Refactored the drawer into offcanvas component. Had to write some hacks to emulate how bootstrap's javascript implementation works as ngBootstrap doesn't have a component yet.

* Cleaned up some of the code

* Rewrote drawer to align it with the new design

* First pass, refactored table of content into it's own component

* Refactored all of the settings logic into a separate component. Everything is broken.

* More settings on on reactive form

* More code cleanup on settings

* Misc fixes around the drawer code. Fixed a bug where range sliders were inheriting background color of normal text inputs

* Fixed dark mode with book reader.

We now clear the theme from the main app so book reader is self-contained. Styles for dark mode are injected into the reading-section. Styles that were previously in scss are now only for the actual menu system.

* Cleaned up drawer styling on header

* Removed an ngIf statement for click to paginate

* Tweaked the accent style to have smaller font size and adjusted style on light mode. Cleaned up some clearTimeout code in a further effort to streamline codebase.

* Refactored Dark mode into a basic theme. Currently styles are hardcoded.

* Patched book theme in from themes branch

* Patched in the backend for Book Theme (not tested yet)

* Fixed a bug in seeding code for book themes. Started integration of themes into the reader settings

* Everything except managing themes is working. Themes are a bit shakey, having second thoughts if we should have them or not.

* Reverted the ability to do custom user book themes. Code is stable with system themes.

* Stablize the Styles (#1128)

* Fixed a bug where adding multiple series to reading list would throw an error on UI, but it was successful.

* When a series has a reading list, we now show the connection on Series detail.

* Removed all baseurl code from UI and not-connected component since we no longer use it.

* Fixed tag badges not showing a border. Added last read time to the series detail page

* Fixed up error interceptor to remove no-connection code

* Changed implementation for series detail. Book libraries will never send chapters back. Volume 0 volumes will not be sent in volumes ever. Fixed up more renaming logic on books to send more accurate representations to the UI.

* Cleaned up the selected tab and tab display logic

* Fixed a bad where statement in reading lists for series

* Fixed up tab logic again

* Fixed a small margin on search backdrop

* Made badge expander button smaller to align with badges

* Fixed a few UIs due to .form-group and .form-row being removed

* Updated Theme component page to help with style testing

* Added more components to theme tester

* Cleaned up some styling

* Fixed opacity on search item hover

* Bump versions by dotnet-bump-version.

* Tweaked the accordion styles for light mode

* Set dark book theme as default. Refactored resetSettings to be much cleaner

* Started the refactor to allow book themes to affect global css variables

* Fixed some issues with my css variable declarations

* Fixed a close model state update

* Lots of work, but dark mode on the book reader is basically done. We have to code the themes much like the site themes

* Some black theme enhancements

* Started working on column layout in book reader.

* Cleaned up the CSS on Reader Settings

* Hooked up reading direction

* Got column and double column layout working

* Implemented some basic virtual paging and hooked in book color theme and layout mode into user preferences.

* Migration wrote, can edit page layout and color theme on book reader. Removed book dark mode since no longer needed.

Fixed a bug on login/register forms where when input is focused, text is white and not black.

* When loading book reader, apply column layout.

* Lots of work around 2 column layout, working on images not splitting. Still not working, committing so i can merge develop in and validate code with new manga reader.

* Fixed images being split into 2 BUT regression on each page boundary, total reading height is smaller and smaller

* Fixed some rendering bugs where toggling column layouts would shrink images on screen constantly.

Fixed a bug where bottom bar wouldn't render on column layout in some conditions (this might need to be reworked)

* Started progress on progress work

* Updated .NET to 6.0.4

* Fixed a bug where DataContextModelSnapshot was being removed on build thus new migrations were broken.

* Tweaked the code around progress saving so that we don't loose track of last scroll element on page load

* Trying to restore progress, but stuck

* Extra merge stuff

* Fixed a bug where volumes that are a range fail to generate series detail

* No gutters on whole app. Book reader backend now applies the image class automatically at the backend.

* Added wiki documentation into invite user flow and register admin user to help users understand email isn't required and they can host their own service.

* Removed bottom padding

* Refactored the document height to be set and removed on nav service, so the book reader and manga reader aren't broken.

* Fixed the height of the action bar to simplify logic and keep the code cleaner. Refactored book service image scoping to be much more streamlined and efficient

* Fixed the height of action bar to 62px and adjusted code to use the hardcoded px. (code commented)

* Removed commented out code from fixed action bar height

* Progress restoration seems to be working

* Code cleanup

* Ensure the bottom action bar is at the bottom of the viewport on small pages

* Fixed book fonts not setting properly and added OpenDyslexic font.

* Fixed up some font issues

* Updated drawer so all sections are open by default

* Switched some LINQ to use MinBy

* When navigating between pages and column layout, adjust the shift for the user.

* Removed some debug code

* Blacklist .qpkg folders and don't scan Recently-Snapshot or recycle folders.

* Renamed the scale width to be scoped to kavita to avoid conflicts.

* Refactored ngx-sliders out to use normal range instead. Changed up the preferences to separate image and book settinngs into own accordion.

* updated user preferences for new migration options (not committed yet)

* Removed some debug code

* Remove console.logs

* Migration committed, let's release this to users.

* A lot of crazy code just to ensure that when you close drawer the toggle reflectst that state.
2022-05-08 17:52:15 -07:00
Joseph Milazzo
553f9b0d98
Fixes, Tweaks, and Series Filtering (#1217)
* From previous fix, added the other locking conditions on the update series metadata.

* Fixed a bug where custom series, collection tag, and reading list covers weren't being removed on cleanup.

* Ensure reading list detail has a margin to align to the standard

* Refactored some event stuff to use dedicated consts. Introduced a new event when users read something, which can update progress bars on cards.

* Added recomended and library tags to the library detail page. This will eventually offer more custom analytics

* Cleanup some code onc arousel

* Adjusted scale to height/width css to better fit

* Small css tweaks to better center images in the manga reader in both axis. This takes care of double page rendering as well.

* When a special has a Title set in the metadata, on series detail page, show that on the card rather than filename.

* Fixed a bug where when paging in manga reader, the scroll to top wasn't working due to changing where scrolling is done

* More css goodness for rendering images in manga reader

* Fixed a bug where clearing a typeahead externally wouldn't clear the x button

* Fixed a bug where filering then using keyboard would select wrong option

* Added a new sorting field for Last Chapter Added (new field) to get a similar on deck feel.

* Tweaked recently updated to hit the NFR of 500ms (300ms fresh start) and still give a much better experience.

* Refactored On deck to now go to all series and also sort by last updated. Recently Added Series now loads all series with sort by created.

* Some tweaks on css for cover image chooser

* Fixed a bug in pagination control where multiple pagination events could trigger on load and thus multiple requests for data on parent controller.

* Updated edit series modal to show when the last chapter was added and when user last read it.

* Implemented a highlight on the fitler button when a filter is active.

* Refactored metadata filter screens to perserve the filters in the url and thus when navigating back and forth, it will retain. users should click side nav to reset the state.

* Hide middle section on companion bar on phones

* Cleaned up some prefilters and console.logs

* Don't open drawer by default when a filter is active
2022-04-14 14:55:06 -07:00
Joseph Milazzo
eddbb7ab18
Event Widget Update (#1098)
* Took care of some notes in the code

* Fixed an issue where Extra might get flagged as special too early, if in a word like Extraordinary

* Moved Tag cleanup code into Scanner service. Added a SplitQuery to another heavy API. Refactored Scan loop to remove parallelism and use async instead.

* Lots of rework on the codebase to support detailed messages and easier management of message sending. Need to take a break on this work.

* Progress is being made, but slowly. Code is broken in this commit.

* Progress is being made, but slowly. Code is broken in this commit.

* Fixed merge issue

* Fixed unit tests

* CoverUpdate is now hooked into new ProgressEvent structure

* Refactored code to remove custom observables and have everything use standard messages$

* Refactored a ton of instances to NotificationProgressEvent style and tons of the UI to respect that too. UI is still a bit buggy, but wholistically the work is done.

* Working much better. Sometimes events come in too fast. Currently cover update progress doesn't display on UI

* Fixed unit tests

* Removed SignalREvent to minimize internal event types. Updated the UI to use progress bars. Finished SiteThemeService.

* Merged metadata refresh progress events and changed library scan events to merge cleaner in the UI

* Changed RefreshMetadataProgress to CoverUpdateProgress to reflect the event better.

* Theme Cleanup (#1089)

* Fixed e-ink theme not properly applying correctly

* Fixed some seed changes. Changed card checkboxes to use our themed ones

* Fixed recently added carousel not going to recently-added page

* Fixed an issue where no results found would show when searching for a library name

* Cleaned up list a bit, typeahead dropdown still needs work

* Added a TODO to streamline series-card component

* Removed ng-lazyload-image module since we don't use it. We use lazysizes

* Darken card on hover

* Fixing accordion focus style

* ux pass updates

- Fixed typeahead width
- Fixed changelog download buttons
- Fixed a select
- Fixed various input box-shadows
- Fixed all anchors to only have underline on hover
- Added navtab hover and active effects

* more ux pass

- Fixed spacing on theme cards
- Fixed some light theme issues
- Exposed text-muted-color for theme card subtitle color

* UX pass fixes

- Changed back to bright green for primary on dark theme
- Changed fa icon to black on e-ink

* Merged changelog component

* Fixed anchor buttons text decoration

* Changed nav tabs to have a background color instead of open active state

* When user is not authenticated, make sure we set default theme (dark)

* Cleanup on carousel

* Updated Users tab to use small buttons with icons to align with Library tab

* Cleaned up brand to not underline, removed default link underline on hover in dropdown and pill tabs

* Fixed collection detail posters not rendering

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>

* Bump versions by dotnet-bump-version.

* Tweaked some of the emitting code

* Some css, but pretty bad. Robbie please save me

* Removed a todo

* styling update

* Only send filename on FileScanProgress

* Some console.log spam cleanup

* Various updates

* Show events widget activity based on activeEvents

* progress bar color updates

* Code cleanup

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2022-02-18 18:57:37 -08:00
Joseph Milazzo
2b0d47d15e
Shakeout Testing Part 1 (#1052)
* Have language from epubs populate metadata

* series detail needs to reload the underlying volumes when scan event comes in, not just metadata.

* Added Id to chapter detail modal (for debugging)

* Implement IDisposable on applicable Unit Tests

* Removed unused using statements

* Fixed a bug where images would flash like crazy during a scan because the code to refresh the underlying image wasn't checking the entity type or Id.

* When filtering rating, only apply the filter to your account.

* Removed Disposable on tests
2022-02-09 16:59:14 -08:00
Joseph Milazzo
b571633eab
Webtoon fixes + Random release stuff (#1048)
* Refactored the way cover images are updated from SignalR to use an explicit event that is sent at a granular level for a given type of entity.

Fixed a bad event listener for RefreshMetadata (now removed) to update metadata on Series Detail. Now uses ScanService, which indicates a series has completed a scan.

* Lots of attempts at making webtoon stable. Kinda working kinda not.

* Added a new boolean to hide images until the first prefetch loads the images, to prevent jankiness

* On Search, remove : from query

* Added HasBookmark and NumberOfLibraries to stat service

* Cleaned up some dead code

* Fixed a bug where page number wasn't reset between chapter loads with infinite scroller

* Added recently added series back into the dashboard.

* Cleaned up some code in search bar
2022-02-08 07:30:54 -08:00
Joseph Milazzo
67ba5e302f
Refactored the way cover images are updated from SignalR to use an explicit event that is sent at a granular level for a given type of entity. (#1046)
Fixed a bad event listener for RefreshMetadata (now removed) to update metadata on Series Detail. Now uses ScanService, which indicates a series has completed a scan.
2022-02-07 17:44:06 -08:00
Joseph Milazzo
efb527035d
Account Email Support (#1000)
* Moved the Server Settings out into a button on nav header

* Refactored Mange Users page to the new design (skeleton). Implemented skeleton code for Invite User.

* Hashed out more of the code, but need to move all the email code to a Kavita controlled API server due to password credentials.

* Cleaned up some warnings

* When no user exists for an api key in Plugin controller, throw 401.

* Hooked in the ability to check if the Kavita instance can be accessed externally so we can determine if the user can invite or not.

* Hooked up some logic if the user's server isn't accessible, then default to old flow

* Basic flow is working for confirm email. Needs validation, error handling, etc.

* Refactored Password validation to account service

* Cleaned up the code in confirm-email to work much better.

* Refactored the login page to have a container functionality, so we can reuse the styles on multiple pages (registration pages). Hooked up the code for confirm email.

* Messy code, but making progress. Refactored Register to be used only for first time user registration. Added a new register component to handle first time flow only.

* Invite works much better, still needs a bit of work for non-accessible server setup. Started work on underlying manage users page to meet new design.

* Changed (you) to a star to indicate who you're logged in as.

* Inviting a user is now working and tested fully.

* Removed the register member component as we now have invite and confirm components.

* Editing a user is now working. Username change and Role/Library access from within one screen. Email changing is on hold.

* Cleaned up code for edit user and disabled email field for now.

* Cleaned up the code to indicate changing a user's email is not possible.

* Implemented a migration for existing accounts so they can validate their emails and still login.

* Change url for email server

* Implemented the ability to resend an email confirmation code (or regenerate for non accessible servers). Fixed an overflow on the confirm dialog.

* Took care of some code cleanup

* Removed 3 db calls from cover refresh and some misc cleanup

* Fixed a broken test
2022-01-30 14:45:57 -08:00
Joseph Milazzo
fb71d54fe6
Misc Fixes and Changes (#927)
* Cleaned up a ton of warnings/suggestions from the IDE.

* Fixed a bug when clearing the filters some presets could be undone.

* Renamed a class in the OPDS spec

* Simplified logic for when Fit To Screen rendering logic occurs. It now works always rather than only on cover images.

* Give some additional info to the user on what the differences between Library Types are

* Don't scan .qpkg folders (QNAP devices)

* Refactored some code to enable ability to test CoverImage Test. This is a broken test, test.zip is waiting on an issue in NetVips.

* Fixed an issue where Extra might get flagged as special too early, if in a word like Extraordinary

* Cleaned up the regex for the extra issue to be more flexible
2022-01-12 15:00:00 -08:00
Joseph Milazzo
ab8c762830
Metadata Performance Scan (#921)
* Refactored updating chapter metadata from ComicInfo into the Scan loop. This let's us avoid an additional N file reads (expensive) in the metadata service, as we already have to read them in the scan loop.

* Refactored Series level metadata aggregation into the scan loop. This allows for the batching of DB updates to be much smaller, thus faster without much overhead of GC.

* Refactored some of the code for ProcessFile to remove a few redundant if statements
2022-01-09 10:36:24 -08:00
Joseph Milazzo
ad3c8bb2c7
Don't tag a series as completed if count is 0. (#917) 2022-01-08 16:31:02 -08:00
Joseph Milazzo
2fbcf203aa
Implemented Publication Status in SeriesMetadata and the ability to filter it. (#915) 2022-01-08 13:10:03 -08:00
Joseph Milazzo
c5e5aa19d5
Misc Fixes (#914)
* Fixed the book reader off by one issue with loading last page

* Fixed a case where scanner would not delete a series if another series with same name but different format was added in that same scan.

* Added some missing tag generation (chapter language and summary)
2022-01-08 11:36:47 -08:00
Joseph Milazzo
0be0e294aa
Metadata Optimizations (#910)
* Added a tooltip to inform user that format and collection filter selections do not only show for the selected library.

* Refactored a lot of code around when we update chapter cover images. Applied an optimization for when we re-calculate volume/series covers, such that it only occurs when the first chapter's image updates.

* Updated code to ensure only lastmodified gets refreshed in metadata since it always follows a scan

* Optimized how metadata is populated on the series. Instead of re-reading the comicInfos, instead I read the data from the underlying chapter entities. This reduces N additional reads AND enables the ability in the future to show/edit chapter level metadata.

* Spelling mistake

* Fixed a concurency issue by not selecting Genres from DB. Added a test for long paths.

* Fixed a bug in filter where collection tag wasn't populating on load

* Cleaned up the logic for changelog to better compare against the installed verison. For nightly users, show the last stable as installed.

* Removed some demo code

* SplitQuery to allow loading tags much faster for series metadata load.
2022-01-08 06:41:47 -08:00
Joseph Milazzo
2b57449a63
Book Reader Issues (#906)
* Refactored the Font Escaping Regex with new unit tests.

* Fonts are now properly escaped, somehow a regression was introduced.

* Refactored most of the book page loading for the reader into the service.

* Fixed a bug where going into fullscreen in non dark mode will cause the background of the reader to go black. Fixed a rendering issue with margin left/right screwing html up. Fixed an issue where line-height: 100% would break book's css, now we remove the styles if they are non-valuable.

* Changed how I fixed the black mode in fullscreen

* Fixed an issue where anchors wouldn't be colored blue in white mode

* Fixed a bug in the code that checks if a filename is a cover where it would choose "backcover" as a cover, despite it not being a valid case.

* Validate if ReleaseYear is a valid year and if not, set it to 0 to disable it.

* Fixed an issue where some large images could blow out the screen when reading on mobile. Now images will force to be max of width of browser

* Put my hack back in for fullscreen putting background color to black

* Change forwarded headers from All to explicit names

* Fixed an issue where Scheme was not https when it should have been. Now the browser will handle which scheme to request.

* Cleaned up the user preferences to stack multiple controls onto one row

* Fixed fullscreen scroll issue with progress, but now sticky top is missing.

* Corrected the element on which we fullscreen
2022-01-07 06:56:28 -08:00
Joseph Milazzo
bbdfe17247
I can't believe it's more fixes! (#863)
* Send stack trace to the UI on prod mode

* Pdfs will now generate cover images. I missed something a few releases ago.

* Ignore @Recently-Snapshot directories for QNAP.

* Refactored Bitmap code to use ImageSharp so it's truly cross platform.

* Updated pdf extraction to use a multi-threaded approach to greatly speed up pdf image extraction

* Hooked in Characters tag from ComicInfo.xml
2021-12-20 11:50:47 -06:00
Joseph Milazzo
3a8a786397
Fixed a bug in how to determine if the volume or series updates cover image. (#857) 2021-12-17 13:21:53 -08:00
Joseph Milazzo
94bad97511
More Filtering and Support for ComicInfo v2.1 (draft) Tags (#851)
* Added a reoccuring task to cleanup db entries that might be abandoned. On library page, the Library in question will be prepoulated.

* Laid out the foundation for customized sorting. Added all series page to the UI when clicking on Libraries section header on home page so user can apply any filtering they like.

* When filtering, the current library filter will now automatically filter out the options for people and genres.

* Implemented Sorting controls

* Clear now clears sorting and read progress. Sorting is disabled on deck and recently added.

* Fixed an issue where all-series page couldn't click to open series

* Don't let the user unselect the last read progress. Added new comicinfo v2.1 draft tags.

* Hooked in Translator tag into backend and UI.

* Fixed an issue where you could open multiple typeaheads at the same time

* Integrated Translator and Tags ComicInfo extension fields. Started work on a badge expander.

* Reworked a bit more on badge expander. Added the UI code for Age Rating and Tags

* Integrated backend for Tags, Translator, and Age Rating

* Metadata tags now collapse if more than 4 present

* Some code cleanup

* Made the not read badge slightly smaller
2021-12-16 13:41:38 -08:00
Joseph Milazzo
28688ada8e
In-Depth Filtering (#850)
* Laying the foundation for the filter rework

* Filtering by Genre is now possible.

* Cleaned up code and preparing for People filtering

* People filtering is hooked up for the frontend

* Filtering now works. On Deck does not work with filtering currently due to a unique implementation.

* More cleanup

* Implemented the ability to reset the filters

* Added a mobile drawer for filtering

* Added some additional cases for NaturalSortComparer. Filter now uses a drawer on smaller screens.

* Fixed a bug where backup service was not pointing to the correct directory.

* Undid the fix, it's working as expected
2021-12-15 08:23:10 -08:00
Joseph Milazzo
384ebcef5c
Tachiyomi Enhancements (#845)
* Added a new endpoint to get all Series with Progress info.

* Fixed up some potential NPEs during scan

* Commented out filter code, not ready for it.

* Fixed up a parsing case for european comics

* Refactored FilterDto to allow for specifying multiple formats to return.

* Refactored FilterDto to allow for specifying multiple formats to return.

* Refactored the UI to show OPDS as 3rd Party Clients since Tachiyomi now uses OPDS url scheme for authentication.
2021-12-10 13:04:52 -08:00
Joseph Milazzo
3b90ef96b5
Misc Fixes (#839)
* Fixed a case where chapter was being parsed incorrectly when the series title ends in a number.

* Updated Kavita to support Tome/T notation found in French comics

* Added support for identifying European specials and expanded support for cleaning some tags used in European comics. During cleaning, if series starts with - or comma, remove it.

* Fixed an issue where add to collection for a single series wasn't calling the bulk action handler

* Fixed a NPE on AgeRating conversion. Fixed a bug where when looking for cover image, file extensions was throwing off sort code.

* Refactored Natural Sort ordering to better follow how Windows behaves. This is a departure from how the original code executes.

* GetCachedPagePath now uses natural sorting to pick the images for reading in a more correct order.

* Updated parser to handle a case where there was more than one space as a separator
2021-12-08 11:27:54 -08:00
Joseph Milazzo
af24c928d7
Feature/local metadata more tags (#832)
* Stashing code

* removed some debug code on series detail page. Now detail is collapsed by default.

* Added AgeRating

* Fixed a crash when NetVips tries to write a cover file and cover directory is not existing.

* When a card is selected for bulk actions, show an outline in addition to select box

* Added AgeRating into the metadata parsing. Added a hack where ComicInfo uses Number in ComicInfo rather than Volume. This is to test out the effects on users libraries.

* Added AgeRating and ReleaseDate to the metadata implelentation.
2021-12-06 11:59:47 -08:00
Joseph Milazzo
bbe8f800f6
.NET 6 Coding Patterns + Unit Tests (#823)
* Refactored all files to have Interfaces within the same file. Started moving over to file-scoped namespaces.

* Refactored common methods for getting underlying file's cover, pages, and extracting into 1 interface.

* More refactoring around removing dependence on explicit filetype testing for getting information.

* Code is buildable, tests are broken. Huge refactor (not completed) which makes most of DirectoryService testable with a mock filesystem (and thus the services that utilize it).

* Finished porting DirectoryService to use mocked filesystem implementation.

* Added a null check

* Added a null check

* Finished all unit tests for DirectoryService.

* Some misc cleanup on the code

* Fixed up some bugs from refactoring scan loop.

* Implemented CleanupService testing and refactored more of DirectoryService to be non-static.

Fixed a bug where cover file cleanup wasn't properly finding files due to a regex bug.

* Fixed an issue in CleanupBackup() where we weren't properly selecting database files older than 30 days. Finished CleanupService Tests.

* Refactored Flatten and RemoveNonImages to directory service to allow CacheService to be testable.

* Finally have CacheService tested. Rewrote GetCachedPagePath() to be much more straightforward & performant.

* Updated DefaultParserTests.cs to contain all existing tests and follow new test layout format.

* All tests fixed up
2021-12-05 08:58:53 -08:00
Joseph Milazzo
e7619e6b0a
Local Metadata Integration Part 1 (#817)
* Started with some basic plumbing with comic info parsing updating Series/Volume.

* We can now get chapter title from comicInfo.xml

* Hooked in the ability to store people into the chapter metadata.

* Removed no longer used imports, fixed up some foreign key constraints on deleting series with person linked.

* Refactored Summary out of the UI for Series into SeriesMetadata. Updated application to .net 6. There is a bug in metadata code for updating.

* Removed the parallel.ForEach with a normal foreach which lets us use async. For I/O heavy code, shouldn't change much.

* Refactored scan code to only check extensions with comic info, fixed a bug on scan events not using correct method name, removed summary field (still buggy)

* Fixed a bug where on cancelling a metadata request in modal, underlying button would get stuck in a disabled state.

* Changed how metadata selects the first volume to read summary info from. It will now select the first non-special volume rather than Volume 1.

* More debugging and found more bugs to fix

* Redid all the migrations as one single one. Fixed a bug with GetChapterInfo returning null when ChapterMetadata didn't exist for that Chapter.

Fixed an issue with mapper failing on GetChapterMetadata. Started work on adding people and a design for people.

* Fixed a bug where checking if file modified now takes into account if file has been processed at least once. Introduced a bug in saving people to series.

* Just made code compilable again

* Fixed up code. Now people for series and chapters add correctly without any db issues.

* Things are working, but I'm not happy with how the management of Person is. I need to take into account that 1 person needs to map to an image and role is arbitrary.

* Started adding UI code to showcase chapter metadata

* Updated workflow to be .NET 6

* WIP of updating card detail to show the information more clearly and without so many if statements

* Removed ChatperMetadata and store on the Chapter itself. Much easier to use and less joins.

* Implemented Genre on SeriesMetadata level

* Genres and People are now removed from Series level if they are no longer on comicInfo

* PeopleHelper is done with unit tests. Everything is working.

* Unit tests in place for Genre Helper

* Starting on CacheHelper

* Finished tests for ShouldUpdateCoverImage. Fixed and added tests in ArchiveService/ScannerService.

* CacheHelper is fully tested

* Some DI cleanup

* Scanner Service now calls GetComicInfo for books. Added ability to update Series Sort name from metadata files (mainly epub as comicinfo doesn't have a field)

* Forgot to move a line of code

* SortName now populates from metadata (epub only, ComicInfo has no tags)

* Cards now show the chapter title name if it's set on hover, else will default back to title.

* Fixed a major issue with how MangaFiles were being updated with LastModified, which messed up our logic for avoiding refreshes.

* Woohoo, more tests and some refactors to be able to test more services wtih mock filesystem. Fixed an issue where SortName was getting set as first chapter, but the Series was in a group.

* Refactored the MangaFile creation code into the DbFactory where we also setup the first LastModified update.

* Has file changed bug is now finally fixed

* Remove dead genres, refactor genre to use title instead of name.

* Refactored out a directory from ShouldUpdateCoverImage() to keep the code clean

* Unit tests for ComicInfo on BookService.

* Refactored series detail into it's own component

* Series-detail now received refresh metadata events to refresh what's on screen

* Removed references to Artist on PersonRole as it has no metadata mapping

* Security audit

* Fixed a benchmark

* Updated JWT Token generator to use new methods in .NET 6

* Updated all the docker and build commands to use net6.0

* Commented out sonar scan since it's not setup for net6.0 yet.
2021-12-02 09:02:34 -08:00
Joseph Milazzo
281352001d
Feature/progress widget (#760)
* Implemented a new widget to show when operations are occuring in the backend (tasks + progress events). Fixed an oversight on progress reporting where I sent 100F instead of 1F.

* Hooked in more progress events for tasks on the backend. Cleaned up code and integrated some RBS into it. CSS needed.

* Show a colored icon when events are active

* Added some styling to the progress widget
2021-11-15 14:50:14 -08:00
Joseph Milazzo
6eb5c34695
Bugfix/webtoon lastpage (#757)
* Some small changes to how maxPage is handled for better page tracking on webtoon reader.

* Fixed a bug in cover image selection (#756)

* Refactored webtoon reader to report page progress exactly as manga reader does. In addition, manga reader moved progress saving to setPageNumber making them the same.
2021-11-15 07:24:04 -08:00
Joseph Milazzo
f6bfabde4c
Backend Bugfixes and Enhanced Selections (#754)
* Updated some signatures to avoid a ToArray() within a loop.

* Use UpdateSeries directly when adding new series, rather than a modified version for new series only.

* Refactored some messages for scanner loop to reduce duplicate code and write messages more clear. Hooked in a RefreshMetadataProgress event (no UI changes).

* Fixed a bug on docker where backup service was using different logic than non-docker, which isn't needed after config change last release.

* Allow user to make more than 1 backup per day

* Implemented a select all checkbox for library access modal
2021-11-14 08:20:12 -08:00
Joseph Milazzo
1d80420155
Change chunk size to be a fixed 50 to validate if it's causing issue with refresh. Added some try catches to see if exceptions are causing issues. (#681) 2021-10-16 11:50:34 -07:00
Joseph Milazzo
355f50fd5d
Logging Cleanup (#668)
* Do not allow non-admins to change their passwords when authentication is disabled

* Clean up the login page so that input field text is black

* cleanup some resizing when typing a password and having a lot of users

* Changed the LastActive for a user to not just be login, but also when they open an already authenticated session.

* Removed some verbose debugging statements and moved some debug to information to be more prevelant to logs for default installs.

* In Progress now sends progress information on the Series

* Add ability to add cards to recently added when new series are added in backend

* Implemented the ability to click the glasses icon to turn off incognito mode from within the reader so you can start tracking progress

* Don't warn the user about authentication when they don't touch that control
2021-10-14 08:14:03 -07:00
Joseph Milazzo
06508fd909
Remove Base Url Support (#652)
* Fixed some issues with base url. All Scheduled jobs are now in user's timezone

* Hide Base Url support from UI and removed some code around it on the backend to prevent it from interfering. Patched back in changes from base-href branch like Timezone on scheduled jobs and enhanced logging.

* Added parser support for "2000 AD 0366 [1984-04-28] (flopbie)" and removed for "01 Spiderman 01".
2021-10-11 06:57:50 -07:00
Joseph Milazzo
2183bf2d59
Misc Bugfixes (#637)
* Use null coalescing for comicinfo

* Removed Sentry. The integrations were not providing any userful information.

* Updated to the latest versions of dependencies
2021-10-04 17:18:42 -07:00
Joseph Milazzo
f17d89ea47
ComicInfo Refactor (#636)
* Implemented methods to parse out the whole ComicInfo file and a mock ComicInfo from epub files.

* Removed unused imports. ScanSeries needs to cleanup deleted chapters and call RefreshMetadata. Ensure after scan we cleanup tags without any series.

* Random cleanup

* Added some comments about data getting stale and not updating.

* Removed old Summary methods in favor of the ComicInfo versions

* Added a missing property

* Fixed unit test
2021-10-04 16:10:48 -07:00
Joseph Milazzo
c5b62d00d0
Hashing (#634)
* Added the ability to see when a scan started at.

* filename based hashing now uses last write time as well to ensure if the underlying file changes it sends a new copy

* Fixed a bug where we would reset dark mode on the book reader to dark mode if our site was on dark mode, despite user setting light mode.

* Added a single feed entry when some sort of collection, reading list, etc doesn't have anything in it.

* Allow + in the normalization to prevent some series that use + to denote the sequel from not getting merged together.
2021-10-04 05:33:28 -07:00
Joseph Milazzo
56cf7be799
Scan Chunking (#604)
* Some performance refactoring around getting Library and avoid a byte[] copy for getting cover images for epubs.

* Initial commit. Rewrote the main series scan loop to use chunks of data at a time. Not fully shaken out.

* Hooked in the ability for the UI to react to series being added or removed from the DB.

* Cleaned up the messaging in the scan loop to be more clear.

* Metadata scan and scan work as expected and populate data to the UI. There is a slow down in speed for overall operation.

Scan series and refresh series metadata does not work fully.

* Fixed a bug where MangaFiles were not having LastModified Updated correctly, meaning they were opening archives every scan.

* Modified the code to be more realistic to the underlying file

* Updated ScanService to properly handle deleted files and not result in a higher-level scan.

* Shuffled around volume related repo apis to the volume repo rather than being in series.

* Rewrote scan series to be much cleaner and more concise on the flow. Fixed an issue in UpdateVolumes such that the debug code to log out removed volumes could throw an exception and actually break updating volumes.

* Refactored the code to set MangaFile last modified timestamp into the MangaFile entity.

* Added Series Name to ScanSeries event

* Added additional checks in ScanSeries to ensure we never go outside the library folder.

Added extra debug messages for when a metadata refresh doesn't actually make changes and for when we regen cover images.

* More logging statements saying where they originate from. Fixed a critical bug which caused only 1 chunk to ever be processed.

* Fixed a concurrency issue with natural sorter which could cause issues in ArchiveService.cs.

* Log cleanups

* Fixed an issue with logging out total time of a scan.

* Only show added toastrs for admins. When kicking off a refresh metadata for series, make sure we regenerate all cover images.

* Code smells on benchmark despite it being ignored
2021-09-30 06:08:05 -07:00
Joseph Milazzo
82b5b599e0
Foundational Cover Image Rework (#584)
* Updating wording on card item when total pages is 0, to be just "Cannot Read" since it could be a non-archive file

* Refactored cover images to be stored on disk. This first commit has the extraction to disk and the metadata service to handle updating when applicable.

* Refactored code to have the actual save to cover image directory done by ImageService.

* Implemented the ability to override cover images.

* Some cleanup on Image service

* Implemented the ability to cleanup old covers nightly

* Added a migration to migrate existing covers to new cover image format (files).

* Refactored all CoverImages to just be the filename, leaving the Join with Cover directory to higher level code.

* Ensure when getting user progress, we pick the first.

* Added cleanup cover images for deleted tags. Don't pull any cover images that are blank.

* After series update, clear out cover image. No change on UI, but just keeps things clear before metadata refresh hits

* Refactored image formats for covers to ImageService.

* Fixed an issue where after refactoring how images were stored, the cleanup service was deleting them after each scan.

* Changed how ShouldUpdateCoverImage works to check if file exists or not even if cover image is locked.

* Fixed unit tests

* Added caching back to cover images.

* Caching on series as well

* Code Cleanup items

* Ensure when checking if a file exists in MetadataService, that we join for cover image directory. After we scan library, do one last filter to delete any series that have 0 pages total.

* Catch exceptions so we don't run cover migration if this is first time run.

* After a scan, only clear out the cache directory and not do a deep clean.

* Implemented the ability to backup custom locked covers only.

* Fixed unit tests

* Trying to figure out why GA crashes when running MetadataServiceTests.cs

* Some debugging on GA tests not running

* Commented out tests that were causing issues in GA.

* Fixed an issue where series cover images wouldn't migrate

* Fixed the updating of links to actually do all series and not just locked
2021-09-21 19:15:29 -05:00
Joseph Milazzo
2725e6042b
WebP Support (#581)
* Added trackby so when series scan event comes through, cards can update too

* Added chapter boundary toasts on book reader

* Handle closing the reader when in a reading list

* Somehow the trackby save didn't happen

* Fixed an issue where after opening a chapter info modal, then trying to open another in specials tab it would fail due to a pass by reference issue with our factory.

* When a series update occurs, if we loose specials tab, but we were on it, reselect volumes/chapters tab

* Fixed an issue where older releases would show as available, even though they were already installed.

* Converted tabs within modals to use vertical orientation (except on mobile)

* Implemented webp support. Only Safari does not support this format natively. MacOS users can use an alternative browser.

* Refactored ScannerService and MetadataService to be fully async
2021-09-15 17:25:18 -07:00
Joseph Milazzo
cf4fd2cb9c
Bugs, Enhancements, and Performance (#580)
* Added parser case for "The Duke of Death and His Black Maid - Ch. 177 - The Ball (3).cbz"

* Removed a file that is created and modified every test run.

* Fixed a bad parser case for "Batman Beyond 02 (of 6) (1999)" which was consuming too many characters

* Removed a lot of "Volume" parsing for Comics that don't make sense. This is prep work for the upcoming Comic Rework release.

* Reworked a lot of parsing cases for comics based on naming conventions observed from releases found online.

* Added a way for external scripts to use a user api key to authenticate

* Fixed an issue if the manga only had one page, the bottom menu would be missing page and chapter controls.

* Fixed a bug where on small phones, nav bar could overflow due to scroll to top

* Tweaked a lot of regex for manga parsing to handle some cases where poorly named files, like "Vol. 03 Ch. 21" would end up parsing as Series "Vol. 03".

* Even more handling of parser cases. Manga parser should be as it was but more robust to handle bad naming.

* Fixed: Don't force metadata refresh on Scan Series, only on refresh metadata

* Implemented the ability to automatically refresh after a series scan based on when server finishes. Remove a duplicate API call from series detail.

* Removed another API call for series metadata that isn't needed.

* Refactored Message creation to a factory, hardcoded strings are centralized, and RefreshSeriesMetadata sends an event and is refactored to be async.

* Fixed a bug when really poorly named files are within a folder that contains the series name, fallback couldn't occur due to it being taken as root folder. Now we detect said condition and will go one level higher, resulting in potentially more I/O, but the series will not be deleted.

* Added the Read in Incognito context item for Chapter cards

* Skip an additional check for series summary for series that aren't EPUB or Archive formats.

* Fixed an issue where cover image generation could occur due to a bad check on LastWriteTime on the underlying file.

* Added some extra comic parser tests

* Added a ScanLibrary event (not hooked up in UI)

* Performance improvement on metadata service. Now when we scan for cover image changes, we emit when a change occurs and only then do we update parent entities (array copy).

* Removed an hr from series detail and ensure we update the cover image for series when scan series finishes.

* Updated the infinite scroller to use a Flags pattern for the debug mode. Updated a few logical conditions for mobile.

* Removed the concurrency check on row progress as if too many calls hit the DB, it will throw, but it doesn't matter.

Fixed a bad logic code which could cause scrolling after hitting the bottom of the chapter.

* Ensure prefetching uses totalPages + 1 since we pass in totalPages as - 1 from manga reader

* Fixed issue where last page of webtoon wouldn't be prefetched due to a < instead of <= on prefetching code

* Implemented ability to send images from archives to the UI without incurring any extra memory pressure.

* Dropdown menus now have a darker background

* Webtoon reader now works on mobile.

* Fixed how keyboard presses for up/down/left/right work with MANGA_UD reading mode. See issue #579

* Fixed cont reader for webtoons on mobile

* Fixed a small issue where top spacer would too quickly switch to prev chapter

* Updated user preferences to use same slider style. Removed some css that is not used.

* Added comic parser case for "Saga 001 (2012) (Digital) (Empire-Zone)"

* Added accessibility toggle to reading list order and aligned sliders to all use the same style.

* Removed a todo for checking on new image serving code. It works great.

* Fixed a missing await

* Auth guard will now check if an existing toast is present giving same message before poping the toast.

* Fixed alignment on phones for reading lists

* Moved sorters so they aren't resused between multiple threads. Slightly higher memory footprint.

* Fixed a broken unit test

* Code smells

* More unit test fixing
2021-09-15 11:06:29 -07:00
Joseph Milazzo
69ebd09fcd
Fixed an issue with ordering of chapters within a volume for selecting first cover image. (#529) 2021-08-27 12:51:06 -07:00
Joseph Milazzo
6edf3b083b
Bugfix/series cover not setting (#527)
* Fixed an issue where cover generation was failing on first add due to not checking correctly on First Volume.
2021-08-27 10:48:55 -07:00
Robbie Davis
a601942ec5
Cover generation issue on first scan flow (#517)
* Cover generation issue on first scan flow

- Fixed logic around whether a chapter cover image should be generated. New logic adds grouping priority, changes an AND to an OR and adds an additional check to see if the cover image has been lock (custom image uploaded)

* Sonar update

* Refactored out the cover image updating logic to a new call (ShouldUpdateCoverImage) and updated ONLY chapters. Added a blank slate unit test to build out conditions.

* Fixed up unit case

* Fixed some logic on when to update a cover image

* Fixed an issue where 1) we were refreshing metadata anytime we adjusted cover image on a series and 2) Cover generation wasn't properly being handled on first run.

* Cleaned up the code for when a cover image change needs to trigger a refresh metadata task

Co-authored-by: Joseph Milazzo <joseph.v.milazzo@gmail.com>
2021-08-22 11:32:38 -07:00