30 Commits

Author SHA1 Message Date
Joseph Milazzo
9cf4cf742b
Reader Bugs + New Features (#1536)
* Updated a typo in manage tasks of Reoccuring -> Recurring

* Fixed a bug in MinimumNumberFromRange where a regex wasn't properly constructed which could skew results.

* Fixed a bug where Volume numbers that were a float wouldn't render correctly in the manga reader menu.

* Added the ability to double click on the image to bookmark it. Optimized the bookmark and unbookmark flows to remove 2 DB calls and reworked some flow of calls to speed it up.

Fixed some logic where when using double (manga) flow, both of the images wouldn't show the bookmark effect, despite both of them being saved. Likewise, fixed a bug where both images weren't updating UI state, so switching from double (manga) to single, the second image wouldn't show as bookmarked without a refresh.

* Double click works perfectly for bookmarking

* Collection cover image chooser will now prompt with all series covers by default.

Reset button is now moved up to the first slot if applicable.

* When a Completed series is fully read by a user, a nightly task will now remove that series from their Want to Read list.

* Added ability to trigger Want to Read cleanup from Tasks page.

* Moved the brightness readout to the label line and fixed a bootstrap migration bug where small buttons weren't actually small.

* Implemented ability to filter against release year (min or max or both).

* Fixed a log message that wasn't properly formatted when scan finished an no files changes.

* Cleaned up some code and merged some methods

* Implemented sort by Release year metadata filter.

* Fixed the code that finds ComicInfo.xml inside archives to only check the root and check explicitly for casing, so it must be ComicInfo.xml.

* Dependency updates

* Refactored some strings into consts and used TriggerJob rather than just enqueuing

* Fixed the prefetcher which wasn't properly loading in the correct order as it was designed.

* Cleaned up all traces of CircularArray from MangaReader

* Removed a debug code

* Fixed a bug with webtoon reader in fullscreen mode where continuous reader wouldn't trigger

* When cleaning up series from users' want to read lists, include both completed and cancelled.

* Fixed a bug where small images wouldn't have the pagination area extend to the bottom on manga reader

* Added a new method for hashing during prod builds and ensure we always use aot

* Fixed a bug where the save button wouldn't enable when color change occured.

* Cleaned up some issues in one of contributor's PR.
2022-09-16 06:06:33 -07:00
Joseph Milazzo
d1a14f7e68
Logging Enhancements (#1521)
* Recreated Kavita Logging with Serilog instead of Default. This needs to be move out of the appsettings now, to allow auto updater to patch.

* Refactored the code to be completely configured via Code rather than appsettings.json. This is a required step for Auto Updating.

* Added in the ability to send logs directly to the UI only for users on the log route. Stopping implementation as Alerts page will handle the rest of the implementation.

* Fixed up the backup service to not rely on Config from appsettings.json

* Tweaked the Logging levels available

* Moved everything over to File-scoped namespaces

* Moved everything over to File-scoped namespaces

* Code cleanup, removed an old migration and changed so debug logging doesn't print sensitive db data

* Removed dead code
2022-09-12 17:25:48 -07:00
Joseph Milazzo
d7f2661655
Release Testing Day 1 (#1491)
* Fixed a bug where typeahead wouldn't automatically show results on relationship screen without an additional click.

* Tweaked the code which checks if a modification occured to check on seconds rather than minutes

* Clear cache will now clear temp/ directory as well.

* Fixed an issue where Chrome was caching api responses when it shouldn't had.

* Added a cleanup temp code

* Ensure genres get removed during series scan when removed from metadata.

* Fixed a bug where all epubs with a volume would show as Volume 0 in reading list

* When a scan is in progress, don't let the user delete the library.
2022-08-29 16:07:39 -07:00
Joseph Milazzo
b07aaf1eb5
Simplify Folder Watcher (#1484)
* Refactored Library Watcher to use Hangfire under the hood.

* Support .kavitaignore at root level.

* Refactored a lot of the library watching code to process faster and handle when FileSystemWatcher runs out of internal buffer space. It's still not perfect, but good enough for basic use.

* Make folder watching as experimental and default it to off by default.

* Revert #1479

* Tweaked the messaging for OPDS to remove a note about download role.

Moved some code closer to where it's used.

* Cleaned up how the events widget reports

* Fixed a null issue when deleting series in the UI

* Cleaned up some debug code

* Added more information for when we skip a scan

* Cleaned up some logging messages in CoverGen tasks

* More log message tweaks

* Added some debug to help identify a rare issue

* Fixed a bug where save bookmarks as webp could get reset to false when saving other server settings

* Updated some documentation on library watcher.

* Make LibraryWatcher fire every 5 mins
2022-08-28 13:20:46 -07:00
Joseph Milazzo
1e535a8184
More Scan Loop Bugfixes (#1471)
* Updated scan time for watcher to 30 seconds for non-dev. Moved ScanFolder off the Scan queue as it doesn't need to be there. Updated loggers

* Fixed jumpbar missing

* Tweaked the messaging for CoverGen

* When we return early due to nothing being done on library and series scan, make sure we kick off other tasks that need to occur.

* Fixed a foreign constraint issue on Volumes when we were adding to a new series.

* Fixed a case where when picking normalized series, capitalization differences wouldn't stack when they should.

* Reduced the logging output on dev and prod settings.

* Fixed a bug in the code that finds the highest directory from a file, where we were not checking against a normalized path.

* Cleaned up some code

* Fixed broken unit tests
2022-08-24 09:27:32 -07:00
Joseph Milazzo
1c9544fc47
Scan Loop Fixes (#1459)
* Added Last Folder Scanned time to series info modal.

Tweaked the info event detail modal to have a primary and thus be auto-dismissable

* Added an error event when multiple series are found in processing a series.

* Fixed a bug where a series could get stuck with other series due to a bad select query.

Started adding the force flag hook for the UI and designing the confirm.

Confirm service now also has ability to hide the close button.

Updated error events and logging in the loop, to be more informative

* Fixed a bug where confirm service wasn't showing the proper body content.

* Hooked up force scan series

* refresh metadata now has force update

* Fixed up the messaging with the prompt on scan, hooked it up properly in the scan library to avoid the check if the whole library needs to even be scanned. Fixed a bug where NormalizedLocalizedName wasn't being calculated on new entities.

Started adding unit tests for this problematic repo method.

* Fixed a bug where we updated NormalizedLocalizedName before we set it.

* Send an info to the UI when series are spread between multiple library level folders.

* Added some logger output when there are no files found in a folder. Return early if there are no files found, so we can avoid some small loops of code.

* Fixed an issue where multiple series in a folder with localized series would cause unintended grouping. This is not supported and hence we will warn them and allow the bad grouping.

* Added a case where scan series fails due to the folder being removed. We will now log an error

* Normalize paths when finding the highest directory till root.

* Fixed an issue with Scan Series where changing a series' folder to a different path but the original series folder existed with another series in it, would cause the series to not be deleted.

* Fixed some bugs around specials causing a series merge issue on scan series.

* Removed a bug marker

* Cleaned up some of the scan loop and removed a test I don't need.

* Remove any prompts for force flow, it doesn't work well. Leave the API as is though.

* Fixed up a check for duplicate ScanLibrary calls
2022-08-22 10:14:31 -07:00
Joseph Milazzo
0eac193248
New Scan Loop (#1447)
* Staging the code for the new scan loop.

* Implemented a basic idea of changes on drives triggering scan loop. Issues: 1. Scan by folder does not work, 2. Queuing system is very hacky and needs a separate thread, 3. Performance degregation could be very real.

* Started writing unit test for new loop code

* Implemented a basic method to scan a folder path with ignore support (not implemented, code in place)

* Added some code to the parser to build out the idea of processing series in batches based on some top level folder.

* Scan Series now uses the new code (folder based parsing) and now handles the LocalizedSeries issue.

* Got library scan working with the new folder-based scan loop. Updated code to set FolderPath (for improved scan times and partial scan support).

* Wrote some notes on update library scan loop.

* Removed migration for merge

* Reapplied the SeriesFolder migration after merge

* Refactored a check that used multiple db calls into one.

* Made lots of progress on ignore support, but some confusion on underlying library. Ticket created. On hold till then.

* Updated Scan Library and Scan Series to exit early if no changes are on the underlying folders that need to be scanned.

* Implemented the ability to have .kavitaignore files within your directories and Kavita will parse them and ignore files and directories based on rules within them.

* Fixed an issue where ignore files nested wouldn't stack with higher level ignores

* Wrote out some basic code that showcases how we can scan series or library based on file events on the underlying system. Very buggy, needs lots of edge case testing and logging and dupplication checking.

* Things are working kinda. I'm getting lost in my own code and complexity. I'm not sure it's worth it.

* Refactored ScanFiles out to Directory Service.

* Refactored more code out to keep the code clean.

* More unit tests

* Refactored the signature of ParsedSeries to use IList. Started writing unit tests and reworked the UpdateLibrary to work how it used to with new scan loop code (note: using async update library/series does not work).

* Fixed the bug where processSeriesInfos was being invoked twice per series and made the code work very similar to old code (except loose leaf files dont work) but with folder based scanning.

* Prep for unit tests (updating broken ones with new implementations)

* Just some notes. Not sure I want to finish this work.

* Refactored the LibraryWatcher with some comments and state variables.

* Undid the migrations in case I don't move forward with this branch

* Started to clean the code and prepare for finishing this work.

* Fixed a bad merge

* Updated signatures to cleanup the code and commit to the new strategy for scanning.

* Swapped out the code with async processing of series on a small library

* The new scan loop is working in both Sync and Async methods. The code is slow and not optimized. This represents a good point to start polling and applying optimizations.

* Refactored UpdateSeries out of Scanner and into a dedicated file.

* Refactored how ProcessTasks are awaited to allow more async

* Fixed an issue where side nav item wouldn't show correct highlight and migrated to OnPush

* Moved where we start to stopwatch to encapsulate the full scan

* Cleaned up SignalR events to report correctly (still needs a redesign)

* Remove the "remove" code until I figure it out

* Put in extremely expensive series deletion code for library scan.

* Have Genre and Tag update the DB immediately to avoid dup issues

* Taking a break

* Moving to a lock with People was successful. Need to apply to others.

* Refactored code for series level and tag and genre with new locking strategy.

* New scan loop works. Next up optimization

* Swapped out the Kavita log with svg for faster load

* Refactored metadata updates to occur when the series are being updated.

* Code cleanup

* Added a new type of generic message (Info) to inform the user.

* Code cleanup

* Implemented an optimization which prevents any I/O (other than an attribute lookup) for Library/Series Scan. This can bring a recently updated library on network storage (650 series) to fully process in 2 seconds.

Fixed a bug where File Analysis was running everytime for each non-epub file.

* Fixed ARM x64 builds not being able to view PDF cover images due to a bad update in DocNet.

* Some code cleanup

* Added experimental signalr update code to have a more natural refresh of library-detail page

* Hooked in ability to send new series events to UI

* Moved all scan (file scan only) tasks into Scan Queue. Made it so scheduled ScanLibraries will now check if any existing task is being run and reschedule for 3 hours, and 10 mins for scan series.

* Implemented the info event in the events widget and added a clear all button to dismiss all infos and errors.  Added --event-widget-info-bg-color

* Remove --drawer-background-color since it's not used

* When new series added, inject directly into the view.

* Some debug code cleanup

* Fixed up the unit tests

* Ensure all config directories exist on startup

* Disabled Library Watching (that will go in next build)

* Ensure update for series is admin only

* Lots of code changes, scan series kinda works, specials are splitting, optimizations are failing. Demotivated on this work again.

* Removed SeriesFolder migration

* Added the SeriesFolder migration

* Added a new pipe for dates so we can provide some nicer defaults. Added folder path to the series detail.

* The scan optimizations now work for NTFS systems.

* Removed a TODO

* Migrated all the times to use DateTime.Now and not Utc.

* Refactored some repo calls to use the includes flag pattern

* Implemented a check for the library scan optimization check to validate if the library was updated (type change, library rename, folder change, or series deleted) and let the optimization be bypassed.

* Added another optimization which will use just folder attribute of last write time if the drive is not NTFS.

* Fixed a unit test

* Some code cleanup
2022-08-19 05:42:38 -07:00
Joseph Milazzo
4e49aa47ce
Change Detection: On Push aka UI Smoothness (#1369)
* Updated Series Info Cards to use OnPush and hooked in progress events when we do a mark as read/unread on entities. These events update progress bars but also will now trigger a re-calculation on Read Time Left.

* Removed Library Card Component

* Refactored manga reader title and subtitle calculation to the backend.

* Coverted card actionables to onPush

* Series Card on push cleanup

* Updated edit collection tags for on push

* Update cover image chooser for on push

* Cleaned up carsouel reel

* Updated cover image to allow for uploading gif and webp files

* Bulk add to collection on push

* Updated bulk operation to use on push. Updated bulk operation to have mark as unread and read buttons explicitly. Updated so add to collection is visible and delete.

Fixed a bug where manage library component wasn't invoking the trackBy function

* Updating entity title for on push

* Removed file info component

* Updated Mange Library for on push

* Entity info cards on push

* List item on push

* Updated icon and title for on push and fixed some missing change detection on series detail

* Restricted the typeahead interface to simplify the design

* Edit Series Relation now shows a value in the dropdown for Parent relationships and disables the field.

* Updated edit series relation to focus on new typeahead when adding a new relationship

* Added some documentation and when Scanning a library, don't allow the user to enqueue the same job multiple times.

* Applied the No-enqueue if already enqueued logic to other tasks

* Library detail on push

* Updated events widget to onpush

* Card detail drawer on push. Card detail cover chooser now will show all chapter's covers for selection in cover chooser.

* Chapter metadata detail on push

* Removed Card Detail modal

* All collections on push

* Removed some comments

* Updated bulk selection to use an observable rather than function calls so new on push strategy works

* collection detail now uses on push and scroller is placed on correct element

* Updated library recommended to on push. Ensure that when mark as read occurs, the appropriate streams are refreshed.

* Updated library detail to on push

* Update metadata fiter to onpush. Bugs found and reported to Project

* person badge on push

* Read more on push

* Updated tag badge to on push

* User login on push

* When initing side nav, don't call an authenticated api until we are sure a user is logged in

* Updated splash container to on push

* Dashboard on push

* Side nav slight refactor around some api calls

* Cleaned up series card on push to use same cdRef naming convention

* Updated Static Files to use caching

* Added width and height to logo image

* shortcuts modal on push

* reading lists on push

* Reading list detail on push

* draggable ordered list on push

* Refactored reading-list-detail to use a new item which drastically reduces renders on operations

* series format on push

* circular loader on push

* Badge Expander on push

* update notification modal on push

* drawer on push

* Edit Series Modal on push

* reset password on push

* review series modal on push

* series metadata detail on push

* theme manager on push

* confirm reset password on push

* register on push

* confirm migration email on push

* confirm email on push

* add email to account migration on push

* user preferences on push. Made global settings default open

* edit series relation on push

* Fixed an edge case bug for next chapter where if the current volume had a single chapter of 1 and the next volume had a chapter number of 0, it would say there are no more chapters.

* Updated infinite scroller with on push support

* Moved some animations over to typeahead, not integrated yet.

* Manga reader is now on push

* Reader settings on push

* refactored how we close the book

* Updated table of contents for on push

* Updated book reader for on push. Fixed a bug where table of contents wasn't showing current page anchor due to a scroll calulation bug

* Small code tweak

* Icon and title on push

* nav header on push

* grouped typeahead on push

* typeahead on push and added a new trackby identity function to allow even faster rendering of big lists

* pdf reader on push

* code cleanup
2022-07-11 08:57:07 -07:00
Joseph Milazzo
8e69b6cfc0
Time Estimation Cleanup (#1301)
* Moved the calculation for time to read to the backend. Tweaked some logic around showing est time to complete.

* Added debug logging to help pinpoint a duplicate issue in Kavita.

* More combination logic is error checked in a special way for Robbie to reproduce an issue.

* Migrated chapter detail card to use backend for time calculation. Ensure we take all chapters into account for volume time calcs

* Tweaked messaging for some critical logs to include file

* Ensure pages count uses comma separated number

* Moved Hangfire annotations to interface level. Adjusted word count service to always recalculate when user requests via analyze series files.
2022-05-29 09:50:59 -05:00
Joseph Milazzo
1a128a3af4
Implemented the ability to read format tag and force special status. (#1284) 2022-05-25 10:05:16 -07:00
Joseph Milazzo
d411ab03f2
Major Search Enhancements (#1238)
* Pull progress information for some of the recommended stuff.

* Fixed some redirection code from last PR

* Implemented the ability to search for files in the search and open the series directly.

* Fixed nav search bar expanding too much

* Fixed a bug in nav module not having router so some links broke

* Fixed an issue where with new localized series tag, merging could fail if the user had 2 series with the series and localized series.

Added extra error handling for tracking series parsed from disk.

* Fixed the slowness when typing in a typeahead by using auditTime vs debounceTime

* Removed some cleaning of Edition tags from the Parser. Only Omnibus and Uncensored will be ignored when cleaning titles, Full Color, Full Contact, etc will now stay in the title for Series name.

* Implemented ability to search against chapter's title (from epub or title in comicinfo). This should help users search for books in a series a lot easier.

* Restrict each search type to 15 records only to keep query performant and UI useful.

* Wrote some extra messaging on invite user flow around email.

* Messaging update
2022-04-30 12:09:54 -05:00
Joseph Milazzo
1e51e39f66
Feature/misc (#1234)
* Fixed a bug where publication status could show as filled in when total number is 0 but there is a max count.

Add ComicInfo support for LocalizedSeries which will populate a Series LocalizedName.

Fixed an issue in tag constraint issues.

* Hooked in LocalizedSeries tag into merge step in scanner.

* Hooked in LocalizedSeries from ComicInfo into Kavita and also use it to help during merge phase to avoid 2 different series, if one file is using the name of the localized series.

* Reduced some extra string creation and updated epub library to ignore bad ToCs.

* Bumped dependencies to latest. When an epub doesn't have a dc:date with publication event type, default back to just a normal dc:date tag.

* Fixed a bug where webtoon reader would error out on first load due to how we passed the function to the reader

* Reverted the centering code
2022-04-28 14:50:31 -07:00
Joseph Milazzo
419eee7835
Publication Status Enhancements (#1231)
* Trim when reading some fields from ComicInfo. Adjusted css on the site to reduce nbsp

* Added Cancelled as a publication status

* Ensure we track volume number from ComicInfo for the count to determine publication status

* Publication Status will now check against volume number or chapter number (parsed or comicinfo). The UI will now display the progress, ie) 10/15 and will show the series as completed with a green tag badge if the progress is 100%.

* Tweaked the ordering of the tabs to make it more streamlined in the reading ordering of Kavita

* Tweaked the logic for filling in tag badge

* Added a new publication status of Ended for series that have finished releasing, but not all items are in Kavita

* Added some fields to edit series modal
2022-04-25 13:52:36 -05:00
Joseph Milazzo
78ffb8a8a2
Release Shakeout (#1186)
* Cleaned up some styles on the progress bar in book reader

* Fixed up some phone-hidden classes and added titles around the codebase. Stat reporting on first run now takes into account that admin user wont exist.

* Fixed manage library page not updating last scan time when a notification event comes in.

* Integrated SeriesSort ComicInfo tag (somehow it got missed)

* Some minor style changes and no results found for bookmarks on chapter detail modal

* Fixed the labels in action bar on book reader so Prev/Next are in same place

* Cleaned up some responsive styles around images and reduced custom classes in light of new display classes on collection detail and series detail pages

* Fixed an issue with webkit browsers and book reader where the scroll to would fail as the document wasn't fully rendered. A 10ms delay seems to fix the issue.

* Cleaned up some code and filtering for collections. Collection detail is missing filtering functionality somehow, disabled the button and will add in future release

* Correctly validate and show a message when a user is not an admin or has change password role when going through forget password flow.

* Fixed a bug on manage libraries where library last scan didn't work on first scan of a library, due to there being no updated series.

* Fixed a rendering issue with text being focused on confirm email page textboxes. Fixed a bug where when deleting a theme that was default, Kavita didn't reset Dark as the default theme.

* Cleaned up the naming and styles for side nav active item hover

* Fixed event widget to have correct styling on eink and light

* Tried to fix a rendering issue on side nav for light themes, but can't figure it out

* On light more, ensure switches are green

* Fixed a bug where opening a page with a preselected filter, the filter toggle button would require 2 clicks to collapse

* Reverted the revert of On Deck.

* Improved the upload by url experience by sending a custom fail error to UI when a url returns 401.

* When deleting a library, emit a series removed event for each series removed so user's dashboards/screens update.

* Fixed an api throwing an error due to text being sent back instead of json.

* Fixed a refresh bug with refreshing pending invites after deleting an invite. Ensure we always refresh pending invites even if user cancel's from invite, as they might invite, then hit cancel, where invite is still active.

* Fixed a bug where invited users with + in the email would fail due to validation, but UI wouldn't properly inform user.
2022-04-02 07:38:14 -07:00
Joseph Milazzo
ba77954d5c
Metadata Editing from the UI! (#1135)
* Added the skeleton code for layout, hooked up Age Rating, Publication Status, and Tags

* Tweaked message of Scan service to Finished scan of to better indicate the total scan time

* Hooked in foundation for person typeaheads

* Fixed people not populating typeaheads on load

* For manga/comics, when parsing, set the SeriesSort from ComicInfo if it exists.

* Implemented the ability to override and create new genre tags. Code is ready to flush out the rest.

* Ability to update metadata from the UI is hooked up. Next is locking.

* Updated typeahead to allow for non-multiple usage. Implemented ability to update Language tag in Series Metadata.

* Fixed a bug in GetContinuePoint for a case where we have Volumes, Loose Leaf chapters and no read progress.

* Added ETag headers on Images to allow for better caching (bookmarks and images in manga reader)

* Built out UI code to show locked indication to user

* Implemented Series locking and refactored a lot of styles in typeahead to make the lock setting work, plus misc cleanup.

* Added locked properties to dtos. Updated typeahead loading indicator to not interfere with close button if present

* Hooked up locking flags in UI

* Integrated regular field locking/unlocking

* Removed some old code

* Prevent input group from wrapping

* Implemented some basic layout for metadata on volume/chapter card modal. Refactored out all metadata from Chapter object in terms of UI and put into a separate call to ensure speedy delivery and simplicity of code.

* Refactored code to hide covers section if not an admin

* Implemented ability to modify a chapter/volume cover from the detail modal

* Removed a few variables and change cover image modal

* Added bookmark to single chapter view

* Put a temp fix in for a ngb v12 z-index bug (reported). Bumped ngb to 12.0 stable and fixed some small rendering bugs

* loading buttons ftw

* Lots of cleanup, looks like the story is finished

* Changed action name from Info to Details

* Style tweaks

* Fixed an issue where Summary would assume it's locked due to a subscription firing on setting the model

* Fixed some misc bugs

* Code smells

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2022-03-04 13:04:15 -08:00
Joseph Milazzo
eddbb7ab18
Event Widget Update (#1098)
* Took care of some notes in the code

* Fixed an issue where Extra might get flagged as special too early, if in a word like Extraordinary

* Moved Tag cleanup code into Scanner service. Added a SplitQuery to another heavy API. Refactored Scan loop to remove parallelism and use async instead.

* Lots of rework on the codebase to support detailed messages and easier management of message sending. Need to take a break on this work.

* Progress is being made, but slowly. Code is broken in this commit.

* Progress is being made, but slowly. Code is broken in this commit.

* Fixed merge issue

* Fixed unit tests

* CoverUpdate is now hooked into new ProgressEvent structure

* Refactored code to remove custom observables and have everything use standard messages$

* Refactored a ton of instances to NotificationProgressEvent style and tons of the UI to respect that too. UI is still a bit buggy, but wholistically the work is done.

* Working much better. Sometimes events come in too fast. Currently cover update progress doesn't display on UI

* Fixed unit tests

* Removed SignalREvent to minimize internal event types. Updated the UI to use progress bars. Finished SiteThemeService.

* Merged metadata refresh progress events and changed library scan events to merge cleaner in the UI

* Changed RefreshMetadataProgress to CoverUpdateProgress to reflect the event better.

* Theme Cleanup (#1089)

* Fixed e-ink theme not properly applying correctly

* Fixed some seed changes. Changed card checkboxes to use our themed ones

* Fixed recently added carousel not going to recently-added page

* Fixed an issue where no results found would show when searching for a library name

* Cleaned up list a bit, typeahead dropdown still needs work

* Added a TODO to streamline series-card component

* Removed ng-lazyload-image module since we don't use it. We use lazysizes

* Darken card on hover

* Fixing accordion focus style

* ux pass updates

- Fixed typeahead width
- Fixed changelog download buttons
- Fixed a select
- Fixed various input box-shadows
- Fixed all anchors to only have underline on hover
- Added navtab hover and active effects

* more ux pass

- Fixed spacing on theme cards
- Fixed some light theme issues
- Exposed text-muted-color for theme card subtitle color

* UX pass fixes

- Changed back to bright green for primary on dark theme
- Changed fa icon to black on e-ink

* Merged changelog component

* Fixed anchor buttons text decoration

* Changed nav tabs to have a background color instead of open active state

* When user is not authenticated, make sure we set default theme (dark)

* Cleanup on carousel

* Updated Users tab to use small buttons with icons to align with Library tab

* Cleaned up brand to not underline, removed default link underline on hover in dropdown and pill tabs

* Fixed collection detail posters not rendering

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>

* Bump versions by dotnet-bump-version.

* Tweaked some of the emitting code

* Some css, but pretty bad. Robbie please save me

* Removed a todo

* styling update

* Only send filename on FileScanProgress

* Some console.log spam cleanup

* Various updates

* Show events widget activity based on activeEvents

* progress bar color updates

* Code cleanup

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2022-02-18 18:57:37 -08:00
Joseph Milazzo
ce3bd92244
Scanner not merging with series that has LocalizedName match (#950)
* When performing a scan, series should group if they share the same localized name as a pre-existing series.

* Fixed a bug where a series with a different name and localized name weren't merging with a different set of files with the same naming as localized name.
2022-01-16 15:48:15 -08:00
Joseph Milazzo
996c68d753
Feature/parse scanned files tests (#934)
* Fixed a bug in CleanupBookmarks where the Except was deleting all files because the path separators didn't match.

* Added unit tests for ParseScannedFiles.cs.

* Fixed some unit tests. Parser will now clear out multiple spaces in a row and replace with a single.
2022-01-13 11:40:45 -08:00
Joseph Milazzo
ab8c762830
Metadata Performance Scan (#921)
* Refactored updating chapter metadata from ComicInfo into the Scan loop. This let's us avoid an additional N file reads (expensive) in the metadata service, as we already have to read them in the scan loop.

* Refactored Series level metadata aggregation into the scan loop. This allows for the batching of DB updates to be much smaller, thus faster without much overhead of GC.

* Refactored some of the code for ProcessFile to remove a few redundant if statements
2022-01-09 10:36:24 -08:00
Joseph Milazzo
a1a6333f09
Bookmark Refactor (#893)
* Fixed a bug which didn't take sort direction when not changing sort field

* Added foundation for Bookmark refactor

* Code broken, need to take a break. Issue is Getting bookmark image needs authentication but UI doesn't send.

* Implemented the ability to send bookmarked files to the web. Implemented ability to clear bookmarks on disk on a re-occuring basis.

* Updated the bookmark design to have it's own card that is self contained. View bookmarks modal has been updated to better lay out the cards.

* Refactored download bookmark codes to select files from bookmark directory directly rather than open underlying files.

* Wrote the basic logic to kick start the bookmark migration.

Added Installed Version into the DB to allow us to know more accurately when to run migrations

* Implemented the ability to change the bookmarks directory

* Updated all references to BookmarkDirectory to use setting from the DB.

Updated Server Settings page to use 2 col for some rows.

* Refactored some code to DirectoryService (hasWriteAccess) and fixed up some unit tests from a previous PR.

* Treat folders that start with ._ as blacklisted.

* Implemented Reset User preferences. Some extra code to prep for the migration.

* Implemented a migration for existing bookmarks to using new filesystem based bookmarks
2022-01-05 09:56:49 -08:00
Joseph Milazzo
28688ada8e
In-Depth Filtering (#850)
* Laying the foundation for the filter rework

* Filtering by Genre is now possible.

* Cleaned up code and preparing for People filtering

* People filtering is hooked up for the frontend

* Filtering now works. On Deck does not work with filtering currently due to a unique implementation.

* More cleanup

* Implemented the ability to reset the filters

* Added a mobile drawer for filtering

* Added some additional cases for NaturalSortComparer. Filter now uses a drawer on smaller screens.

* Fixed a bug where backup service was not pointing to the correct directory.

* Undid the fix, it's working as expected
2021-12-15 08:23:10 -08:00
Joseph Milazzo
bbe8f800f6
.NET 6 Coding Patterns + Unit Tests (#823)
* Refactored all files to have Interfaces within the same file. Started moving over to file-scoped namespaces.

* Refactored common methods for getting underlying file's cover, pages, and extracting into 1 interface.

* More refactoring around removing dependence on explicit filetype testing for getting information.

* Code is buildable, tests are broken. Huge refactor (not completed) which makes most of DirectoryService testable with a mock filesystem (and thus the services that utilize it).

* Finished porting DirectoryService to use mocked filesystem implementation.

* Added a null check

* Added a null check

* Finished all unit tests for DirectoryService.

* Some misc cleanup on the code

* Fixed up some bugs from refactoring scan loop.

* Implemented CleanupService testing and refactored more of DirectoryService to be non-static.

Fixed a bug where cover file cleanup wasn't properly finding files due to a regex bug.

* Fixed an issue in CleanupBackup() where we weren't properly selecting database files older than 30 days. Finished CleanupService Tests.

* Refactored Flatten and RemoveNonImages to directory service to allow CacheService to be testable.

* Finally have CacheService tested. Rewrote GetCachedPagePath() to be much more straightforward & performant.

* Updated DefaultParserTests.cs to contain all existing tests and follow new test layout format.

* All tests fixed up
2021-12-05 08:58:53 -08:00
Joseph Milazzo
e7619e6b0a
Local Metadata Integration Part 1 (#817)
* Started with some basic plumbing with comic info parsing updating Series/Volume.

* We can now get chapter title from comicInfo.xml

* Hooked in the ability to store people into the chapter metadata.

* Removed no longer used imports, fixed up some foreign key constraints on deleting series with person linked.

* Refactored Summary out of the UI for Series into SeriesMetadata. Updated application to .net 6. There is a bug in metadata code for updating.

* Removed the parallel.ForEach with a normal foreach which lets us use async. For I/O heavy code, shouldn't change much.

* Refactored scan code to only check extensions with comic info, fixed a bug on scan events not using correct method name, removed summary field (still buggy)

* Fixed a bug where on cancelling a metadata request in modal, underlying button would get stuck in a disabled state.

* Changed how metadata selects the first volume to read summary info from. It will now select the first non-special volume rather than Volume 1.

* More debugging and found more bugs to fix

* Redid all the migrations as one single one. Fixed a bug with GetChapterInfo returning null when ChapterMetadata didn't exist for that Chapter.

Fixed an issue with mapper failing on GetChapterMetadata. Started work on adding people and a design for people.

* Fixed a bug where checking if file modified now takes into account if file has been processed at least once. Introduced a bug in saving people to series.

* Just made code compilable again

* Fixed up code. Now people for series and chapters add correctly without any db issues.

* Things are working, but I'm not happy with how the management of Person is. I need to take into account that 1 person needs to map to an image and role is arbitrary.

* Started adding UI code to showcase chapter metadata

* Updated workflow to be .NET 6

* WIP of updating card detail to show the information more clearly and without so many if statements

* Removed ChatperMetadata and store on the Chapter itself. Much easier to use and less joins.

* Implemented Genre on SeriesMetadata level

* Genres and People are now removed from Series level if they are no longer on comicInfo

* PeopleHelper is done with unit tests. Everything is working.

* Unit tests in place for Genre Helper

* Starting on CacheHelper

* Finished tests for ShouldUpdateCoverImage. Fixed and added tests in ArchiveService/ScannerService.

* CacheHelper is fully tested

* Some DI cleanup

* Scanner Service now calls GetComicInfo for books. Added ability to update Series Sort name from metadata files (mainly epub as comicinfo doesn't have a field)

* Forgot to move a line of code

* SortName now populates from metadata (epub only, ComicInfo has no tags)

* Cards now show the chapter title name if it's set on hover, else will default back to title.

* Fixed a major issue with how MangaFiles were being updated with LastModified, which messed up our logic for avoiding refreshes.

* Woohoo, more tests and some refactors to be able to test more services wtih mock filesystem. Fixed an issue where SortName was getting set as first chapter, but the Series was in a group.

* Refactored the MangaFile creation code into the DbFactory where we also setup the first LastModified update.

* Has file changed bug is now finally fixed

* Remove dead genres, refactor genre to use title instead of name.

* Refactored out a directory from ShouldUpdateCoverImage() to keep the code clean

* Unit tests for ComicInfo on BookService.

* Refactored series detail into it's own component

* Series-detail now received refresh metadata events to refresh what's on screen

* Removed references to Artist on PersonRole as it has no metadata mapping

* Security audit

* Fixed a benchmark

* Updated JWT Token generator to use new methods in .NET 6

* Updated all the docker and build commands to use net6.0

* Commented out sonar scan since it's not setup for net6.0 yet.
2021-12-02 09:02:34 -08:00
Joseph Milazzo
d5d03528d4
Regression Fix (#680)
* Ensure we mount the backups directory for Docker users

* Fixed a huge logic bug that deleted files in users libraries
2021-10-16 11:24:30 -07:00
Joseph Milazzo
98e5cb87ed
Bulk Operations for In Progress and Recently Added (#677)
* Don't log a message about bad match if the file is a cover image

* Enable bulk operations for In Progress and Recently Added

* Fixed a bad logic case
2021-10-15 14:32:17 -07:00
Joseph Milazzo
355f50fd5d
Logging Cleanup (#668)
* Do not allow non-admins to change their passwords when authentication is disabled

* Clean up the login page so that input field text is black

* cleanup some resizing when typing a password and having a lot of users

* Changed the LastActive for a user to not just be login, but also when they open an already authenticated session.

* Removed some verbose debugging statements and moved some debug to information to be more prevelant to logs for default installs.

* In Progress now sends progress information on the Series

* Add ability to add cards to recently added when new series are added in backend

* Implemented the ability to click the glasses icon to turn off incognito mode from within the reader so you can start tracking progress

* Don't warn the user about authentication when they don't touch that control
2021-10-14 08:14:03 -07:00
Joseph Milazzo
82b5b599e0
Foundational Cover Image Rework (#584)
* Updating wording on card item when total pages is 0, to be just "Cannot Read" since it could be a non-archive file

* Refactored cover images to be stored on disk. This first commit has the extraction to disk and the metadata service to handle updating when applicable.

* Refactored code to have the actual save to cover image directory done by ImageService.

* Implemented the ability to override cover images.

* Some cleanup on Image service

* Implemented the ability to cleanup old covers nightly

* Added a migration to migrate existing covers to new cover image format (files).

* Refactored all CoverImages to just be the filename, leaving the Join with Cover directory to higher level code.

* Ensure when getting user progress, we pick the first.

* Added cleanup cover images for deleted tags. Don't pull any cover images that are blank.

* After series update, clear out cover image. No change on UI, but just keeps things clear before metadata refresh hits

* Refactored image formats for covers to ImageService.

* Fixed an issue where after refactoring how images were stored, the cleanup service was deleting them after each scan.

* Changed how ShouldUpdateCoverImage works to check if file exists or not even if cover image is locked.

* Fixed unit tests

* Added caching back to cover images.

* Caching on series as well

* Code Cleanup items

* Ensure when checking if a file exists in MetadataService, that we join for cover image directory. After we scan library, do one last filter to delete any series that have 0 pages total.

* Catch exceptions so we don't run cover migration if this is first time run.

* After a scan, only clear out the cache directory and not do a deep clean.

* Implemented the ability to backup custom locked covers only.

* Fixed unit tests

* Trying to figure out why GA crashes when running MetadataServiceTests.cs

* Some debugging on GA tests not running

* Commented out tests that were causing issues in GA.

* Fixed an issue where series cover images wouldn't migrate

* Fixed the updating of links to actually do all series and not just locked
2021-09-21 19:15:29 -05:00
Joseph Milazzo
a9ea03469f
Bugfix/scanner issue (#576)
* Refactored the scanner to hopefully fix a hard to reproduce KeyNotFoundException from  GetInfosByName.

* Added parsing support for "A Compendium of Ghosts - 031 - The Third Story_ Part 12 (Digital) (Cobalt001)"
2021-09-12 09:48:21 -07:00
Joseph Milazzo
ebd4ec25bf
Shakeout Fixes (#422)
# Fixed
- Fixed: Clean the pdf extension from Series title for PDF types
- Fixed: Fixed a bug where a forced metadata refresh wouldn't trigger a volume to force a refresh of cover image
- Fixed: Fixed an issue where Removing series no longer on disk would not use the Series Format and thus after deleting files, they would not be removed.
- Fixed: Fixed an issue with reading a single image file, where the cache code would not properly move the file
- Fixed: For Specials, Get Next/Prev Chapter should use the filename instead of arbitrary Number (which is always 0). Use the same sorting logic when requesting volumes on series detail, so sorting can happen in the backend.

# Added
- Added: (Accessibility) Nearly every page has had a title set for it 

===============================================================================

* Clean the pdf extension from ParseSeries

* Fixed a bug where forced metadata refresh wouldn't trigger the volume to update it's image.

* Added titles to most pages to help distinguish back/forward history.

Fixed a bug in the scanner which didn't account for Format when calculating if we need to remove a series not on disk.

* For Specials, Get Next/Prev Chapter should use the filename instead of arbitrary Number (which is always 0). Use the same sorting logic when requesting volumes on series detail, so sorting can happen in the backend.

* Fixed unit tests
2021-07-23 18:02:14 -05:00
Joseph Milazzo
b0df67cdda
PDF Support + MORE!!!! (#416)
# Added
- Added support for PDFs within Kavita. PDFs will open in the Manga reader and you can read through them as images. PDFs are heavier than archives, so they may take longer to open for reading. (Fixes #187)

# Changed
- Changed: Major change in how Kavita libraries work. Kavita libraries will now allow for mixed media types, that means you can have raw images, archives, epubs, and pdfs all within your Manga library. In the case that the same Series exists between 2 different types of medias, they will be separated and an icon will show to help you identify the types. The correct reader will open regardless of what library you are on. Note: Nightly users need to delete their Raw Images libraries before updating.

# Fixed
- Fixed: Fixed an issue where checking if a file was modified since last scan always returned true, meaning we would do more I/O than was needed (Fixes #415)
- Fixed: There wasn't enough spacing on the top menu bar on the Manga reader
- Fixed: Fixed a bug where user preferences dark mode control always showed true, even if you were not using dark mode

# Dev stuff
- For image extraction, if there is only 1 image we will extract  just that, else we will extract only images
- Refactored all the Parser code out of the ScannerService into a self contained class. The class should be created for any scans, allowing multiple tasks to run without any chance of cross over.



* Fixed indentation for cs files

* Fixed an issue where the logic for if a file had been modified or not was not working and always saying modified, meaning we were doing more file I/O than needed.

* Implemented the ability to have PDF books. No reader functionality.

* Implemented a basic form of scanning for PDF files. Reworked Image based libraries to remove the need to separate in a special library and instead just work within the Manga/Comic library.

* Removed the old library types.

* Removed some extra code around old raw library types

* Fully implemented PDF support into Kavita by using docnet. Removed old libraries we tried that did not work. PDFs take about 200ms to save the file to disk, so they are much slower than reading archives.

* Refactored Libraries so that they can have any file extension and the UI will decide which reader to use.

* Reworked the Series Parsing code.

We now use a separate instance for each task call, so there should be no cross over if 2 tasks are running at the same time.

Second, we now store Format with the Series, so we can have duplicate Series with the same name, but a different type of files underneath.

* Fixed PDF transparency issues

- Used this code to fix an issue when a PDF page doesn't have a background. https://github.com/GowenGit/docnet/issues/8#issuecomment-538985672

- This also fixes the same issue for cover images

* Fixed an issue where if a raw image was in a directory with non-image files, those would get moved to cache when trying to open the file.

* For image extraction, if there is only 1 image, just copy that to cache instead of everything else in the directory that is an image.

* Add some spacing to the top menu bar

* Added an icon to the card to showcase the type of file

* Added a tag badge to the series detail page

* Fixed a bug in user preferences where dark mode control would default to true, even if you weren't on it

* Fixed some tests up

* Some code smells

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2021-07-22 21:13:24 -05:00