mirror of
https://github.com/Kareadita/Kavita.git
synced 2025-06-21 14:30:33 -04:00
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Joe Milazzo
|
b62d340bb3
|
Performance Improvements and Some Polish (#1702)
* Auto scale reading timeline * Added benchmarks for SharpImage and NetVips. When an epub has a malformed page, catch the error and present it better to the user. * Added a hint for an upcoming feature * Slightly sped up word count for epubs * Added one more test to reflect actual code. * Some light cleanup * Use compact number for stat lists * Fixed brightness being broken on manga reader * Replaced CoverToWebP SharpImage version with NetVips which is MUCH lighter on memory and CPU. * Added last modified on the progress dto for CdDisplayEx. * Code cleanup * Forgot one cleanup |
||
|
Joseph Milazzo
|
0eac193248
|
New Scan Loop (#1447)
* Staging the code for the new scan loop. * Implemented a basic idea of changes on drives triggering scan loop. Issues: 1. Scan by folder does not work, 2. Queuing system is very hacky and needs a separate thread, 3. Performance degregation could be very real. * Started writing unit test for new loop code * Implemented a basic method to scan a folder path with ignore support (not implemented, code in place) * Added some code to the parser to build out the idea of processing series in batches based on some top level folder. * Scan Series now uses the new code (folder based parsing) and now handles the LocalizedSeries issue. * Got library scan working with the new folder-based scan loop. Updated code to set FolderPath (for improved scan times and partial scan support). * Wrote some notes on update library scan loop. * Removed migration for merge * Reapplied the SeriesFolder migration after merge * Refactored a check that used multiple db calls into one. * Made lots of progress on ignore support, but some confusion on underlying library. Ticket created. On hold till then. * Updated Scan Library and Scan Series to exit early if no changes are on the underlying folders that need to be scanned. * Implemented the ability to have .kavitaignore files within your directories and Kavita will parse them and ignore files and directories based on rules within them. * Fixed an issue where ignore files nested wouldn't stack with higher level ignores * Wrote out some basic code that showcases how we can scan series or library based on file events on the underlying system. Very buggy, needs lots of edge case testing and logging and dupplication checking. * Things are working kinda. I'm getting lost in my own code and complexity. I'm not sure it's worth it. * Refactored ScanFiles out to Directory Service. * Refactored more code out to keep the code clean. * More unit tests * Refactored the signature of ParsedSeries to use IList. Started writing unit tests and reworked the UpdateLibrary to work how it used to with new scan loop code (note: using async update library/series does not work). * Fixed the bug where processSeriesInfos was being invoked twice per series and made the code work very similar to old code (except loose leaf files dont work) but with folder based scanning. * Prep for unit tests (updating broken ones with new implementations) * Just some notes. Not sure I want to finish this work. * Refactored the LibraryWatcher with some comments and state variables. * Undid the migrations in case I don't move forward with this branch * Started to clean the code and prepare for finishing this work. * Fixed a bad merge * Updated signatures to cleanup the code and commit to the new strategy for scanning. * Swapped out the code with async processing of series on a small library * The new scan loop is working in both Sync and Async methods. The code is slow and not optimized. This represents a good point to start polling and applying optimizations. * Refactored UpdateSeries out of Scanner and into a dedicated file. * Refactored how ProcessTasks are awaited to allow more async * Fixed an issue where side nav item wouldn't show correct highlight and migrated to OnPush * Moved where we start to stopwatch to encapsulate the full scan * Cleaned up SignalR events to report correctly (still needs a redesign) * Remove the "remove" code until I figure it out * Put in extremely expensive series deletion code for library scan. * Have Genre and Tag update the DB immediately to avoid dup issues * Taking a break * Moving to a lock with People was successful. Need to apply to others. * Refactored code for series level and tag and genre with new locking strategy. * New scan loop works. Next up optimization * Swapped out the Kavita log with svg for faster load * Refactored metadata updates to occur when the series are being updated. * Code cleanup * Added a new type of generic message (Info) to inform the user. * Code cleanup * Implemented an optimization which prevents any I/O (other than an attribute lookup) for Library/Series Scan. This can bring a recently updated library on network storage (650 series) to fully process in 2 seconds. Fixed a bug where File Analysis was running everytime for each non-epub file. * Fixed ARM x64 builds not being able to view PDF cover images due to a bad update in DocNet. * Some code cleanup * Added experimental signalr update code to have a more natural refresh of library-detail page * Hooked in ability to send new series events to UI * Moved all scan (file scan only) tasks into Scan Queue. Made it so scheduled ScanLibraries will now check if any existing task is being run and reschedule for 3 hours, and 10 mins for scan series. * Implemented the info event in the events widget and added a clear all button to dismiss all infos and errors. Added --event-widget-info-bg-color * Remove --drawer-background-color since it's not used * When new series added, inject directly into the view. * Some debug code cleanup * Fixed up the unit tests * Ensure all config directories exist on startup * Disabled Library Watching (that will go in next build) * Ensure update for series is admin only * Lots of code changes, scan series kinda works, specials are splitting, optimizations are failing. Demotivated on this work again. * Removed SeriesFolder migration * Added the SeriesFolder migration * Added a new pipe for dates so we can provide some nicer defaults. Added folder path to the series detail. * The scan optimizations now work for NTFS systems. * Removed a TODO * Migrated all the times to use DateTime.Now and not Utc. * Refactored some repo calls to use the includes flag pattern * Implemented a check for the library scan optimization check to validate if the library was updated (type change, library rename, folder change, or series deleted) and let the optimization be bypassed. * Added another optimization which will use just folder attribute of last write time if the drive is not NTFS. * Fixed a unit test * Some code cleanup |
||
Joseph Milazzo
|
041cd22c8f
|
Last Shakeout (#1342)
* Fixed a bug where series estimate reading time could be calculated before we restore esisting time. * Cleaned up debug code for the reader * Fixed an issue where pagination areas on wide images wasn't proper height * Fixed a pagination height calc * Small change |
||
|
Joseph Milazzo
|
4fd4ca6f07
|
Release Shakeout (#1340)
* Fixed a bug where analyze series would not force a re-analysis. Fixed a bug where if files weren't changed since last analysis, then series word count got reset to 0. * Fixed epub images not loading in detail drawer * Fixed a bug on double page layout where the reader would be wonky when moving to and from mobile layout. * package-lock.json updated * Cleaned up some wording * Moved a conditional on jump bar * Bugfix for Double Page Rendering skipping pages (#1339) * Bump versions by dotnet-bump-version. * Double (Manga) fixes . Fixed: *ngIf condition, last page loading Double . Added: isLoose, pageAmount, and CanvasImageNextDouble to keep track of double sequence breaks (nn/n/w) . Fixed: ShouldRenderReverseDouble and pageAmount conditions . Added: Setting isLoose on loadPage() . Added: canvasImageNextDouble in loadPage() Co-authored-by: majora2007 <josephmajora@gmail.com> * Added comments for Magunjun's PR. Co-authored-by: Marcelo Guimarães Junior <75567460+magujun@users.noreply.github.com> |
||
|
Joseph Milazzo
|
c809597cf0
|
More Stat collection (#1337)
* Ensure that Scan Series triggers a file analysis task. * Tweaked concurrency for Analyze Files * Implemented new stats tracking for upcoming performance release. |
||
|
Joseph Milazzo
|
f54eb5865b
|
All Around Polish (#1328)
* Added --card-list-item-bg-color for the card list items * Updated the card list item progress to match how cards render * Implemented the ability to configure how many backups are retained. * Fixed a bug where odd jump keys could cause a bad index error for jump bar * Commented out more code for the pagination route if we go with that. * Reverted a move of DisableConcurrentExecution to interface, as it seems to not work there. * Updated manga format utility code to pipes * Fixed bulk selection on series detail page * Fixed bulk selection on all other pages * Changed card item to OnPush * Updated image component to OnPush * Updated Series Card to OnPush * Updated Series Detail to OnPush * Lots of changes here. Integrated parentscroll support on card detail layout. Added jump bar (custom js implementation) on collection, reading list and all series pages. Updated UserParams to default to no pagination. Lots of cleanup all around * Updated some notes on a module use * Some code cleanup * Fixed up a broken test due to the mapper not being configured in the test. * Applied TabID pattern to edit collection tags * Applied css from series detail to collection detail page to remove double scrollbar * Implemented the ability to sort by Time To Read. * Throw an error to the UI when we extract an archive and it contains invalid characters in the filename for the Server OS. * Tweaked how the page scrolls for jumpbar on collection detail. We will have to polish another release * Cleaned up the styling on directory picker * Put some code in but it doesn't work for scroll to top on virtual scrolling. I'll do it later. * Fixed a container bug |
||
|
Joseph Milazzo
|
3ab3a10ae7
|
New PDF Reader (#1324)
* Refactored all the code that opens the reader to use a unified function. Added new library and setup basic pdf reader route. * Progress saving is implemented. Targeting ES6 now. * Customized the toolbar to remove things we don't want, made the download button download with correct filename. Adjusted zoom setting to work well on first load regardless of device. * Stream the pdf file to the UI rather than handling the download ourselves. * Started implementing a custom toolbar. * Fixed up the jump bar calculations * Fixed filtering being broken * Pushing up for Robbie to cleanup the toolbar layout * Added an additional button. Working on logic while robbie takes styling * Tried to fix the code for robbie * Tweaks for fonts * Added button for book mode, but doesn't seem to work after renderer is built * Removed book mode * Removed the old image caching code for pdfs as it's not needed with new reader * Removed the interfaces to extract images from pdf. * Fixed original pagination area not scaling correctly * Integrated series remove events to library detail * Cleaned up the getter naming convention * Cleaned up some of the manga reader code to reduce cluter and improve re-use * Implemented Japanese parser support for volume and chapters. * Fixed a bug where resetting scroll in manga reader wasn't working * Fixed a bug where word count grew on each scan. * Removed unused variable * Ensure we calculate word count on files with their own cache timestamp * Adjusted size of reel headers * Put some code in for moving on original image with keyboard, but it's not in use. * Cleaned up the css for the pdf reader * Cleaned up the code * Tweaked the list item so we show scrollbar now when fully read |
||
|
Joseph Milazzo
|
bbc48a5f5b
|
Infinite Scroll + List View + Cover Upload Redesign (#1319)
* Started with the redesign of the cover image chooser redesign to be less click intensive for volume/chapter images. Made some headings bold in card detail drawer. * Tweaked the styles * Moved where the info cards show * Added an ability to open a page settings drawer * Cleaned up some old code that isn't needed anymore. * Started implementing a list view. Refactored some title code to a dedicated component * List view implemented but way too many API calls. Either need caching or adjusting the SeriesDetail api. * Fixed a bug where if the progress bar didn't render on a card item while a download was in progress, the download indicator would be removed. * Large refactor to move a lot of the needed fields to the chapter and volume dtos for series detail. All fields are noted when only used in series detail. * Implemented cards for other tabs (except related) * Fixed the unit test which needed a mocked reader service call. * More cleanup around age rating and removing old code from the refactor. Commented out sorting till i feel motivated to work on that. * Some cleanup and restored cards as initial layout. Time to test this out and see if there is value add. * Added ability for Chapters tab to show the volume chapters belong to (if applicable) * Adding style fixes * Cover image updates, don't allow the first image (which is what is currently set) to respond to cover changes. Hide the ID field on list item for series detail. * Refactored the title for list item to be injectable * Cleaned up the selection code to make it less finicky on mobile when tap scrolling. * Refactored chapter tab to show volume as well on list view. * Ensure word count shows for Volumes * Started adding virtual scrolling, pushing up so Robbie can mess around * Started adding virtual scrolling, pushing up so Robbie can mess around * Fixed a bug where all chapters would come under specials * Show title data as accent if set. * Style fixes for virtual scroller * Restyling scroll * Implemented a way to show storyline with virtual scrolling * Show Word Count for chapters and cleaned up some logics. * I might have card layout working with virtual scroll code. * Some cleanup to hide more system like properties from info bar on series detail page. Fixed some missing time estimate info on storyline chapters. * Fixed a regression on series service when I integrated VolumeTitle. * Refactored read time to the backend. Added WordCount to the volume itself so we don't need to calculate on frontend. When asking to analyze files from a series, force the calculation. * Fixed SeriesDetail api code * Fixed up the code in the drawer to better update list/card mode * Basic infinite scroll implemented, however due to how we are updating the list to render, we are re-rending cards that haven't been touched. * Updated how we render and layout data for infinite scroll on library detail. It's almost there. * Started laying foundation for loading pages backwards. Removed lazy loading of images since we are now using virtual paging. * Hooked in some basic code to allow user to load a prev page with infinite scroll. * Fixed up series detail api and undid the non-lazy loaded images. Changed the router to help with this infinite loading on Firefox issue. * Fixed up some naming issues with Series Detail and added a new test. * This is an infinite scroll without pagination implementation. It is not fully done, but off to a good start. Virtual scroller with jump bar is working pretty well, def needs more polishing and tweaking. There are hacks in this implementation that need to be revisited. * Refactored code so that we don't use any pagination and load all results by default. * Misc code cleanup from build warnings. * Cleaned up some logic for how to display titles in list view. * More title cleanup for specials * Hooked up page layout to user preferences and renamed an existing user pref name to match the dto. * Swapped out everything but storyline with virtual-scroller over CDK * Removed CDK from series detail. * Default value for migration on page layout * Updating card layout for library detail page * fixing height for mobile * Moved scrollbar * Tweaked some styling for layouts when there is no data * Refactored the series cards into their own component to make it re-usable. * More tweaks on series info cards layout and enhanced a few pages with trackby functions. * Removed some dead code * Added download on series detail to actionables to fit in with new scroll strategy. * Fixed language not being updated and sent to the backend for series update. * Fixed a bad migration (if you ran any prior migration in this branch, you need to undo before you use this commit) * Adding sticky tabs * fixed mobile gap on sticky tab * Enhanced the card title for books to show number up front. * Adjusted the gutters on admin dashboard * Removed debug code * Removing duplicate book title * Cleaned up old references to cdk scroller * Implemented a basic jump bar scaling algorithm. Not perfect, but works pretty well. * Code smells Co-authored-by: Robbie Davis <robbie@therobbiedavis.com> |
||
|
Joseph Milazzo
|
8e69b6cfc0
|
Time Estimation Cleanup (#1301)
* Moved the calculation for time to read to the backend. Tweaked some logic around showing est time to complete. * Added debug logging to help pinpoint a duplicate issue in Kavita. * More combination logic is error checked in a special way for Robbie to reproduce an issue. * Migrated chapter detail card to use backend for time calculation. Ensure we take all chapters into account for volume time calcs * Tweaked messaging for some critical logs to include file * Ensure pages count uses comma separated number * Moved Hangfire annotations to interface level. Adjusted word count service to always recalculate when user requests via analyze series files. |
||
|
Joseph Milazzo
|
63475722ea
|
Drawers, Estimated Reading Time, Korean Parsing Support (#1297)
* Started building out idea around detail drawer. Need code from word count to continue * Fixed the logic for caluclating time to read on comics * Adding styles * more styling fixes * Cleaned up the styles a bit more so it's at least functional. Not sure on the feature, might abandon. * Pulled Robbie's changes in and partially migrated them to the drawer. * Add offset overrides for offcanvas so it takes our header into account * Implemented a basic time left to finish the series (or at least what's in Kavita). Rough around the edges. * Cleaned up the drawer code. * Added Quick Catch ups to recommended page. Updated the timeout for scan tasks to ensure we don't run 2 at the same time. * Quick catchups implemented * Added preliminary support for Korean filename parsing. Reduced an array alloc that is called many thousands of times per scan. * Fixing drawer overflow * Fixed a calculation bug with average reading time. * Small spacing changes to drawer * Don't show estimated reading time if the user hasn't read anything * Bump eventsource from 1.1.1 to 2.0.2 in /UI/Web Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.1.1 to 2.0.2. - [Release notes](https://github.com/EventSource/eventsource/releases) - [Changelog](https://github.com/EventSource/eventsource/blob/master/HISTORY.md) - [Commits](https://github.com/EventSource/eventsource/compare/v1.1.1...v2.0.2) --- updated-dependencies: - dependency-name: eventsource dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Added image to series detail drawer Co-authored-by: Robbie Davis <robbie@therobbiedavis.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Joseph Milazzo
|
c1490d6e86
|
Word Count (#1286)
* Adding some code for Robbie * See more on series detail metadata area is now at the bottom on the section * Cleaned up subtitle headings to use a single class for offset with actionables * Added some markup for the new design, waiting for Robbie to finish it off * styling age-rating badge * Started hooking up basic analyze file service and hooks in the UI. Basic code to implement the count is implemented and in benchmarks. * Hooked up analyze ui to backend * Refactored Series Detail metadata area to use a new icon/title design * Cleaned up the new design * Pushing for robbie to do css * Massive performance improvement to scan series where we only need to scan folders reported that have series in them, rather than the whole library. * Removed theme page as we no longer need it. Added WordCount to DTOs so the UI can show them. Added new pipe to format numbers in compact mode. * Hooked up actual reading time based on user's words per hour * Refactor some magic numbers to consts * Hooked in progress reporting for series word count * Hooked up analyze files * Re-implemented time to read on comics * Removed the word Last Read * Show proper language name instead of iso tag on series detail page. Added some error handling on word count code. * Reworked error handling * Fixed some security vulnerabilities in npm. * Handle a case where there are no text nodes and instead of returning an empty list, htmlagilitypack returns null. * Tweaked the styles a bit on the icon-and-title * Code cleanup Co-authored-by: Robbie Davis <robbie@therobbiedavis.com> |