15 Commits

Author SHA1 Message Date
Joe Milazzo
fd6ee42f5f
Reading List Polish (#1879)
* Use Reading Order to count epub pages rather than raw HTML files.

* Send email on background thread for initial invite flow.

* Reorder default writing style for new users so Horizontal is default

* Changed reading activity to use average hours read rather than events to bring more meaningful data.

* added ability to start reading incognito from the top of series detail, needs a bit of styling help though.

* Refactored extensions out into their own package, added new fields for reading list to cover total run, cbl import now takes those dates and overrides on import. Replaced many instances of numbers to be comma separated.

* Added ability to edit reading list run start and end year/month. Refactored some code for valid month/year into a helper method.

* Added a way to see the reading list's release years.

* Added some merged image code, but had to remove due to cover dimensions not fixed.

* tweaked style for accessibility mode on reading list items

* Tweaked css for non virtualized and virtualized containers

* Fixed release updates failing

* Commented out the merge code.

* Typo on words read per year

* Fixed unit tests

* Fixed virtualized scroll

* Cleanup CSS
2023-03-14 06:45:22 -07:00
Joe Milazzo
b55d9e3994
CBL Import Rework (#1862)
* Fixed a typo in a log

* Invalid XML files now "validate" correctly by sending back a failure.

* Cleaned up messaging on backend and frontend to provide some linking on series name when collision, handle corrupt xml files, etc.

* When reading list conflict occurs, show the reading list name that's conflicting. Started refactoring the code to allow multiple files to be imported at once.

* Started adding new CBL elements for some enhancements I have planned with maintainers.

* Default to empty string for IpAddress to allow to fallback into existing experience

* Tweaked the layout of reading list page (not complete), moved some not used much controls to page extras and reordered the buttons for reading list

* Edit Reading Lists now allows selection of cover image from existing items

* Fixed a bug where cover chooser base64 to image would fail to write webp files.

* Refactored the validate step to now handle multiple files in one go.

* Clean up code

* Don't show CBL name if there were xml errors that prevented showing it

* Don't allow user to go prev step after they perform the import.

* Cleaned up the heading code for accordions

* Fixed a bug with import keeping failed items

* Sort the failures to the bottom of result windows

* CBL import is pretty solid. Need one pass from Robbie on Reading List Page
2023-03-07 13:18:26 -08:00
Joe Milazzo
5d1dd7b3f0
.NET 7 + Spring Cleaning (#1677)
* Updated to net7.0

* Updated GA to .net 7

* Updated System.IO.Abstractions to use New factory.

* Converted Regex into SourceGenerator in Parser.

* Updated more regex to source generators.

* Enabled Nullability and more regex changes throughout codebase.

* Parser is 100% GeneratedRegexified

* Lots of nullability code

* Enabled nullability for all repositories.

* Fixed another unit test

* Refactored some code around and took care of some todos.

* Updating code for nullability and cleaning up methods that aren't used anymore. Refctored all uses of Parser.Normalize() to use new extension

* More nullability exercises. 500 warnings to go.

* Fixed a bug where custom file uploads for entities wouldn't save in webP.

* Nullability is done for all DTOs

* Fixed all unit tests and nullability for the project. Only OPDS is left which will be done with an upcoming OPDS enhancement.

* Use localization in book service after validating

* Code smells

* Switched to preview build of swashbuckle for .net7 support

* Fixed up merge issues

* Disable emulate comic book when on single page reader

* Fixed a regression where double page renderer wouldn't layout the images correctly

* Updated to swashbuckle which support .net 7

* Fixed a bad GA action

* Some code cleanup

* More code smells

* Took care of most of nullable issues

* Fixed a broken test due to having more than one test run in parallel

* I'm really not sure why the unit tests are failing or are so extremely slow on .net 7

* Updated all dependencies

* Fixed up build and removed hardcoded framework from build scripts. (this merge removes Regex Source generators). Unit tests are completely busted.

* Unit tests and code cleanup. Needs shakeout now.

* Adjusted Series model since a few fields are not-nullable. Removed dead imports on the project.

* Refactored to use Builder pattern for all unit tests.

* Switched nullability down to warnings. It wasn't possible to switch due to constraint issues in DB Migration.
2023-03-05 12:55:13 -08:00
Joe Milazzo
d88a4d5d0c
CBL Import (#1834)
* Wrote my own step tracker and added a prev button. Works up to first conflict flow.

* Everything but final import is hooked up in the UI. Polish still needed, but getting there.

* Making more progress in the CBL import flow.

* Ready for the last step

* Cleaned up some logic to prepare for the last step and reset

* Users like order to be starting at 1

* Fixed a few bugs around cbl import

* CBL import is ready for some basic testing

* Added a reading list hook on side nav

* Fixed up unit tests

* Added icons and color to the import flow

* Tweaked some phrasing

* Hooked up a loading variable but disabled the component as it didn't look good.

* Styling it up

* changed an icon to better fit

---------

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2023-03-03 14:51:11 -08:00
Joe Milazzo
3f24dc7392
Reading List Fixes (#1784)
* Add ability to save readinglist comicinfo fields in Chapter.

* Added the appropriate fields and migration for Reading List generation.

* Started the reading list code

* Started building out the CBL import code with some initial unit tests.

* Fixed first unit test

* Started refactoring control code into services and writing unit tests for ReadingLists. Found a logic issue around reading list title between create/update. Will be corrected in this branch with unit tests.

* Can't figure out how to mock UserManager, so had to uncomment a few tests.

* Tooltip for total pages read shows the full number

* Tweaked the math a bit for average reading per week.

* Fixed up the reading list unit tests. Fixed an issue where when inserting chapters into a blank reading list, the initial reading list item would have an order of 1 instead of 0.

* Cleaned up the code to allow the reading list code to be localized easily and fixed up a bug in last PR.

* Fixed a sorting issue on reading activity

* Tweaked the code around reading list actionables not showing due to some weird filter.

* Fixed edit library settings not opening on library detail page

* Fixed a bug where reading activity dates would be out of order due to a bug in how charts works. A temp hack has been added.

* Disable promotion in edit reading list modal since non-admins can (and should have) been able to use it.

* Fixed a bug where non-admins couldn't update their OWN reading lists. Made uploading a cover image for readinglists now check against the user's reading list access to allow non-admin's to set images.

* Fixed an issue introduced earlier in PR where adding chapters to reading list could cause order to get skewed.

* Fixed another regression from earlier commit

* Hooked in Import CBL flow. No functionality yet.

* Code is a mess. Shifting how the whole import process is going to be done. Commiting so I can pivot drastically.

* Very rough code for first step is done.

* Ui has started, I've run out of steam for this feature.

* Cleaned up the UI code a bit to make the step tracker nature easier without a dedicated component.

* Much flow implementation and tweaking to how validation checks and what is sent back.

* Removed import via cbl code as it's not done. Pushing to next release.
2023-02-12 08:20:51 -08:00
Joe Milazzo
7616eb5b0f
UTC Dates + CDisplayEx API Enhancements (#1781)
* Introduced a new claim on the Token to get UserId as well as Username, thus allowing for many places of reduced DB calls. All users will need to reauthenticate.

Introduced UTC Dates throughout the application, they are not exposed in all DTOs, that will come later when we fully switch over. For now, Utc dates will be updated along side timezone specific dates.

Refactored get-progress/progress api to be 50% faster by reducing how much data is loaded from the query.

* Speed up the following apis:
collection/search, download/bookmarks, reader/bookmark-info, recommended/quick-reads, recommended/quick-catchup-reads, recommended/highly-rated, recommended/more-in, recommended/rediscover, want-to-read/

* Added a migration to sync all dates with their new UTC counterpart.

* Added LastReadingProgressUtc onto ChapterDto for some browsing apis, but not all.

Added LastReadingProgressUtc to reading list items.

Refactored the migration to run raw SQL which is much faster.

* Added LastReadingProgressUtc onto ChapterDto for some browsing apis, but not all.

Added LastReadingProgressUtc to reading list items.

Refactored the migration to run raw SQL which is much faster.

* Fixed the unit tests

* Fixed an issue with auto mapper which was causing progress page number to not get sent to UI

* series/volume has chapter last reading progress

* Added filesize and library name on reading list item dto for CDisplayEx.

* Some minor code cleanup

* Forgot to fill a field
2023-02-11 04:01:24 -08:00
Joe Milazzo
1a729adf40
OPDS Enhancements (#1687)
* Bump express from 4.17.2 to 4.18.2 in /UI/Web

Bumps [express](https://github.com/expressjs/express) from 4.17.2 to 4.18.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.17.2...4.18.2)

---
updated-dependencies:
- dependency-name: express
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump decode-uri-component from 0.2.0 to 0.2.2 in /UI/Web

Bumps [decode-uri-component](https://github.com/SamVerschueren/decode-uri-component) from 0.2.0 to 0.2.2.
- [Release notes](https://github.com/SamVerschueren/decode-uri-component/releases)
- [Commits](https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.0...v0.2.2)

---
updated-dependencies:
- dependency-name: decode-uri-component
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump qs and express in /UI/Web

Bumps [qs](https://github.com/ljharb/qs) and [express](https://github.com/expressjs/express). These dependencies needed to be updated together.

Updates `qs` from 6.5.3 to 6.11.0
- [Release notes](https://github.com/ljharb/qs/releases)
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.5.3...v6.11.0)

Updates `express` from 4.17.2 to 4.18.2
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.17.2...4.18.2)

---
updated-dependencies:
- dependency-name: qs
  dependency-type: indirect
- dependency-name: express
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Added genre and authors to Series level, added summary to volume and chapter level.

Force order on reading list title as Chunky enforces their own sort order and doesn't respect the spec.

* Moved all the reading list formatting logic to the backend. This allows us to re-use the UI logic for OPDS streams.

* Fixed a broken unit test

* Code smells

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-10 09:07:55 -08:00
Joe Milazzo
089658e469
UX Alignment and bugfixes (#1663)
* Refactored the design of reading list page to follow more in line with list view. Added release date on the reading list items, if it's set in underlying chapter.

Fixed a bug where reordering the list items could sometimes not update correctly with drag and drop.

* Removed a bug marker that I just fixed

* When generating library covers, make them much smaller as they are only ever icons.

* Fixed library settings not showing the correct image.

* Fixed a bug where duplicate collection tags could be created.

Fixed a bug where collection tag normalized title was being set to uppercase.

Redesigned the edit collection tag modal to align with new library settings and provide inline name checks.

* Updated edit reading list modal to align with new library settings modal pattern. Refactored the backend to ensure it flows correctly without allowing duplicate names.

Don't show Continue point on series detail if the whole series is read.

* Added some more unit tests around continue point

* Fixed a bug on series detail when bulk selecting between volume and chapters, the code which determines which chapters are selected didn't take into account mixed layout for Storyline tab.

* Refactored to generate an OpenAPI spec at root of Kavita. This will be loaded by a new API site for easy hosting.

Deprecated EnableSwaggerUi preference as after validation new system works, this will be removed and instances can use our hosting to hit their server (or run a debug build).

* Test GA

* Reverted GA and instead do it in the build step. This will just force developers to commit it in.

* GA please work

* Removed redundant steps from test since build already does it.

* Try another GA

* Moved all test actions into initial build step, which should drastically cut down on time. Only run sonar if the secret is present (so not for forks). Updated build requirements for develop and stable docker pushes.

* Fixed env variable

* Okay not possible to do secrets in if statement

* Fixed the build step to output the openapi.json where it's expected.
2022-11-20 12:32:21 -08:00
Joe Milazzo
38a169818b
Bookmark and Reader bugs (#1632)
* Updated swiper and some packages for reported security issues

* Fixed reading lists promotion not working

* Refactor RenameFileForCopy to use iterative recursion, rather than functional.

* Ensured that bookmarks are fetched and ordered by Created date.

* Fixed a bug where bookmarks were coming back in the correct order, but due to filenames, would not sort correctly.

* Default installs to Debug log level given errors users have and Debug not being too noisy

* Added jumpbar to bookmarks page

* Now added jumpbar to bookmarks

* Refactored some code into pipes and added some debug messaging for prefetcher

* Try loading next and prev chapter's first/last page to cache so it renders faster

* Updated GetImage to do a bound check on max page.

Fixed a critical bug in how manga reader updates image elements src to prefetch/load pages. I was not creating a new reference which broke Angular's ability to update DOM on changes.

* Refactored the image setting code to use a single method which tries to use a cached image always.

* Refactored code to use getPage which favors cache and simplifies image creation code
2022-11-02 20:10:19 -05:00
Joseph Milazzo
d1a14f7e68
Logging Enhancements (#1521)
* Recreated Kavita Logging with Serilog instead of Default. This needs to be move out of the appsettings now, to allow auto updater to patch.

* Refactored the code to be completely configured via Code rather than appsettings.json. This is a required step for Auto Updating.

* Added in the ability to send logs directly to the UI only for users on the log route. Stopping implementation as Alerts page will handle the rest of the implementation.

* Fixed up the backup service to not rely on Config from appsettings.json

* Tweaked the Logging levels available

* Moved everything over to File-scoped namespaces

* Moved everything over to File-scoped namespaces

* Code cleanup, removed an old migration and changed so debug logging doesn't print sensitive db data

* Removed dead code
2022-09-12 17:25:48 -07:00
Joseph Milazzo
8edd5e476a
Misc UI Fixes (#1477)
* Tweaked a Migration to log correctly only if something is going to be done.

* Refactored Reading List Controller code into a dedicated service and cleaned up some methods that aren't needed anymore.

* Fixed a bug where adding a new item to a reading list wasn't adding it at the end.

* Fixed an issue where collection page would re-render the same covers on multiple items.

* Fixed a missing margin-top which made the page extras drawer not render correctly and hence unclosable on small screens.

* Added some timeout on manage users screen to give data time to flush.

Added a dedicated token log for account flows, in case url encoding plays a part (but from testing it doesn't).

* Reverted back to building for ES6 instead of es2020 for old Safari 12.5.5 browsers (10MB difference in build size).

* Cleaned up the logic in removing series not found during scan loop.

* Tweaked the timings for Library Watcher to 1 min and reprocess queue every 30 seconds.
2022-08-25 09:34:48 -07:00
Joseph Milazzo
f54eb5865b
All Around Polish (#1328)
* Added --card-list-item-bg-color for the card list items

* Updated the card list item progress to match how cards render

* Implemented the ability to configure how many backups are retained.

* Fixed a bug where odd jump keys could cause a bad index error for jump bar

* Commented out more code for the pagination route if we go with that.

* Reverted a move of DisableConcurrentExecution to interface, as it seems to not work there.

* Updated manga format utility code to pipes

* Fixed bulk selection on series detail page

* Fixed bulk selection on all other pages

* Changed card item to OnPush

* Updated image component to OnPush

* Updated Series Card to OnPush

* Updated Series Detail to OnPush

* Lots of changes here. Integrated parentscroll support on card detail layout. Added jump bar (custom js implementation) on collection, reading list and all series pages. Updated UserParams to default to no pagination. Lots of cleanup all around

* Updated some notes on a module use

* Some code cleanup

* Fixed up a broken test due to the mapper not being configured in the test.

* Applied TabID pattern to edit collection tags

* Applied css from series detail to collection detail page to remove double scrollbar

* Implemented the ability to sort by Time To Read.

* Throw an error to the UI when we extract an archive and it contains invalid characters in the filename for the Server OS.

* Tweaked how the page scrolls for jumpbar on collection detail. We will have to polish another release

* Cleaned up the styling on directory picker

* Put some code in but it doesn't work for scroll to top on virtual scrolling. I'll do it later.

* Fixed a container bug
2022-06-22 10:25:52 -07:00
Joseph Milazzo
3bbb02f574
On Deck + Misc Fixes and Changes (#1215)
* Added playwright and started writing e2e tests.

* To make things easy, disabled other browsers while I get confortable. Added a login flow (assumes my dev env)

* More tests on login page

* Lots more testing code, trying to figure out auth code.

* Ensure we don't track DBs inside config

* Added a new date property for when chapters are added to a series which helps with OnDeck calculations. Changed a lot of heavy api calls to use IEnumerable to stream repsonse to UI.

* Fixed OnDeck with a new field for when last chapter was added on Series. This is a streamlined way to query.

Updated Reading List with NormalizedTitle, CoverImage, CoverImageLocked.

* Implemented the ability to read a random item in the reading list and for the reading list to be intact for order.

* Tweaked the style for webtoon to not span the whole width, but use max width

* When we update a cover image just send an event so we don't need to have logic for when updates occur

* Fixed a bad name for entity type on cover updates

* Aligned the edit collection tag modal to align with new tab design

* Rewrote code for picking the first file for metadata to ensure it always picks the correct file, esp if the first chapter of a series starts with a float (1.1)

* Refactored setting LastChapterAdded to ensure we do it on the Series.

* Updated Chapter updating in scan loop to avoid nested for loop and an additional loop.

* Fixed a bug where locked person fields wouldn't persist between scans.

* Updated Contributing to reflect how to view the swagger api
2022-04-11 15:43:40 -07:00
Joseph Milazzo
f5229fd0e6
Bulk Operations (#596)
* Implemented the ability to perform multi-selections on cards. Basic selection code is done, CSS needed and exposing actions.

* Implemented a bulk selection bar. Added logic to the card on when to force show checkboxes.

* Fixed some bad parsing groups and cases for Comic Chapters.

* Hooked up some bulk actions on series detail page. Not hooked up to backend yet.

* Fixes #593. URI Enocde library names as sometimes they can have & in them.

* Implemented the ability to mark volume/chapters as read/unread.

* Hooked up mark as unread with specials as well.

* Add to reading list hooked up for Series Detail

* Implemented ability to add multiple series to a reading list.

* Implemented bulk selection for series cards

* Added comments to the new code in ReaderService.cs

* Implemented proper styling on bulk operation bar and integrated for collections.

* Fixed an issue with shift clicking

* Cleaned up css of bulk operations bar

* Code cleanup
2021-09-24 17:27:47 -07:00
Joseph Milazzo
cf7a9aa71e
Reading Lists & More (#564)
* Added continous reading to the book reader. Clicking on the max pages to right of progress bar will now go to last page.

* Forgot a file for continous book reading

* Fixed up some code regarding transitioning between chapters. Arrows now show to represent a chapter transition.

* Laid the foundation for reading lists

* All foundation is laid out. Actions are wired in the UI. Backend repository is setup. Redid the migration to have ReadingList track modification so we can order them for the user.

* Updated add modal to have basic skeleton

* Hooked up ability to fetch reading lists from backend

* Made a huge performance improvement to GetChapterIdsForSeriesAsync() by reducing a JOIN and an iteration loop. Improvement went from 2 seconds -> 200 ms.

* Implemented the ability to add all chapters in a series to a reading list.

* Fixed issue with adding new items to reading list not being in a logical order. Lots of work on getting all the information around the reading list view. Added some foreign keys back to chapter so delete should clean up after itself.

* Added ability to open directly the series

* Reading List Items now have progress attached

* Hooked up list deletion and added a case where if doesn't exist on load, then redirect to library.

* Lots of changes. Introduced a dashboard component for the main app. This will sit on libraries route for now and will have 3 tabs to show different sections.

Moved libraries reel down to bottom as people are more likely to access recently added or in progress than explore their whole library.

Note: Bundles are messed up, they need to be reoptimized and routes need to be updated.

* Added pagination to the reading lists api and implemented a page to show all lists

* Cleaned up old code from all-collections component so now it only handles all collections and doesn't have the old code for an individual collection

* Hooked in actions and navigation on reading lists

* When the user re-arranges items, they are now persisted

* Implemented remove read, but performance is pretty poor. Needs to be optimized.

* Lots of API fixes for adding items to a series, returning items, etc. Committing before fixing incorrect fetches of items for a readingListId.

* Rewrote the joins for GetReadingListItemDtosByIdAsync() to not return extra records.

* Remove bug marker now that it is fixed

* Refactor update-by-series to move more of the code to a re-usable function for update-by-volume/chapter APIs

* Implemented the ability to add via series, volume or chapter.

* Added OPDS support for reading lists. This included adding VolumeId to the ReadingListDto.

* Fixed a bug with deleting items

* After we create a library inform user that a scan has started

* Added some extra help information for users on directory picker, since linux users were getting confused.

* Setup for the reading functionality

* Fixed an issue where opening the edit series modal and pressing save without doing anything would empty collection tags. Would happen often when editing cover images.

* Fixed get-next-chapter for reading list. Refactored all methods to use the new GetUserIdByUsernameAsync(), which is much faster and uses less memory.

* Hooked in prev chapter for continuous reading with reading list

* Hooked up the read code for manga reader and book reader to have list id passed

* Manga reader now functions completely with reading lists

* Implemented reading list and incognito mode into book reader

* Refactored some common reading code into reader service

* Added support for "Series -  - Vol. 03 Ch. 023.5 - Volume 3 Extras.cbz" format that can occur with FMD2.

* Implemented continuous reading with a reading list between different readers. This incurs a 3x performance hit on the book info api.

* style changes. Don't emit an event if position of draggable item hasn't changed

* Styling and added the edit reading list flow.

* Cleaned up some extra spaces when actionables isn't shown. Lots of cleanup for promoted lists.

* Refactored some filter code to a common service

* Added an RBS check in getting Items for a given user.

* Code smells

* More smells
2021-09-08 12:03:27 -05:00