371 Commits

Author SHA1 Message Date
Joseph Milazzo
073dd5da99
UI Tweaks (#722)
* Added Date Added time to volume/chapter card detail modals to help users understand when new chapters are added after something has been fully read. Changed Created -> Added for better clarity.

* updating workflow discord parser

- Added ` and > to parser

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2021-11-04 19:48:23 -07:00
Joseph Milazzo
18c2cdf6ef
Fixed a migration issue on docker happening too many times or throwing exception when source wasn't there. (#719) 2021-11-03 17:23:46 -07:00
Joseph Milazzo
d143330e97
Shakeout Changes (#717)
* Make the appsettings public on Configuration and change how we detect when to migrate for non-docker users.

* Fixed up non-docker copy command and removed duplicate check on source directory for a copy.

* Don't delete files unless we know we are successful
2021-11-03 12:13:38 -07:00
Joseph Milazzo
44f15f25c6
Breaking Changes: Docker Parity (#715)
* Fixed a bug in the copy directory to directory in the migration

* Somehow GetFiles lost static modifier.
2021-11-03 07:43:03 -07:00
Joseph Milazzo
7bfd2c432d
Feature/docker parity (#714)
* Refactored all the config files for Kavita to be loaded from config/. This will allow docker to just mount one folder and for Update functionality to be trivial.

* Cleaned up documentation around new update method.

* Updated docker files to support config directory

* Removed entrypoint, no longer needed

* Update appsettings to point to config directory for logs

* Updated message for docker users that are upgrading

* Ensure that docker users that have not updated their mount points from upgrade cannot start the server

* Code smells

* More cleanup

* Added entrypoint to fix bind mount issues

* Updated README with new folder structure

* Fixed build system for new setup

* Updated string path if user is docker

* Updated the migration flow for docker to work properly and Fixed LogFile configuration updating.

* Migrating docker images is now working 100%

* Fixed config from bad code

* Code cleanup

* Fixed monorepo-build.sh

Co-authored-by: Chris Plaatjes <kizaing@gmail.com>
2021-11-03 07:07:04 -07:00
Joseph Milazzo
a29b11c366
Breaking Changes: Docker Parity (#698)
* Refactored all the config files for Kavita to be loaded from config/. This will allow docker to just mount one folder and for Update functionality to be trivial.

* Cleaned up documentation around new update method.

* Updated docker files to support config directory

* Removed entrypoint, no longer needed

* Update appsettings to point to config directory for logs

* Updated message for docker users that are upgrading

* Ensure that docker users that have not updated their mount points from upgrade cannot start the server

* Code smells

* More cleanup

* Added entrypoint to fix bind mount issues

* Updated README with new folder structure

* Fixed build system for new setup

* Updated string path if user is docker

* Updated the migration flow for docker to work properly and Fixed LogFile configuration updating.

* Migrating docker images is now working 100%

* Fixed config from bad code

* Code cleanup

Co-authored-by: Chris Plaatjes <kizaing@gmail.com>
2021-11-03 06:36:04 -07:00
Joseph Milazzo
7ce13babad
Vacation Fixes (#709)
* Ignore system and hidden folders when performing directory scan.

* Fixed the comic parser tests not using Comic mode for parsing.

* Accept all forwarded headers and use them.

* Ignore some changes from another branch
2021-11-01 11:34:43 -07:00
Joseph Milazzo
49d1021049
Cover Image Picking + Forwarding Headers with EPUBs (#700)
* Ensure Kavita knows about forwarding headers (fixes issue with epub urls not going through https with reverse proxy). Fixed a case where cover image selection preferred nested folders vs files in root directory.

* Fixed broken unit test

* Added bug that I fixed to the unit tests
2021-10-21 14:51:14 -05:00
Joseph Milazzo
6d6eee999a
Bulk Deletion (#697)
* Implemented bulk deletion of series

* Don't show unauthorized exception on UI, just redirect to the login page.
2021-10-20 10:49:58 -07:00
Joseph Milazzo
88214c5c6b
KavitaStats Cleanup (#695)
* Refactored Stats code to be much cleaner and user better naming.

* Cleaned up the actual http code to use Flurl and to return if the upload was successful or not so we can delete the file where appropriate.

* More refactoring for the stats code to clean it up and keep it consistent with our standards.

* Removed a confusing log statement

* Added support for old api key header from original stat server

* Use the correct endpoint, not the new one.

* Code smell
2021-10-20 10:03:51 -07:00
Joseph Milazzo
c813d55124
Fallback to Folder Parsing Issue (#694)
* Fixed a bug in the scanner where we fall back to parsing from folders for poorly named files. The code was exiting early if a chapter or volume could be parsed out.

* Fixed a unit test by tweaking a regex for fallback
2021-10-20 06:29:05 -07:00
Joseph Milazzo
6fb01eefa4
Fixed an issue where docker users were not properly backing up the database. Removed an empty File for when covers/ had nothing in it. (#692) 2021-10-19 08:35:52 -07:00
Joseph Milazzo
8e3121447d
More EPUB Scoping Fixes (#691)
* Added better handling around when importing css files that are empty. Moved comment removal on css files to before some css whitespace cleanup to get better matches.

* Some enhancements on the checks to see if we need the bottom action bar on reader. Now we don't query DOM and have something that works more reliably.
2021-10-19 06:12:49 -07:00
Joseph Milazzo
60dd66f6ae
EPUB CSS Parsing Issues (#690)
* WIP. Rewrote some of the Regex to better support css escaping. We now escape background-image, border-image, and list-style-image within css files.

* Added position relative to help with positioning on books that are just absolute positioned elements.

* When there is absolute positioning, like in some epub based comics, supress the bottom action bar since it wont render in the correct location.

* Fixed tests

* Commented out tests
2021-10-18 16:28:07 -07:00
Joseph Milazzo
0666f87b4b
Stat upload will now schedule itself between midnight and 6am in server time for upload. (#688) 2021-10-18 13:30:58 -07:00
Joseph Milazzo
6adf092668
When the DB fails to save, log out all the series the user should look into for constraint issues and push a message to the admins connected to webui. (#687) 2021-10-18 08:45:39 -07:00
Joseph Milazzo
26d1fbc599
When we have a special marker, ensure we fall back to folder parsing to try and group correctly to the actual series before just accepting what we parsed. (#684)
Fixed a missed parsing case where comic special parsing wasn't being called on comic libraries.
2021-10-17 12:44:30 -07:00
Joseph Milazzo
f3ebc21b97
Fixed a bug where searching on localized name would fail to show on the search. Fixed a bug where extra spaces would cause the search results not to show properly. (#682) 2021-10-17 07:08:17 -07:00
Joseph Milazzo
1d80420155
Change chunk size to be a fixed 50 to validate if it's causing issue with refresh. Added some try catches to see if exceptions are causing issues. (#681) 2021-10-16 11:50:34 -07:00
Joseph Milazzo
d5d03528d4
Regression Fix (#680)
* Ensure we mount the backups directory for Docker users

* Fixed a huge logic bug that deleted files in users libraries
2021-10-16 11:24:30 -07:00
Joseph Milazzo
98e5cb87ed
Bulk Operations for In Progress and Recently Added (#677)
* Don't log a message about bad match if the file is a cover image

* Enable bulk operations for In Progress and Recently Added

* Fixed a bad logic case
2021-10-15 14:32:17 -07:00
Joseph Milazzo
52f8fbe3db
Bulk Add to Collection (#674)
* Fixed the typeahead not having the same size input box as other inputs

* Implemented the ability to add multiple series to a collection through bulk operations flow. Updated book parser to handle "@import url('...');" syntax as well as @import '...';

* Implemented the ability to create a new Collection tag via bulk operations flow.
2021-10-14 17:23:21 -07:00
YEGCSharpDev
98b3c483be Revert "Changed the stats that are sent back to stat server from installed server."
This reverts commit 644cb6d1f67de9531ea1a1dfd3853709e0329ce7.
2021-10-14 16:00:01 -06:00
YEGCSharpDev
644cb6d1f6 Changed the stats that are sent back to stat server from installed server. 2021-10-14 15:56:38 -06:00
Joseph Milazzo
355f50fd5d
Logging Cleanup (#668)
* Do not allow non-admins to change their passwords when authentication is disabled

* Clean up the login page so that input field text is black

* cleanup some resizing when typing a password and having a lot of users

* Changed the LastActive for a user to not just be login, but also when they open an already authenticated session.

* Removed some verbose debugging statements and moved some debug to information to be more prevelant to logs for default installs.

* In Progress now sends progress information on the Series

* Add ability to add cards to recently added when new series are added in backend

* Implemented the ability to click the glasses icon to turn off incognito mode from within the reader so you can start tracking progress

* Don't warn the user about authentication when they don't touch that control
2021-10-14 08:14:03 -07:00
Joseph Milazzo
70f324669b
Misc Updates (#665)
* Do not allow non-admins to change their passwords when authentication is disabled

* Clean up the login page so that input field text is black

* cleanup some resizing when typing a password and having a lot of users

* Changed the LastActive for a user to not just be login, but also when they open an already authenticated session.
2021-10-13 11:13:55 -07:00
Joseph Milazzo
b197f6f334
Release Shakeout (#655)
* Cleaned up some code. Fixed an issue on books with good table of contents not allowing line tracking (progress) from being saved. Changed Save to Defaults on light mode to be primary.

* Fixed a bug where deleting reading items would not actually delete them

* Fixed a bug where after ordering reading lists then deleting the order would be undone (develop)

* Code cleanup
2021-10-11 16:32:56 -07:00
Christoffer Green
12f96a5504
Improve comic-downloader parsing (#650)
Sometimes comic-downloader will have filenames like spawn-chapter-123.
Previously we did not support the -chapter- part, this patch adds support
for it.
2021-10-11 08:17:37 -07:00
Joseph Milazzo
06508fd909
Remove Base Url Support (#652)
* Fixed some issues with base url. All Scheduled jobs are now in user's timezone

* Hide Base Url support from UI and removed some code around it on the backend to prevent it from interfering. Patched back in changes from base-href branch like Timezone on scheduled jobs and enhanced logging.

* Added parser support for "2000 AD 0366 [1984-04-28] (flopbie)" and removed for "01 Spiderman 01".
2021-10-11 06:57:50 -07:00
Robbie Davis
3293e5b424
Comic enhancements (#645)
* Adding multiple cases for comic naming conventions

* Changing "Chapter" to "Issue" for comic libraries

* Fixed an issue where the Parse method was using filename with extension to run regex matching, while it should be running on name without extension.

* Refactored to use Getter

* Cleaned up file to use conditional labelling rather than conditional html fragments

* Refactored code to properly check against library type for a given readinglist item

* Cleaned up series detail

* Conditionally remove special tags during parse

* Setup ParseInfoTests for ComicParserTests and also added unit tests from other comic issues created.

* Added more regex cases for naming patterns reported to be common with comics. Some cases added without regex.

* Pushing up changes

Fixed issue with cleanTitleTest.
Tried some patterns for "Cyberpunk 2077" but reverted

* Updated some cases and some spacing on Parser. Cyberpunk 2077 is not implemented as long as there is a # before issue number.

* Fixed the case for Special parsing on TPB. Fixed a piece of code that got deleted that prevented specials from rendering on volumes tab.

* Potential fix for parsing Cyberpunk 2077

- Added a ComicsSeriesSpecialCasesRegex and passed any filename that contains "Cyberpunk 2077" over to it so we can parse it separately. This could be used for any other potential problem series.

* Revert "Potential fix for parsing Cyberpunk 2077"

This reverts commit a14417e640ddb7ab27f66bcc27ff5ecc41581b25.

* Added more tests

* Refactored all places in Kavita to use Book, Issue, or Chapter depending on the Library type. Updated Volumes/Chapters to remove Volumes to make it cleaner.

* Removed some leftover test code

Co-authored-by: Joseph Milazzo <joseph.v.milazzo@gmail.com>
2021-10-07 07:49:13 -07:00
Joseph Milazzo
977e364d5a
Moved BaseUrl from appsettings.json to Database and fixed an issue in UI for setting base url based on a hack, rather than asking backend for it. (#644) 2021-10-06 11:03:14 -07:00
Joseph Milazzo
27aaa040c4
Base Url Support (#642)
* Added base url config

* UI side is not working

* Working base url more

* Attempt to get UI to work with base url

* Implemented the ability to set the Base URL for the app

* Hooked in Base URL as a managed setting

* Ensure we always start with / for base url

* Removed default base href from debug builds. Cleaned up an issue with base url migration.

* Fixed an issue with our BaseURL migration
2021-10-06 05:42:02 -07:00
Joseph Milazzo
2727dfa7b9
Book Polishing (#639)
* Downgraded ExCSS, the new version has a regression that breaks toCss().

* Cleaned up the next/prev disabling to work more reliably.

Fixed an issue with double arrow on next not disappearing after hitting last page then going back one page.

* Cleaned up some css padding on the side nav progress bar

* Remove the ability to save

* Updated colors of nav bar and side drawer to be darker to separate them from the reader

* Fixed a missed element

* Removed dead code

* Removed TODO, it can't be changed

* Refactored bookmarking on the book progress to have pinpoint accuracy for remembering scroll position. Now will choose the top line on the page.

* Fixed a style issue when applying dark mode from light mode in book reader
2021-10-05 17:42:43 -07:00
Joseph Milazzo
2183bf2d59
Misc Bugfixes (#637)
* Use null coalescing for comicinfo

* Removed Sentry. The integrations were not providing any userful information.

* Updated to the latest versions of dependencies
2021-10-04 17:18:42 -07:00
Joseph Milazzo
f17d89ea47
ComicInfo Refactor (#636)
* Implemented methods to parse out the whole ComicInfo file and a mock ComicInfo from epub files.

* Removed unused imports. ScanSeries needs to cleanup deleted chapters and call RefreshMetadata. Ensure after scan we cleanup tags without any series.

* Random cleanup

* Added some comments about data getting stale and not updating.

* Removed old Summary methods in favor of the ComicInfo versions

* Added a missing property

* Fixed unit test
2021-10-04 16:10:48 -07:00
Joseph Milazzo
c5b62d00d0
Hashing (#634)
* Added the ability to see when a scan started at.

* filename based hashing now uses last write time as well to ensure if the underlying file changes it sends a new copy

* Fixed a bug where we would reset dark mode on the book reader to dark mode if our site was on dark mode, despite user setting light mode.

* Added a single feed entry when some sort of collection, reading list, etc doesn't have anything in it.

* Allow + in the normalization to prevent some series that use + to denote the sequel from not getting merged together.
2021-10-04 05:33:28 -07:00
Joseph Milazzo
3907414ae4
Shakeout Part 2 (#630)
* When flattening directories, ensure the order or directories being enumerated follows a natural sort. Some users are discovering directories in a different order than other machines.

* Added a case for volume parsing and fixed a poorly designed negative lookahead.

Added a sentence case pipe for formatting things.

Added time for all dates.

* Some more sentence case

* Register user now has a white input

* Fixed an issue with Manga up/down reading mode where top of the page was going forwards, when it should have gone backwards

* Reworked some code to ensure that scanseries doesn't show errors where in fact there was just nothing to update.

* Last updated should be working as intended for new library flow.

* Code smell
2021-10-03 15:40:22 -07:00
Joseph Milazzo
6e85fe8c0a
Interlude/Half Volumes (#626)
* Refactored Parser to handle parts

* Fixed a bug where marking multiple entities as unread would actually make them look read on the UI

* Implemented the ability to have float volume numbers

* Removed two unit test cases

* Code smells
2021-10-03 13:30:31 -07:00
Christoffer Green
63d7439938
Add comics-downloader parser (#611)
* Add comics-downloader parser

The utility comics-downloader can be used to download comics
and produces files of the type "seriesname-chapternumberpadded".
This adds support for that format.

For more info https://github.com/Girbons/comics-downloader

* Adjusted the test cases and added .+? to consume less characters.

Co-authored-by: Joseph Milazzo <joseph.v.milazzo@gmail.com>
2021-10-03 05:42:40 -07:00
Joseph Milazzo
d750ce77a0
Shakeout Bugs (#623)
* Fixed an issue where ScanSeries would not fetch all the entities and thus files would get duplicated on the Chapter

* Remove building extra language binaries on build.

* Fixed an issue where first scan would cause an issue with websocket due to trying to send NaN over the wire.

* Fixed an issue where on new scans scan in progress indicators wouldn't turn off due to the way we were consuming events off the pipe.

* Ensure login page doesn't flash on first load

* Don't process touch events at all unless selection is enabled.
2021-10-03 05:27:35 -07:00
Joseph Milazzo
30a302bf99
Changed how login page works based on user feedback. By default, we have the old login flow when authentication is enabled and in addition, we will auto fill the username based on last login on device. (#621) 2021-10-02 11:35:38 -07:00
Joseph Milazzo
a5b6bf1b52
Disable Authentication & Login Page Rework (#619)
* Implemented the ability to disable authentication on a server instance. Admins will require authentication, but non-admin accounts can be setup without any password requirements.

* WIP for new login page.

* Reworked code to handle disabled auth better. First time user flow is moved into the user login component.

* Removed debug code

* Removed home component, shakeout testing is complete.

* remove a file accidently committed

* Fixed a code smell from last PR

* Code smells
2021-10-02 11:23:58 -05:00
Joseph Milazzo
6d86008607
Cleanup from last develop release (#614)
* Added extra information for Swagger Gen

* Ensure that the last scanned time gets updated after a scan finishes.

Remove title case from Library name UI display.

Show time on last scanned

* Don't force title case for library names. Show them as the user created them.
2021-10-01 15:00:51 -07:00
Joseph Milazzo
ce2f4b6a9f
Polishing and Bugfixes (#613)
* Added a new field to Library for showing Last Scan time for a library.

Manage library page now shows last scan for each library. Tweaked the websocket code to make scan progress a bit more reliable.

* Updated docnet to provide ARM support from our recent PR. Pi users can now have a version for PDF support out of the box.

* Parser is now culture invariant to hopefully fix an issue on Italian systems not detecting . correctly

* Added the ability for the collection detail page to update when a new series is added.

* Fixed an issue where multiple chapters stacked in a volume and reading in incognito, wouldn't sort the chapters and would open the wrong one.

* Code smell
2021-10-01 08:13:55 -07:00
Joseph Milazzo
0ac54e682f
Warn on Refresh Metadata (#610)
* Warn the user about the dangers of refresh metadata and promote them to use a scan instead.

* Removed presence hub and moved it over to message hub.

When a library scan is in progress, now a spinner will show on manage libraries page.

* Code cleanup
2021-09-30 17:36:58 -07:00
Joseph Milazzo
56cf7be799
Scan Chunking (#604)
* Some performance refactoring around getting Library and avoid a byte[] copy for getting cover images for epubs.

* Initial commit. Rewrote the main series scan loop to use chunks of data at a time. Not fully shaken out.

* Hooked in the ability for the UI to react to series being added or removed from the DB.

* Cleaned up the messaging in the scan loop to be more clear.

* Metadata scan and scan work as expected and populate data to the UI. There is a slow down in speed for overall operation.

Scan series and refresh series metadata does not work fully.

* Fixed a bug where MangaFiles were not having LastModified Updated correctly, meaning they were opening archives every scan.

* Modified the code to be more realistic to the underlying file

* Updated ScanService to properly handle deleted files and not result in a higher-level scan.

* Shuffled around volume related repo apis to the volume repo rather than being in series.

* Rewrote scan series to be much cleaner and more concise on the flow. Fixed an issue in UpdateVolumes such that the debug code to log out removed volumes could throw an exception and actually break updating volumes.

* Refactored the code to set MangaFile last modified timestamp into the MangaFile entity.

* Added Series Name to ScanSeries event

* Added additional checks in ScanSeries to ensure we never go outside the library folder.

Added extra debug messages for when a metadata refresh doesn't actually make changes and for when we regen cover images.

* More logging statements saying where they originate from. Fixed a critical bug which caused only 1 chunk to ever be processed.

* Fixed a concurrency issue with natural sorter which could cause issues in ArchiveService.cs.

* Log cleanups

* Fixed an issue with logging out total time of a scan.

* Only show added toastrs for admins. When kicking off a refresh metadata for series, make sure we regenerate all cover images.

* Code smells on benchmark despite it being ignored
2021-09-30 06:08:05 -07:00
Joseph Milazzo
5e2606b0de
Feature/release cleanup (#597)
* Deselect all after bulk operations complete.

* Implemented a way to trigger selection code on mobile.

* When selection mode is active, make the clickable area the whole image.

* Series detail shouldn't use gutters for card layout as it can cause skewing on mobile

* Long press on card items to trigger the selection on mobile

* Code cleanup

* One more

* Misread the code issue
2021-09-26 05:40:52 -07:00
Joseph Milazzo
f5229fd0e6
Bulk Operations (#596)
* Implemented the ability to perform multi-selections on cards. Basic selection code is done, CSS needed and exposing actions.

* Implemented a bulk selection bar. Added logic to the card on when to force show checkboxes.

* Fixed some bad parsing groups and cases for Comic Chapters.

* Hooked up some bulk actions on series detail page. Not hooked up to backend yet.

* Fixes #593. URI Enocde library names as sometimes they can have & in them.

* Implemented the ability to mark volume/chapters as read/unread.

* Hooked up mark as unread with specials as well.

* Add to reading list hooked up for Series Detail

* Implemented ability to add multiple series to a reading list.

* Implemented bulk selection for series cards

* Added comments to the new code in ReaderService.cs

* Implemented proper styling on bulk operation bar and integrated for collections.

* Fixed an issue with shift clicking

* Cleaned up css of bulk operations bar

* Code cleanup
2021-09-24 17:27:47 -07:00
Joseph Milazzo
c44ef6b04d
Additional Memory Pressure Enhancements (#590)
* Added volume migrations. Added parser case for "Chapter 63 - The Promise Made for 520 Cenz.cbr"

* Added some info statements for when full library scans occur. For image apis, return the name of the file to aid in caching.

* When managing users, show the current logged in user at the top of the list. Added a message when no libraries have been setup but you are trying to add a user to a library.

* Removed an extra stream operation from SharpCompress cover image work. Removed an extra ToArray() from Book Reader for extracting PDF pages.

* Removed the left over comment

* Added parsing case for "Batman Beyond 04 (of 6) (1999)"

* Removed dead code
2021-09-22 07:36:32 -07:00
Joseph Milazzo
a872165747
Feature/image rework cleanup (#589)
* Added volume migrations. Added parser case for "Chapter 63 - The Promise Made for 520 Cenz.cbr"

* Added some info statements for when full library scans occur. For image apis, return the name of the file to aid in caching.

* When managing users, show the current logged in user at the top of the list. Added a message when no libraries have been setup but you are trying to add a user to a library.
2021-09-22 06:00:14 -07:00