39 Commits

Author SHA1 Message Date
Joseph Milazzo
6069d93c38
OPDS Support (#526)
* Added some basic OPDS implementation

* Fixed an issue with feed href

* More changes

* Added library routes and moved user code to a method so we can hack in fixed code without authentication

* Images now load on the OPDS reusing our existing Image infrastructure.

* Added the ability to download and moved some download code to a dedicated service

* Download is working, pagination is implemented.

* Refactored libraries to use pagination

* Laid foundation for OpenSearch implementation

* Fixed up some serialization issues and some old code that wasn't referencing helper methods

* Ensure chapters are sorted when we send them over OPDS

* OpenSearch implemented

* Removed any support for OPDS-PS due to lack of apps supporting it.

* Don't distribute development.json nor stats directory on build.

* Implemented In Progress feed as well.

* Ability to enable OPDS for server. OPDS now accepts initial call as POST in case app uses username/password.

* UI now properly renders state for OPDS enablement. Added Collections routes.

* Fixed pagination startIndex on OPDS feeds when there is less than 1 page.

* Chunky Reader now works. It only accepts UTF-8 encodings

* More Chunky fixes

* More chunky changes, such a fussy client.

* Implemented the ability to have a custom api key assigned to a user and use that api key as your authentication token against OPDS routing.

* Implemented the ability to reset your API Key

* Fixed favicon not being sent back correctly

* Fixed an issue where images wouldn't send on OPDS feed.

* Implemented Page streaming and fixed a pagination bug

* Hooked in the ability to save progress in Kavita when Page Streaming
2021-08-27 10:19:25 -07:00
Joseph Milazzo
3ed99afd32
Misc Bugfixes and Enhancements (#507)
* Removed some extra spam for the console

* Implemented the code to update RowVersion, which is our concurrency check

* Fixed a critical issue where more than one bookmark could occur for a given chapter due to a race condition. Now we use concurrency checks and we also gracefully allow more than one row, by only grabbing first.

* Cleaned up the logic for IHasConcurencyToken and updated the setters to be private.

* Lots of comments and when deleting a library, remove any user progress items for which chapters don't exist.

* When deleting a Series, cleanup user progress rows.

* Now after a scan of library, if a series is removed, collection tags are pruned as well if there are no longer any series bound to it.

* Updated the image on the Readme to show a better picture

* Small code cleanup to remove null check modifier as I check for null just before then

* Fixed images loading multiple times due to using function in binding with random. You can now click chapter images to read that chapter specifically.

* Fixed cards being different sizes when read vs unread

* Moved over Robbie's workflow changes from notifier. Commented out activity indicators as that is not shipping with this release.

* Remove code that isn't needed

* Reverted GA

* Changed GA to trigger only when HEAD is updated
2021-08-17 14:15:32 -07:00
Joseph Milazzo
2fd02f0d2b
Custom Cover Images (#499)
* Added some documentation. Removed Require Admin Role from Search Tags. Added Summary to be updated on UpdateTag.

* Added Swagger xml doc generation to beef up the documentation. Started adding xml comments to the APIs. This is a needed, slow task for upcoming Plugins system.

* Implemented the ability to upload a custom series image to override the existing cover image.

Refactored some code out to use ImageService and added more documentation

* When a page cache fails, delete cache directory so user can try to reload.

* Implemented the ability to lock a series cover image such that after user uploads something, it wont get refreshed by Kavita.

* Implemented the ability to reset cover image for series by unlocking

* Kick off a series refresh after a cover is unlocked.

* Ability to press enter to load a url

* Ability to reset selection

* Cleaned up cover chooser such that reset is nicer, errors inform user to use file upload, series edit modal now doesn't use scrollable body. Mobile tweaks. CoverImageLocked is now sent to the UI.

* More css changes to look better

* When no bookmarks, don't show both markups

* Fixed issues where images wouldn't refresh after cover image was changed.

* Implemented the ability to change the cover images for collection tags.

* Added property and API for chapter cover image update

* Added UI code to prepare for updating cover image for chapters. need to rearrange components

* Moved a ton of code around to separate card related screens into their own module.

* Implemented the ability to update a chapter/volume cover image

* Refactored action for volume to say edit to reflect modal action

* Fixed issue where after editing chapter cover image, the underlying card wouldn't update

* Fixed an issue where we were passing volumeId to the reset chapter lock. Changed some logic in volume cover image generation.

* Automatically apply when you hit reset cover image
2021-08-15 10:36:47 -07:00
Joseph Milazzo
e9ec6671d5
Bookmarking Pages within the Reader (#469)
# Added
- Added: Added the ability to bookmark certain pages within the manga (image) reader and later download them from the series context menu. 

# Fixed
- Fixed: Fixed an issue where after adding a new folder to an existing library, a scan wouldn't be kicked off
- Fixed: In some cases, after clicking the background of a modal, the modal would close, but state wouldn't be handled as if cancel was pushed

# Changed
- Changed: Admin contextual actions on cards will now be separated by a line to help differentiate. 
- Changed: Performance enhancement on an API used before reading

# Dev
- Bumped dependencies to latest versions

=============================================
* Bumped versions of dependencies and refactored bookmark to progress.

* Refactored method names in UI from bookmark to progress to prepare for new bookmark entity

* Basic code is done, user can now bookmark a page (currently image reader only).

* Comments and pipes

* Some accessibility for new bookmark button

* Fixed up the APIs to work correctly, added a new modal to quickly explore bookmarks (not implemented, not final).

* Cleanup on the UI side to get the modal to look decent

* Added dismissed handlers for modals where appropriate

* Refactored UI to only show number of bookmarks across files to simplify delivery. Admin actionables are now separated by hr vs non-admin actions.

* Basic API implemented, now to implement the ability to actually extract files.

* Implemented the ability to download bookmarks.

* Fixed a bug where adding a new folder to an existing library would not trigger a scan library task.

* Fixed an issue that could cause bookmarked pages to get copied out of order.

* Added handler from series-card component
2021-08-10 18:18:07 -05:00
Joseph Milazzo
b0df67cdda
PDF Support + MORE!!!! (#416)
# Added
- Added support for PDFs within Kavita. PDFs will open in the Manga reader and you can read through them as images. PDFs are heavier than archives, so they may take longer to open for reading. (Fixes #187)

# Changed
- Changed: Major change in how Kavita libraries work. Kavita libraries will now allow for mixed media types, that means you can have raw images, archives, epubs, and pdfs all within your Manga library. In the case that the same Series exists between 2 different types of medias, they will be separated and an icon will show to help you identify the types. The correct reader will open regardless of what library you are on. Note: Nightly users need to delete their Raw Images libraries before updating.

# Fixed
- Fixed: Fixed an issue where checking if a file was modified since last scan always returned true, meaning we would do more I/O than was needed (Fixes #415)
- Fixed: There wasn't enough spacing on the top menu bar on the Manga reader
- Fixed: Fixed a bug where user preferences dark mode control always showed true, even if you were not using dark mode

# Dev stuff
- For image extraction, if there is only 1 image we will extract  just that, else we will extract only images
- Refactored all the Parser code out of the ScannerService into a self contained class. The class should be created for any scans, allowing multiple tasks to run without any chance of cross over.



* Fixed indentation for cs files

* Fixed an issue where the logic for if a file had been modified or not was not working and always saying modified, meaning we were doing more file I/O than needed.

* Implemented the ability to have PDF books. No reader functionality.

* Implemented a basic form of scanning for PDF files. Reworked Image based libraries to remove the need to separate in a special library and instead just work within the Manga/Comic library.

* Removed the old library types.

* Removed some extra code around old raw library types

* Fully implemented PDF support into Kavita by using docnet. Removed old libraries we tried that did not work. PDFs take about 200ms to save the file to disk, so they are much slower than reading archives.

* Refactored Libraries so that they can have any file extension and the UI will decide which reader to use.

* Reworked the Series Parsing code.

We now use a separate instance for each task call, so there should be no cross over if 2 tasks are running at the same time.

Second, we now store Format with the Series, so we can have duplicate Series with the same name, but a different type of files underneath.

* Fixed PDF transparency issues

- Used this code to fix an issue when a PDF page doesn't have a background. https://github.com/GowenGit/docnet/issues/8#issuecomment-538985672

- This also fixes the same issue for cover images

* Fixed an issue where if a raw image was in a directory with non-image files, those would get moved to cache when trying to open the file.

* For image extraction, if there is only 1 image, just copy that to cache instead of everything else in the directory that is an image.

* Add some spacing to the top menu bar

* Added an icon to the card to showcase the type of file

* Added a tag badge to the series detail page

* Fixed a bug in user preferences where dark mode control would default to true, even if you weren't on it

* Fixed some tests up

* Some code smells

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
2021-07-22 21:13:24 -05:00
Joseph Milazzo
be2b78fa5a
Manga Redesign (#321)
* Code cleanup, refactored FileRepository into Unit of Work.

* Added AutoCloseMenu and ReaderMode user perferences to match UI

* Added extra information to ChapterInfo

* Build changes

* Updated the readme to have open collective information and thanks to sponsors

* Fixed an issue with UnitOfWork refactor and how stats was bootsrapped. Replaced stats.kavitareader with a temp url to test out redirection bug.
2021-06-24 19:31:42 -05:00
Joseph Milazzo
aa7439178c
Don't log exceptions to Sentry when debugging locally. Fixed a constraint issue with collection tags that prevented deleting series. Ensure when we scan we add SeriesMetadata objects to existing series. (#265) 2021-06-04 17:49:19 -05:00
Joseph Milazzo
4910f1d1d0
Book Reading Progress Enhancement (#259)
* Added book reader reading direction preference

* Adds a new marker to the AppUserProgress to capture nearest anchor for resuming scroll point when reading books. Refactored bookmark api to return a BookmarkDto which includes this new data.
2021-06-03 18:33:13 -05:00
Joseph Milazzo
03c9b7a11d
Added book reader reading direction preference (#249) 2021-06-03 09:05:07 -05:00
Joseph Milazzo
d7d7f9b529
Collection Support (#234)
* Readme refactored to be more clean and clear, taking inspiration from wiki.js's readme.

* Initial backend for Collections and basic metadata implemented.

* More build flavors for Raspberry Pi users and updated Install since we don't need users to set their own JWT Token Key. Update a typo in appsettings.json file for prod.

* Fixed #224. Sort before getting a First?Last() chatper

* The rough ability to add and get series metadata and tags.

* Fix a bug on getting metadata for when it doesn't exist.

* Fixed a bug where flattening directories with some unique filenames could cause reading order of images to be out of order.

* Added a seed code to ensure all series have SeriesMetdata

* Ensure all instances of opening an epub is using "using" so we don't lock the file. When we have a malformed html file, log the issues and inform the user we can't open the file.

* Book reader now handles @Import "" statements in CSS and inlines the css into css file that references them. This allows for them to be scoped. In addition, if the html or body tag had classes, we now send back a single div with those classes.

* Fixed GetSeriesDtoForCollectionAsync which was not properly returning series

* Implemented cover image for collection tag. Fixed an issue in metadata update call.

* Add check for user access when resolving series for a collection tag. When asking for all tags, if the user is not an admin, only give promotoed tags back.

* Implemented updateTag api

* Implemented the ability to update series the tags have access to.

* Cleanup, sorting, and null check

* More sorting changes

* Ensure we can delete tags when editing a series tags

* Fix order of update to make sure a tag is properly deleted

* Code smells
2021-05-30 17:24:23 -05:00
Joseph Milazzo
8b1fbc0096
Implemented Dark Mode (#203) 2021-05-09 08:44:12 -05:00
Joseph Milazzo
2be1630af3
Tap to Paginate User Pref (#197)
* Fixed In Progress and removed comments

* Tap to Paginate user setting is implemented. Fixes #193
2021-05-04 17:01:27 -05:00
Joseph Milazzo
a01613f80f
EPUB Support (#178)
* Added book filetype detection and reorganized tests due to size of file

* Added ability to get basic Parse Info from Book and Pages.

* We can now scan books and get them in a library with cover images.

* Take the first image in the epub if the cover isn't set.

* Implemented the ability to unzip the ebup to cache. Implemented a test api to load html files.

* Just some test code to figure out how to approach this.

* Fixed some merge conflicts

* Removed some dead code from merge

* Snapshot: I can now load everything properly into the UI by rewriting the urls before I send them back. I don't notice any lag from this method. It can be optimized further.

* Implemented a way to load the content in the browser not via an iframe.

* Added a note

* Anchor mappings is complete. New anchors are updated so references now resolve to javascript:void() for UI to take care of internally loading and the appropriate page is mapped to it. Anchors that are external have target="_blank" added so they don't force you out of the app and styles are of course inlined.

* Oops i need this

* Table of contents api implemented (rough) and some small enhancements to codebase for books.

* GetBookPageResources now only loads files from within the book. Nested chapter list support and images now use html parsing instead of string parsing.

* Fonts now are remapped to load from endpoint.

* book-resources now uses a key, ensuring the file is in proper format for lookup. Changed chapter list based on structure with one HEADER and nested chapters.

* Properly handle svg resource requests and when there are part anchors that are clickable, make sure we handle them in the UI by adding a kavita-page handler.

* Add Chapter group page even if one isn't set by using first page (without part) from nestedChildren.

* Added extra debug code for issue #163.

* Added new user preferences for books and updated the css so we scope it to our reading section.

* Cleaned up style code

* Implemented ability to save book preferences and some cleanup on existing apis.

* Added an api for checking if a user has read something in a library type before.

* Forgot to make sure the has reading progress is against a user lol.

* Remove cacheservice code for books, sine we use an in-memory method

* Handle svg images as well

* Enhanced cover image extraction to check for a "cover" image if the cover image wasn't set in OPF before falling back to the first image.

* Fixed an issue with special books not properly generating metadata due to not having filename set.

* Cleanup, removed warmup task code from statup/program and changed taskscheduler to schedule tasks on startup only (or if tasks are changed from UI).

* Code cleanup

* Code cleanup

* So much code. Lots of refactors to try to test scanner service. Moved a lot of the queries into Extensions to allow to easier test, even though it's hacky. Support @font-face src:url swaps with ' and ". Source summary information from epubs.

* Well...baseURL needs to come from BE and not from UI lol.

* Adjusted migrations so default values match Entity

* Removed comment

* I think I finally fixed #163! The issue was that when i checked if it had a parserInfo, i wasn't considering that the chapter range might have a - in it (0-6) and so when the code to check if range could parse out a number failed, it treated it like a special and checked range against info's filename.

* Some bugfixes

* Lots of testing, extracting code to make it easier to test. This code is buggy, but fixed a bug where 1) If we changed the normalization code, we would remove the whole db during a scan and 2) We weren't actually removing series properly.

Other than that, code is being extracted to remove duplication and centralize logic.

* More code cleanup and test cleanup to ensure scan loop is working as expected and matches expectaions from tests.

* Cleaned up the code and made it so if I change normalization, which I do in this branch, it wont break existing DBs.

* Some comic parser changes for partial chapter support.

* Added some code for directory service and scanner service along with python code to generate test files (not used yet). Fixed up all the tests.

* Code smells
2021-04-28 16:16:22 -05:00
Joseph Milazzo
237542b493
Special Grouping (#134)
* More cases for parsing regex

* Implemented a change to fix old special grouping. Added some TODOs as well for a future enhancement
2021-03-31 12:10:26 -05:00
Joseph Milazzo
f85918b5bf When we scan a library, store the last write time for a file so we can skip page calculation if there were no modifications. 2021-03-23 17:28:48 -05:00
Joseph Milazzo
d73bd22db2 Some code cleanup 2021-03-23 12:22:47 -05:00
Joseph Milazzo
9035b6cc4e Hangfire now dies gracefully when using CTRL+C rather than Stop button in Rider. Implemented one stream method for testing. Regenerated a few migrations due to oversight in index not taking account of library. 2021-03-15 08:43:43 -05:00
Joseph Milazzo
1f24725905 Added indexes to Series table for searching. Removed byte[] from SearchResultDTO. Added response caching for all APIs. Responses are cached for 10 seconds. 2021-03-12 18:21:13 -06:00
Joseph Milazzo
c6e1fec9f2 Implemented the ability to correct a match from UI and further scans will show that new Series name. 2021-03-08 09:33:35 -06:00
Joseph Milazzo
9461b89725 A hefty refactor of the ScanLibrary code. There were significant fallouts due to duplicate entities getting created and SingleOrDefaults failing. 2021-02-08 12:03:52 -06:00
Joseph Milazzo
bd5a1338c4 Fixed a deployment bug where we weren't listening on port properly. New way will force firewall exception dialog on Windows and work across board. Implemented user preferences and ability to update them. 2021-02-06 13:08:48 -06:00
Joseph Milazzo
1050fa4e54 Ability to update settings. Take effect on next reboot. 2021-02-04 16:49:48 -06:00
Joseph Milazzo
53e85317f9 Refactored DB to be Series->Volumes->Chapters instead. All functionality that previously worked still works. Cleanup still needed. 2021-02-01 09:29:11 -06:00
Joseph Milazzo
18385a4f80 Implemented ability to have server settings. Currently cache directory is there but it is not configurable (or used in this commit) 2021-01-23 09:01:10 -06:00
Joseph Milazzo
ac993a59ba Implemented ability to leave a rating (up to 5 stars) and a text review (not UI supported until v0.2). 2021-01-19 17:06:26 -06:00
Joseph Milazzo
effdf07cef Very messy code that implements read status tracking. Needs major cleanup. 2021-01-17 15:05:27 -06:00
Joseph Milazzo
28ce2bbba1 Refactored volume to contain pages and removed /reader/info API endpoint. 2021-01-11 17:36:11 -06:00
Joseph Milazzo
59a4921ba9 Refactored ScanLibrary to produce page numbers on the Manga File, Format and to update existing series/volumes rather than always create new entries. 2021-01-09 15:27:04 -06:00
Andrew Song
451d459473 adding initial coverimage functionality 2021-01-03 19:50:10 -06:00
Joseph Milazzo
ac2b40aba6 Added migration for removing IsAdmin column since we use IdentityService Roles instead. 2021-01-03 14:16:06 -06:00
Joseph Milazzo
9168e12483 Refactored Volume to have Name and Number (int) so that we can properly sort and still handle possible split volumes.
Refactored ScanLibrary into Library controller and updated it so it adds the new library to all admins.
2021-01-02 12:21:36 -06:00
Joseph Milazzo
d632e53f18 Added ability to automatically track last modified and created timestamps for entities via an interface. DBContext will automatically update for us. 2021-01-02 10:59:52 -06:00
Joseph Milazzo
c429c50ba2 Fixed a bug in ScanLibrary that caused duplicated Volumes. Implemented APIs for navigating down to Volume for webui.
This is rough code and needs to be polished and refactored.
2021-01-01 14:04:31 -06:00
Joseph Milazzo
380c3e7b3c Rough version of Saving Series, Volumes, and MangaFiles to the DB. Relies on Cascaded delete rather than manually handling updating of file changes. 2020-12-30 11:30:12 -06:00
Joseph Milazzo
4fd9943b91 Added Hangfire with LiteDB for a task running system. At the most basic, this allows us to monitor tasks running on the system (during dev only) and run tasks on a reoccuring or ad-hoc basis. 2020-12-26 14:03:35 -06:00
Andrew Song
8f7df85d49 Refractor token auth stuff to use identiycore framework 2020-12-21 09:24:21 -06:00
Joseph Milazzo
b6e0e05205 Added new many to many migration for libraries and users. Add Library now works and you can get Library (entity) objects from a user. LibraryDto project is upcoming. 2020-12-18 12:01:45 -06:00
Joseph Milazzo
d5eed4e85d Lots of changes to get code ready to add library. 2020-12-17 11:27:19 -06:00
Joseph Milazzo
5da41ea6f3 Added User with ability to login and register. By default, user is not an admin. DTO expects an integer and will convert to Boolean. 2020-12-13 16:07:25 -06:00