20 Commits

Author SHA1 Message Date
Joseph Milazzo
a01613f80f
EPUB Support (#178)
* Added book filetype detection and reorganized tests due to size of file

* Added ability to get basic Parse Info from Book and Pages.

* We can now scan books and get them in a library with cover images.

* Take the first image in the epub if the cover isn't set.

* Implemented the ability to unzip the ebup to cache. Implemented a test api to load html files.

* Just some test code to figure out how to approach this.

* Fixed some merge conflicts

* Removed some dead code from merge

* Snapshot: I can now load everything properly into the UI by rewriting the urls before I send them back. I don't notice any lag from this method. It can be optimized further.

* Implemented a way to load the content in the browser not via an iframe.

* Added a note

* Anchor mappings is complete. New anchors are updated so references now resolve to javascript:void() for UI to take care of internally loading and the appropriate page is mapped to it. Anchors that are external have target="_blank" added so they don't force you out of the app and styles are of course inlined.

* Oops i need this

* Table of contents api implemented (rough) and some small enhancements to codebase for books.

* GetBookPageResources now only loads files from within the book. Nested chapter list support and images now use html parsing instead of string parsing.

* Fonts now are remapped to load from endpoint.

* book-resources now uses a key, ensuring the file is in proper format for lookup. Changed chapter list based on structure with one HEADER and nested chapters.

* Properly handle svg resource requests and when there are part anchors that are clickable, make sure we handle them in the UI by adding a kavita-page handler.

* Add Chapter group page even if one isn't set by using first page (without part) from nestedChildren.

* Added extra debug code for issue #163.

* Added new user preferences for books and updated the css so we scope it to our reading section.

* Cleaned up style code

* Implemented ability to save book preferences and some cleanup on existing apis.

* Added an api for checking if a user has read something in a library type before.

* Forgot to make sure the has reading progress is against a user lol.

* Remove cacheservice code for books, sine we use an in-memory method

* Handle svg images as well

* Enhanced cover image extraction to check for a "cover" image if the cover image wasn't set in OPF before falling back to the first image.

* Fixed an issue with special books not properly generating metadata due to not having filename set.

* Cleanup, removed warmup task code from statup/program and changed taskscheduler to schedule tasks on startup only (or if tasks are changed from UI).

* Code cleanup

* Code cleanup

* So much code. Lots of refactors to try to test scanner service. Moved a lot of the queries into Extensions to allow to easier test, even though it's hacky. Support @font-face src:url swaps with ' and ". Source summary information from epubs.

* Well...baseURL needs to come from BE and not from UI lol.

* Adjusted migrations so default values match Entity

* Removed comment

* I think I finally fixed #163! The issue was that when i checked if it had a parserInfo, i wasn't considering that the chapter range might have a - in it (0-6) and so when the code to check if range could parse out a number failed, it treated it like a special and checked range against info's filename.

* Some bugfixes

* Lots of testing, extracting code to make it easier to test. This code is buggy, but fixed a bug where 1) If we changed the normalization code, we would remove the whole db during a scan and 2) We weren't actually removing series properly.

Other than that, code is being extracted to remove duplication and centralize logic.

* More code cleanup and test cleanup to ensure scan loop is working as expected and matches expectaions from tests.

* Cleaned up the code and made it so if I change normalization, which I do in this branch, it wont break existing DBs.

* Some comic parser changes for partial chapter support.

* Added some code for directory service and scanner service along with python code to generate test files (not used yet). Fixed up all the tests.

* Code smells
2021-04-28 16:16:22 -05:00
Joseph Milazzo
d3c14863d6
Performance, Scan Loop, Specials, and cleanup (#150)
* More cases for parsing regex

* Fixed a bug where chapter cover images weren't being updated due to a missed not.

* Removed a piece of code that was needed for upgrading, since all beta users agreed to wipe db.

* Fixed InProgress to properly respect order and show more recent activity first. Issue is with IEntityDate LastModified not updating in DataContext.

* Updated dependencies to lastest stable.

* LastModified on Volumes wasn't updating, validated it does update when data is changed.

* Rewrote a check to avoid a small heap object warning.

* Ensure UpdateSeries checks all libraries for unique name.

* Took care of some todos, removed unused imports, on dev go ahead and schedule reoocuring jobs since LiteDB caused the locking issue.

* No Tracking when we aren't using entities.

* Added code to remove abandoned progress rows after a chapter gets deleted.

* RefreshMetadata uses one large query rather than many trips to DB for updating metadata. Significantly faster.

* Fixed a bug where UpdateSeries would always complain about a unique name even when we weren't updating name.

* Files that are linked to a series but can't parse out Vol/Chapter information are properly grouped like other Specials.

* Refresh metadata on UI should call the task directly

* Fixed a bug on updating series to make sure we don't complain if we aren't trying to update the name to an existing name.

* Fixed #142 - Library cards should be sorted.

* Refactored the name of some variables to be more agnostic to comics.

* Implemented ScanLibrary but abandoning it.

* Code Cleanup & removing ScanSeries code.

* Some more tests and new Comparators for natural sorting.

* Fixed #137 - When performing I/O on archives, ignore __MACOSX folders completely.

* Fixed #137 - When performing I/O on archives, ignore __MACOSX folders completely.

* All entities that will show under specials tab should be marked special, rather than just what has a special keyword.

* Don't let specials generate cover images

* Don't let specials generate cover images

* SearchResults should send LocalizedName back since we are searching against it.

* Added some tests around macosx folders found from my actual server.

* Put extra notes about a case where duplicates come about, logger will now tell user about this issue.

* Missed a build issue somehow...

* Some code smells
2021-04-05 08:37:45 -05:00
Joseph Milazzo
d73bd22db2 Some code cleanup 2021-03-23 12:22:47 -05:00
Joseph Milazzo
a125b2ac0b Fixed GetInProgress API - The condition for PagesRead needed to be a sum for all progress, not each row. 2021-03-18 14:18:56 -05:00
Joseph Milazzo
0a85555f38 Temp commit to record attempts. Stream APIs are implemented and working. Added some new test cases based on deployed Kavita server testing. 2021-03-17 14:29:50 -05:00
Joseph Milazzo
6e6e5ee9f2 Implemented a startup task that will instantiate all the DI so first API isn't having to eat that overhead. 2021-03-13 17:44:29 -06:00
Joseph Milazzo
ecfb40cb2d Moved some files around, fixed Log file fetching to use zip and work more reliably. Refactored some code in directory service. 2021-02-24 15:58:17 -06:00
Joseph Milazzo
bbb4240e20 Implemented download log files (not in service). Refactored backupservice to handle log file splitting. Improved a few interfaces and added some unit tests around them. 2021-02-24 11:59:16 -06:00
Joseph Milazzo
b4ee16d8d1 Code cleanup. Implemented ability to schedule Library Backups. 2021-02-17 08:58:36 -06:00
Joseph Milazzo
d8d01ffaf6 More refactoring around ScannerService. Moved metadata into its own service. Focusing on cleaning up ScanLibrary code to work 100%. 2021-02-08 16:44:18 -06:00
Joseph Milazzo
f430595d11 Attempted to Test CacheService, but can't figure it out. 2021-01-26 14:35:50 -06:00
Joseph Milazzo
a057e3ce1d Lots of bug fixes around publishing and handling weird cases on a real manga library. Implemented ability to have Volumes number 0 aka just latest chapters. Refactored DirectoryService code for scanning into it's own service. Lots of debug code, will be cleaned up later. 2021-01-23 17:44:48 -06:00
Joseph Milazzo
825afd83a2 Removed some dead code on the interfaces. Introduced UnitOfWork to simplify repo injection. 2021-01-18 13:07:48 -06:00
Joseph Milazzo
731e3a9c5e Cache cleanup implemented 2021-01-11 15:39:25 -06:00
Joseph Milazzo
cd8a1d2892 Added a basic cache service to handle interations with the underlying cache implementation. Refactored some code to be more robust. 2021-01-09 16:14:28 -06:00
Joseph Milazzo
380c3e7b3c Rough version of Saving Series, Volumes, and MangaFiles to the DB. Relies on Cascaded delete rather than manually handling updating of file changes. 2020-12-30 11:30:12 -06:00
Joseph Milazzo
4fd9943b91 Added Hangfire with LiteDB for a task running system. At the most basic, this allows us to monitor tasks running on the system (during dev only) and run tasks on a reoccuring or ad-hoc basis. 2020-12-26 14:03:35 -06:00
Joseph Milazzo
d5eed4e85d Lots of changes to get code ready to add library. 2020-12-17 11:27:19 -06:00
Joseph Milazzo
13ed323949 Added new API for getting Member (aka Users but for use in FE). User is just used for login/registering. 2020-12-14 14:33:09 -06:00
Joseph Milazzo
2b521924d0 Setup basic code for login. 2020-12-12 20:14:56 -06:00