Kavita/API/Controllers/DownloadController.cs
Joseph Milazzo 33db123e81
v0.4.8 Release (#720)
* Bump versions by dotnet-bump-version.

* Bump versions by dotnet-bump-version.

* Workflow updates (#658)

# Added
- Added: Added automatic character parsing for discord notifier. Now if the PR is over a certain character limit, it will trim and add an appropriate link to the full changelog. (Release for Stable, PR for Dev)

# Removed
- Removed: Removed Sentry map task from the workflow since Sentry is no longer used.

* Bump versions by dotnet-bump-version.

* Misc Updates (#665)

* Do not allow non-admins to change their passwords when authentication is disabled

* Clean up the login page so that input field text is black

* cleanup some resizing when typing a password and having a lot of users

* Changed the LastActive for a user to not just be login, but also when they open an already authenticated session.

* Bump versions by dotnet-bump-version.

* Logging Cleanup (#668)

* Do not allow non-admins to change their passwords when authentication is disabled

* Clean up the login page so that input field text is black

* cleanup some resizing when typing a password and having a lot of users

* Changed the LastActive for a user to not just be login, but also when they open an already authenticated session.

* Removed some verbose debugging statements and moved some debug to information to be more prevelant to logs for default installs.

* In Progress now sends progress information on the Series

* Add ability to add cards to recently added when new series are added in backend

* Implemented the ability to click the glasses icon to turn off incognito mode from within the reader so you can start tracking progress

* Don't warn the user about authentication when they don't touch that control

* Bump versions by dotnet-bump-version.

* Changed the stats that are sent back to stat server from installed server.

* Revert "Changed the stats that are sent back to stat server from installed server."

This reverts commit 644cb6d1f67de9531ea1a1dfd3853709e0329ce7.

* Bump versions by dotnet-bump-version.

* Bump versions by dotnet-bump-version.

* Bulk Add to Collection (#674)

* Fixed the typeahead not having the same size input box as other inputs

* Implemented the ability to add multiple series to a collection through bulk operations flow. Updated book parser to handle "@import url('...');" syntax as well as @import '...';

* Implemented the ability to create a new Collection tag via bulk operations flow.

* Bump versions by dotnet-bump-version.

* Bulk Operations for In Progress and Recently Added (#677)

* Don't log a message about bad match if the file is a cover image

* Enable bulk operations for In Progress and Recently Added

* Fixed a bad logic case

* Bump versions by dotnet-bump-version.

* Regression Fix (#680)

* Ensure we mount the backups directory for Docker users

* Fixed a huge logic bug that deleted files in users libraries

* Bump versions by dotnet-bump-version.

* Change chunk size to be a fixed 50 to validate if it's causing issue with refresh. Added some try catches to see if exceptions are causing issues. (#681)

* Bump versions by dotnet-bump-version.

* Fixed a bug where searching on localized name would fail to show on the search. Fixed a bug where extra spaces would cause the search results not to show properly. (#682)

* Bump versions by dotnet-bump-version.

* When we have a special marker, ensure we fall back to folder parsing to try and group correctly to the actual series before just accepting what we parsed. (#684)

Fixed a missed parsing case where comic special parsing wasn't being called on comic libraries.

* Bump versions by dotnet-bump-version.

* iOS Admin page dropdown fix (#686)

# Fixed:
- Fixed: Fixed an issue where the dropdown on the admin server page would not work on Safari or other iOS browsers.

* When the DB fails to save, log out all the series the user should look into for constraint issues and push a message to the admins connected to webui. (#687)

* Bump versions by dotnet-bump-version.

* Bump versions by dotnet-bump-version.

* Stat upload will now schedule itself between midnight and 6am in server time for upload. (#688)

* Bump versions by dotnet-bump-version.

* EPUB CSS Parsing Issues (#690)

* WIP. Rewrote some of the Regex to better support css escaping. We now escape background-image, border-image, and list-style-image within css files.

* Added position relative to help with positioning on books that are just absolute positioned elements.

* When there is absolute positioning, like in some epub based comics, supress the bottom action bar since it wont render in the correct location.

* Fixed tests

* Commented out tests

* Bump versions by dotnet-bump-version.

* More EPUB Scoping Fixes (#691)

* Added better handling around when importing css files that are empty. Moved comment removal on css files to before some css whitespace cleanup to get better matches.

* Some enhancements on the checks to see if we need the bottom action bar on reader. Now we don't query DOM and have something that works more reliably.

* Bump versions by dotnet-bump-version.

* Fixed an issue where docker users were not properly backing up the database. Removed an empty File for when covers/ had nothing in it. (#692)

* Bump versions by dotnet-bump-version.

* Fallback to Folder Parsing Issue (#694)

* Fixed a bug in the scanner where we fall back to parsing from folders for poorly named files. The code was exiting early if a chapter or volume could be parsed out.

* Fixed a unit test by tweaking a regex for fallback

* Bump versions by dotnet-bump-version.

* KavitaStats Cleanup (#695)

* Refactored Stats code to be much cleaner and user better naming.

* Cleaned up the actual http code to use Flurl and to return if the upload was successful or not so we can delete the file where appropriate.

* More refactoring for the stats code to clean it up and keep it consistent with our standards.

* Removed a confusing log statement

* Added support for old api key header from original stat server

* Use the correct endpoint, not the new one.

* Code smell

* Bump versions by dotnet-bump-version.

* Bulk Deletion (#697)

* Implemented bulk deletion of series

* Don't show unauthorized exception on UI, just redirect to the login page.

* Bump versions by dotnet-bump-version.

* Cover Image Picking + Forwarding Headers with EPUBs (#700)

* Ensure Kavita knows about forwarding headers (fixes issue with epub urls not going through https with reverse proxy). Fixed a case where cover image selection preferred nested folders vs files in root directory.

* Fixed broken unit test

* Added bug that I fixed to the unit tests

* Cover Image Picking + Forwarding Headers with EPUBs (#702)

* Updating GA Bump version temporarily for fix (#703)

* Bump versions by dotnet-bump-version.

* Cover Image Picking + Forwarding Headers with EPUBs (GA Fix) (#704)

* Bump versions by dotnet-bump-version.

* Vacation Fixes (#709)

* Ignore system and hidden folders when performing directory scan.

* Fixed the comic parser tests not using Comic mode for parsing.

* Accept all forwarded headers and use them.

* Ignore some changes from another branch

* Bump versions by dotnet-bump-version.

* Breaking Changes: Docker Parity (#698)

* Refactored all the config files for Kavita to be loaded from config/. This will allow docker to just mount one folder and for Update functionality to be trivial.

* Cleaned up documentation around new update method.

* Updated docker files to support config directory

* Removed entrypoint, no longer needed

* Update appsettings to point to config directory for logs

* Updated message for docker users that are upgrading

* Ensure that docker users that have not updated their mount points from upgrade cannot start the server

* Code smells

* More cleanup

* Added entrypoint to fix bind mount issues

* Updated README with new folder structure

* Fixed build system for new setup

* Updated string path if user is docker

* Updated the migration flow for docker to work properly and Fixed LogFile configuration updating.

* Migrating docker images is now working 100%

* Fixed config from bad code

* Code cleanup

Co-authored-by: Chris Plaatjes <kizaing@gmail.com>

* Bump versions by dotnet-bump-version.

* Feature/docker parity (#714)

* Refactored all the config files for Kavita to be loaded from config/. This will allow docker to just mount one folder and for Update functionality to be trivial.

* Cleaned up documentation around new update method.

* Updated docker files to support config directory

* Removed entrypoint, no longer needed

* Update appsettings to point to config directory for logs

* Updated message for docker users that are upgrading

* Ensure that docker users that have not updated their mount points from upgrade cannot start the server

* Code smells

* More cleanup

* Added entrypoint to fix bind mount issues

* Updated README with new folder structure

* Fixed build system for new setup

* Updated string path if user is docker

* Updated the migration flow for docker to work properly and Fixed LogFile configuration updating.

* Migrating docker images is now working 100%

* Fixed config from bad code

* Code cleanup

* Fixed monorepo-build.sh

Co-authored-by: Chris Plaatjes <kizaing@gmail.com>

* Breaking Changes: Docker Parity (#715)

* Fixed a bug in the copy directory to directory in the migration

* Somehow GetFiles lost static modifier.

* Bump versions by dotnet-bump-version.

* Build issue (#716)

* Fixed a bug in the copy directory to directory in the migration

* Somehow GetFiles lost static modifier.

* Please work

* Bump versions by dotnet-bump-version.

* Bump versions by dotnet-bump-version.

* Shakeout Changes (#717)

* Make the appsettings public on Configuration and change how we detect when to migrate for non-docker users.

* Fixed up non-docker copy command and removed duplicate check on source directory for a copy.

* Don't delete files unless we know we are successful

* Bump versions by dotnet-bump-version.

* Fixed a migration issue on docker happening too many times or throwing exception when source wasn't there. (#719)

* Bump versions by dotnet-bump-version.

* Version bump for release (#718)

* Bump versions by dotnet-bump-version.

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
Co-authored-by: YEGCSharpDev <89283498+YEGCSharpDev@users.noreply.github.com>
Co-authored-by: Chris Plaatjes <kizaing@gmail.com>
2021-11-04 05:29:02 -07:00

192 lines
8.6 KiB
C#

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using API.Comparators;
using API.DTOs.Downloads;
using API.Entities;
using API.Entities.Enums;
using API.Extensions;
using API.Interfaces;
using API.Interfaces.Services;
using API.Services;
using Kavita.Common;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
namespace API.Controllers
{
[Authorize(Policy = "RequireDownloadRole")]
public class DownloadController : BaseApiController
{
private readonly IUnitOfWork _unitOfWork;
private readonly IArchiveService _archiveService;
private readonly IDirectoryService _directoryService;
private readonly ICacheService _cacheService;
private readonly IDownloadService _downloadService;
private readonly NumericComparer _numericComparer;
private const string DefaultContentType = "application/octet-stream";
public DownloadController(IUnitOfWork unitOfWork, IArchiveService archiveService, IDirectoryService directoryService, ICacheService cacheService, IDownloadService downloadService)
{
_unitOfWork = unitOfWork;
_archiveService = archiveService;
_directoryService = directoryService;
_cacheService = cacheService;
_downloadService = downloadService;
_numericComparer = new NumericComparer();
}
[HttpGet("volume-size")]
public async Task<ActionResult<long>> GetVolumeSize(int volumeId)
{
var files = await _unitOfWork.VolumeRepository.GetFilesForVolume(volumeId);
return Ok(DirectoryService.GetTotalSize(files.Select(c => c.FilePath)));
}
[HttpGet("chapter-size")]
public async Task<ActionResult<long>> GetChapterSize(int chapterId)
{
var files = await _unitOfWork.ChapterRepository.GetFilesForChapterAsync(chapterId);
return Ok(DirectoryService.GetTotalSize(files.Select(c => c.FilePath)));
}
[HttpGet("series-size")]
public async Task<ActionResult<long>> GetSeriesSize(int seriesId)
{
var files = await _unitOfWork.SeriesRepository.GetFilesForSeries(seriesId);
return Ok(DirectoryService.GetTotalSize(files.Select(c => c.FilePath)));
}
[HttpGet("volume")]
public async Task<ActionResult> DownloadVolume(int volumeId)
{
var files = await _unitOfWork.VolumeRepository.GetFilesForVolume(volumeId);
var volume = await _unitOfWork.VolumeRepository.GetVolumeByIdAsync(volumeId);
var series = await _unitOfWork.SeriesRepository.GetSeriesByIdAsync(volume.SeriesId);
try
{
if (files.Count == 1)
{
return await GetFirstFileDownload(files);
}
var (fileBytes, _) = await _archiveService.CreateZipForDownload(files.Select(c => c.FilePath),
$"download_{User.GetUsername()}_v{volumeId}");
return File(fileBytes, DefaultContentType, $"{series.Name} - Volume {volume.Number}.zip");
}
catch (KavitaException ex)
{
return BadRequest(ex.Message);
}
}
private async Task<ActionResult> GetFirstFileDownload(IEnumerable<MangaFile> files)
{
var (bytes, contentType, fileDownloadName) = await _downloadService.GetFirstFileDownload(files);
return File(bytes, contentType, fileDownloadName);
}
[HttpGet("chapter")]
public async Task<ActionResult> DownloadChapter(int chapterId)
{
var files = await _unitOfWork.ChapterRepository.GetFilesForChapterAsync(chapterId);
var chapter = await _unitOfWork.ChapterRepository.GetChapterAsync(chapterId);
var volume = await _unitOfWork.VolumeRepository.GetVolumeByIdAsync(chapter.VolumeId);
var series = await _unitOfWork.SeriesRepository.GetSeriesByIdAsync(volume.SeriesId);
try
{
if (files.Count == 1)
{
return await GetFirstFileDownload(files);
}
var (fileBytes, _) = await _archiveService.CreateZipForDownload(files.Select(c => c.FilePath),
$"download_{User.GetUsername()}_c{chapterId}");
return File(fileBytes, DefaultContentType, $"{series.Name} - Chapter {chapter.Number}.zip");
}
catch (KavitaException ex)
{
return BadRequest(ex.Message);
}
}
[HttpGet("series")]
public async Task<ActionResult> DownloadSeries(int seriesId)
{
var files = await _unitOfWork.SeriesRepository.GetFilesForSeries(seriesId);
var series = await _unitOfWork.SeriesRepository.GetSeriesByIdAsync(seriesId);
try
{
if (files.Count == 1)
{
return await GetFirstFileDownload(files);
}
var (fileBytes, _) = await _archiveService.CreateZipForDownload(files.Select(c => c.FilePath),
$"download_{User.GetUsername()}_s{seriesId}");
return File(fileBytes, DefaultContentType, $"{series.Name}.zip");
}
catch (KavitaException ex)
{
return BadRequest(ex.Message);
}
}
[HttpPost("bookmarks")]
public async Task<ActionResult> DownloadBookmarkPages(DownloadBookmarkDto downloadBookmarkDto)
{
// We know that all bookmarks will be for one single seriesId
var series = await _unitOfWork.SeriesRepository.GetSeriesByIdAsync(downloadBookmarkDto.Bookmarks.First().SeriesId);
var totalFilePaths = new List<string>();
var tempFolder = $"download_{series.Id}_bookmarks";
var fullExtractPath = Path.Join(DirectoryService.TempDirectory, tempFolder);
if (new DirectoryInfo(fullExtractPath).Exists)
{
return BadRequest(
"Server is currently processing this exact download. Please try again in a few minutes.");
}
DirectoryService.ExistOrCreate(fullExtractPath);
var uniqueChapterIds = downloadBookmarkDto.Bookmarks.Select(b => b.ChapterId).Distinct().ToList();
foreach (var chapterId in uniqueChapterIds)
{
var chapterExtractPath = Path.Join(fullExtractPath, $"{series.Id}_bookmark_{chapterId}");
var chapterPages = downloadBookmarkDto.Bookmarks.Where(b => b.ChapterId == chapterId)
.Select(b => b.Page).ToList();
var mangaFiles = await _unitOfWork.ChapterRepository.GetFilesForChapterAsync(chapterId);
switch (series.Format)
{
case MangaFormat.Image:
DirectoryService.ExistOrCreate(chapterExtractPath);
_directoryService.CopyFilesToDirectory(mangaFiles.Select(f => f.FilePath), chapterExtractPath, $"{chapterId}_");
break;
case MangaFormat.Archive:
case MangaFormat.Pdf:
_cacheService.ExtractChapterFiles(chapterExtractPath, mangaFiles.ToList());
var originalFiles = DirectoryService.GetFilesWithExtension(chapterExtractPath,
Parser.Parser.ImageFileExtensions);
_directoryService.CopyFilesToDirectory(originalFiles, chapterExtractPath, $"{chapterId}_");
DirectoryService.DeleteFiles(originalFiles);
break;
case MangaFormat.Epub:
return BadRequest("Series is not in a valid format.");
default:
return BadRequest("Series is not in a valid format. Please rescan series and try again.");
}
var files = DirectoryService.GetFilesWithExtension(chapterExtractPath, Parser.Parser.ImageFileExtensions);
// Filter out images that aren't in bookmarks
Array.Sort(files, _numericComparer);
totalFilePaths.AddRange(files.Where((_, i) => chapterPages.Contains(i)));
}
var (fileBytes, _) = await _archiveService.CreateZipForDownload(totalFilePaths,
tempFolder);
DirectoryService.ClearAndDeleteDirectory(fullExtractPath);
return File(fileBytes, DefaultContentType, $"{series.Name} - Bookmarks.zip");
}
}
}