144 Commits

Author SHA1 Message Date
Michael Genson
6418a10428
fix: misused update over patch in last_update call (#2168)
* fixed mealplan timeline event task
fixed indentation to only look at one group at a time
changed grumpy update to happy patch

* updated pytest to catch this error

* I don't know how this got past the pre-commit
2023-02-23 13:10:47 -09:00
Michael Genson
fd03d468d4
Fix: Allow Last Made to be Updated on Locked Recipes (#2140)
* allow certain props to be updated on locked recipe

* pytest

* added "last_made" to hardcoded datetime fields

* refactored last made to its own route

* codegen/types

* updated pytest
2023-02-21 18:59:22 -09:00
Michael Genson
a6c46a7420
Feature: Shopping List Label Section Improvements (#2090)
* added backend for shopping list label config

* updated codegen

* refactored shopping list ops to service
removed unique contraint
removed label settings from main route/schema
added new route for label settings

* codegen

* made sure label settings output in position order

* implemented submenu for label order drag and drop

* removed redundant label and tweaked formatting

* added view by label to user preferences

* made items draggable within each label section

* moved reorder labels to its own button

* made dialog scrollable

* fixed broken model

* refactored labels to use a service
moved shopping list label logic to service
modified label seeder to use service

* added tests

* fix for first label missing the tag icon

* fixed wrong mapped type

* added statement to create existing relationships

* fix restore test, maybe
2023-02-21 18:58:41 -09:00
Michael Genson
5562effd66
feat: select ingredients to add to shopping List (#2136)
* added recipe ingredient override to backend

* pytest

* new dialog to filter recipe items added to list
2023-02-19 16:20:32 -09:00
Michael Genson
35124ea875
feat: Shopping List Item Pagination Route (#2145)
* added pagination route for list items

* pytest
2023-02-19 15:48:19 -09:00
Michael Genson
05e2566c35
fix: recipe scraper image cleaning (#2139)
* updated image cleaner
enabled image cleaner
added case for nested image dicts

* refactored image cleaner to return a list of urls
2023-02-19 15:43:52 -09:00
Sören
6a5f9d7f6b
feat: diacritic-insensitive search (#2132)
* add normalized columns and use them for search

* add migration to fill all normalized columns
2023-02-19 15:40:18 -09:00
Hayden
71f8c1066a
feat: server side search (#2112) (#2117)
* feat: server side search API (#2112)

* refactor repository_recipes filter building

* add food filter to recipe repository page_all

* fix query type annotations

* working search

* add tests and make sure title matches are ordered correctly

* remove instruction matching again

* fix formatting and small issues

* fix another linting error

* make search test no rely on actual words

* fix failing postgres compiled query

* revise incorrectly ordered migration

* automatically extract latest migration version

* test migration orderes

* run type generators

* new search function

* wip: new search page

* sortable field options

* fix virtual scroll issue

* fix search casing bug

* finalize search filters/sorts

* remove old composable

* fix type errors

---------

Co-authored-by: Sören <fleshgolem@gmx.net>
2023-02-11 21:26:10 -09:00
Sören
7d4a379f0d
feat: improve database indexing (#2104)
* add indices to all foreign keys and some fields that are used for ordering and filtering

* add missing migrations

* update migration orders

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2023-02-11 10:40:53 -09:00
Carter
da60e56982
fix: user login and creation with LDAP (#2107)
* Corrected if statement to check if a results was returned by the LDAP search. And decoded the user_attributes from binary data to string

* removed trailing spaces

* Revert asserts in LDAP unit test back

Since an empty tuple is still a result, an user is created and the result should not be false.

* Simplified code

* Extended the LDAP implementation

* fix ldap authentication and user creation

* modified docs to include new LDAP environment variables

* update tests and linting

* add libldap-2.4-2 as runtime dependency for the api

---------

Co-authored-by: Erik Landkroon <eriklandkroon@gmail.com>
2023-02-11 10:16:33 -09:00
Michael Genson
5f7ac92c96
feat: timeline event for mealplans (#2050)
* added related user to mealplans

* made timeline event message actually optional

* added task to create events for mealplan recipes

* replaced fk constraint ops with bulk ops

* fixed event creation and adjusted query range

* indentation is hard

* added missing recipe id query filter

* added tests
2023-02-11 10:08:53 -09:00
Hayden
9e77a9f367
prs-fleshgolem-2070: feat: sqlalchemy 2.0 (#2096)
* upgrade sqlalchemy to 2.0

* rewrite all db models to sqla 2.0 mapping api

* fix some importing and typing weirdness

* fix types of a lot of nullable columns

* remove get_ref methods

* fix issues found by tests

* rewrite all queries in repository_recipe to 2.0 style

* rewrite all repository queries to 2.0 api

* rewrite all remaining queries to 2.0 api

* remove now-unneeded __allow_unmapped__ flag

* remove and fix some unneeded cases of "# type: ignore"

* fix formatting

* bump black version

* run black

* can this please be the last one. okay. just. okay.

* fix repository errors

* remove return

* drop open API validator

---------

Co-authored-by: Sören Busch <fleshgolem@gmx.net>
2023-02-06 18:43:12 -09:00
Michael Genson
4fc4ba934d
fix: remove network calls from tests (#2055)
* abstracted scraper get_html method

* applied mock to all scrapers

* fixed incorrect var reference
2023-02-05 09:14:57 -09:00
Michael Genson
2340ee5bfb
Dev Improvements (#2058)
* added ruff cache to git ignore

* moved venv to project and added interpreter path

* added dummy method to LDAP test

* removed unused setting
2023-01-28 15:49:09 -09:00
Michael Genson
617cc1fdfb
Refactor Shopping List API (#2021)
* tidied up shopping list item models
redefined recipe refs and updated models
added calculated display attribute to unify shopping list item rendering
added validation to use a food's label if an item's label is null

* fixed schema reference

* refactored shopping list item service
route all operations through one central method to account for edgecases
return item collections for all operations to account for merging
consolidate recipe items before sending them to the shopping list

* made fractions prettier

* replaced redundant display text util

* fixed edgecase for zero quantity items on a recipe

* fix for pre-merging recipe ingredients

* fixed edgecase for merging create_items together

* fixed bug with merged updated items creating dupes

* added test for self-removing recipe ref

* update items are now merged w/ existing items

* refactored service to make it easier to read

* added a lot of tests

* made it so checked items are never merged

* fixed bug with dragging + re-ordering

* fix for postgres cascade issue

* added prevalidator to recipe ref to avoid db error
2023-01-28 15:45:02 -09:00
Sören
3415a9c310
Convert scraper to use async (#1915)
* add httpx depedency for async http requests

* rework scraper strategies to download recipe html asynchronously

* rework recipe_data_service to download recipe images asynchronously

* fix recipe_parser test, so it can use async results

* fix bulk import so that it also works with async scraper

* fix broken recipe_parser tests

* Fix issues found by scanners

* Add additional checks for ingredient and instruction count in test_create_by_url

* Revert changes in test recipe_data
Since we are checking ingredients and instructions in test_create_url now, these would fail with the stored html of recipe data

* Add explicit type annotation in recipe_data_service.largest_content_len

* Fix typo in annotation
2023-01-28 15:43:27 -09:00
Sören
0cd892059b
Fix/issue#2003 - Unable to remove all instructions/ingredients from a recipe (#2008)
* Fix issue where recipes could not have all their ingredients/instructions removed

* Add test for removing all instructions and ingredients from a recipe
2023-01-08 10:46:51 -09:00
Hayden
4d5550ad85
chore: mypy, ruff, extract, openapi, and recipes-scraper updates (#1968)
* bump ruff and mypy

* mypy fixes

* bump and fix openapi

* drop debug statement

* bump extruct

* bump recipes-scrapers
2023-01-01 13:47:27 -09:00
Michael Genson
c4eebaccca
Fix for shopping list recipe delete route (#1954)
* fixed broken types

* changed remove recipe route from delete to post
2022-12-30 22:09:22 -09:00
Michael Genson
46cc3898ab
feat: Improved Recipe Scaling Support and Shopping List Recipe Refactor (#1847)
* propogate scale changes to print view

* fixed incorrect variable reference

* refactored shopping list recipe routes
cleaned up existing logic
added support for recipe scaling

* updated current revision

* adding to shopping list respects UI recipe scale

* added field annotations

* added tests for recipe scaling

* made column nullable and set to 1 during migration
2022-12-30 13:47:35 -09:00
Hayden
d9c39cc1d0
chore: bump aiofiles, dotenv, fastapi, pydantic, uvicorn (#1944)
* update aiofiles and dotenv

* bump fastapi and pydantic

* update testclient

* bump ruff and uvicorn
2022-12-30 11:44:54 -09:00
Philipp
33dffccaa5
feat: duplicate recipes (#1750)
* feature/frontend: Add duplicate button to recipe

* feature/backend: Add recipe duplication endpoint

* feature/frontend: add duplication API call

* Regenerate API docs

* Fix linter errors

* Fix backend linter error

* Move recipe duplication logic to recipe service

* Add test for recipe duplication

* Improve recipe ingredients copy test

* generate types

* import type

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-11-30 20:57:26 -09:00
Michael Genson
a2dcdc1adf
feat: "I Made This" Dialog (#1801)
* added chef hat

* removed unnecessary log

* modified recipe and recipe timeline event schema
changed timeline event "message" -> "event_message"
added "last made" timestamp to recipe

* added "I made this" dialog to recipe action menu

* added missing field and re-ran code-gen

* moved dialog out of context menu and refactored
removed references in action menu and context menu
refactored dialog to be triggered by a button instead
added route to update recipe last made timestamp
added visual for last made timestamp to recipe header and title

* added sorting by last made

* switched event type to comment

* replaced alter column with pydantic alias

* added tests for event message alias
2022-11-13 14:12:53 -09:00
Hayden
89d0cae51d
refactor: rewrite cleaner functions for parsing recipe dicts (#1743)
* rewrite cleaner functions

* unify verbage

* try importing dep during check

* fix syntax

* allow override defaults

* satisfy mypy
2022-11-10 15:16:51 -09:00
Michael Genson
6ee64535df
feat: recipe timeline backend api (#1685)
* added recipe_timeline_events table to db

* added schema and routes for recipe timeline events

* added missing mixin and fixed update schema

* added tests

* adjusted migration revision tree

* updated alembic revision test

* added initial timeline event for new recipes

* added additional tests

* added event bus support

* renamed event_dt to timestamp

* add timeline_events to ignore list

* run code-gen

* use new test routes implementation

* use doc string syntax

* moved event type enum from db to schema

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-11-01 00:12:26 -08:00
Hayden
9ecef4c25f
chore: file generation cleanup (#1736)
This PR does too many things :( 

1. Major refactoring of the dev/scripts and dev/code-generation folders. 

Primarily this was removing duplicate code and cleaning up some poorly written code snippets as well as making them more idempotent so then can be re-run over and over again but still maintain the same results. This is working on my machine, but I've been having problems in CI and comparing diffs so running generators in CI will have to wait. 

2. Re-Implement using the generated api routes for testing

This was a _huge_ refactor that touched damn near every test file but now we have auto-generated typed routes with inline hints and it's used for nearly every test excluding a few that use classes for better parameterization. This should greatly reduce errors when writing new tests. 

3. Minor Perf improvements for the All Recipes endpoint

  A. Removed redundant loops
  B. Uses orjson to do the encoding directly and returns a byte response instead of relying on the default 
       jsonable_encoder.

4. Fix some TS type errors that cropped up for seemingly no reason half way through the PR.

See this issue https://github.com/phillipdupuis/pydantic-to-typescript/issues/28

Basically, the generated TS type is not-correct since Pydantic will automatically fill in null fields. The resulting TS type is generated with a ? to indicate it can be null even though we _know_ that i can't be.
2022-10-18 14:49:41 -08:00
Hayden
a8f0fb14a7
chore: upgrade pre-commit hooks (#1735)
* change pep585 hook to pyupgrade

* run pyupgrade + cleanup lint errors
2022-10-17 14:37:06 -08:00
Hayden
e516a2e801
fix: unclosed sessions (#1734)
* resolve session leak

* cleanup session management functions
2022-10-17 14:11:40 -08:00
Michael Genson
796e55b7d5
refactor: webhook events (#1661)
* refactored EventBusService to work outside FastAPI

* extended event models

* refactored webhooks to run through event bus

* added basic webhook test route

* changed get_all to page_all

* fixed incorrectly implemented Vue variables

* fixed broken webhook test

* changed factory from staticmethod to classmethod

* made query boundary definitions easier to read
2022-09-27 18:55:20 -08:00
Michael Genson
8271c3001e
feat: add support for API extras on shopping lists, shopping list items, and food data (#1619)
* added api extras to other tables
genericized api extras model from recipes
added extras column to ingredient foods
added extras column to shopping lists
added extras column to shopping list items

* updated alembic version test

* made mypy happy

* added TODO on test that does nothing

* added extras tests for lists, items, and foods

* added docs for new extras

* modified alembic versions to eliminate branching
2022-09-27 18:53:22 -08:00
Ben Boeckel
2e6b877ba9
docs: fix typos (#1665)
* docs: fix typos

* typos: fix typos found by `codespell` across the codebase

* docs: fix `macOS` spelling

* docs: fix `authentification` terminology

"Authentification" is not a thing.

* docs: fix `localhost` typo in example link

* typos: fix in-code typos

These are potentially higher risk, but no other mentions of these typos
show up in the codebase.
2022-09-25 15:17:27 -08:00
Hayden
39adea4ee3
feat (WIP): bring png OCR scanning support (#1670)
* Add pytesseract

* Add simple ocr endpoint

replace extension argument

* feat/ocr-editor gui

* fix frontend linting issues

* Add service unit tests

* Add split text modes & single ingredient/instruction editing

* make split mode really reactive

* Remove default step and ingredient

* make the linter haappy

* Accept only image uploads

* Add automatic recipe title suggestion

* Correct regex

* fix incorrect array.map method usage

* make the linter happy again

* Swap route to use asset name

* Rearange buttons

* fix test data

* feat: Allow making image the recipe image

* Add translation

* Make the linter happy

* Restrict function setPropertyValueByPath generic

* Restrict template literal type

* Add a more friendly icon to creation page

* update poetry lock file

* Correct sloppy ocr classes

* Make MyPy happy

* Rewrite safer tests

* Add tesseract to backend test CI container dependencies

* Make canvas element a component global

* Remove unwanted spaces in selected text

* Add way to know if recipe was created with ocr

* Access to ocr-editor for ocr recipes

* Update Alembic revision

* Make the frontend build

* Fix scrolling offset bug

* Allow creation of recipes with custom settings

* Fix rebasing mistakes

* Add format_tsv_output test

* Exclude the tests data directory only

* Enforce camelCase for frontend functions

* Remove import of unused component

* Fix type and class initialization

* Add multi-language support

* Highlight words in mount

* Fix image ratio bug

* Better ocr creation page

* Revert awkward feature to scroll in Selection mode

* Rebasing alembic migrations sux

* Remove obsolete getShared function

* Add function docstring

* Move down ocr creation option

* Make toolbar icons more generic

* Show help at the bottom of the page

* move ocr types to own file

* Use template ref for the canvas

* Use i18n.tc to get strings directly

* Correct naming mistake

* Move Ocr editor to own directory

* Create Ocr Editor parts

* Safeguard recipe properties access

* Add loading frontend animation due to longer request time

* minor cleanup chores

Co-authored-by: Miroito <alban.vachette@gmail.com>
2022-09-25 15:00:45 -08:00
Elegant
11eeab1b51
feat: LDAP improvements (#1487)
* Use Base DN for LDAP and fetch user attrs

Requires that a Base DN be set for LDAP
Set `full_name` and `email` based on LDAP attributes when creating user

* Add support for secure LDAP

Allow insecure LDAP connection (disabled by default)
Use CA when connecting to secure LDAP server

* Added missing quotes to example

* Update security.py

* Update security.py formatting

* Update security.py

Switched to f-String formatting

* formatting

* Update test_security.py

Added at attributes for testing

* Update test_security.py

Modified tests for base DN

* Update test_security.py

Set proper base DN for testing

* Update test_security.py

Corrected testing for LDAP

* Update test_security.py

Defined base_dn

* Authenticated user not in base DN

Add check for when user can authenticate but is not in base DN

* Update test_security.py

LDAP user cannot exist as it is searched before it is created and the list returns False

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-09-15 19:33:36 -08:00
Hayden
18b2c92a76
feat: public recipe access (#1610)
* initial public explorer API endpoint

* public API endpoint

* cleanup recipe page

* wip: init explorer page

* use public URLs for shared recipes

* refactor private share tokens to use shared page
2022-08-28 20:08:33 -08:00
Michael Genson
23c039b42d
refactor: event bus refactor (#1574)
* refactored event dispatching
added EventDocumentType and EventOperation to Event
added event listeners to bulk recipe changes
overhauled shopping list item events to be more useful
modified shopping list item repo to return more information

* added internal documentation for event types

* renamed message_types.py to event_types.py

* added unique event id and fixed instantiation

* generalized event listeners and publishers
moved apprise publisher to new apprise event listener
fixed duplicate message bug with apprise publisher

* added JWT field for user-specified integration id

* removed obselete test notification route

* tuned up existing notification tests

* added dependency to get integration_id from jwt

* added base crud controller to facilitate events

* simplified event publishing

* temporarily fixed test notification
2022-08-27 10:52:45 -08:00
Michael Genson
aaeb162dd5
feat: unify recipe card sections (#1560)
* removed unused import

* moved categories/tags to new recipe card section

* nuked old frontend sort code
minor refactoring

* bug fixes

* added backend recipes filter for tools

* removed debug log

* removed unusued props

* fixed sort for recipes by tool

* added tests for getting recipes by tool
2022-08-20 10:59:49 -08:00
Hayden
b3c41a4bd0
security: implement user lockout (#1552)
* add data-types required for login security

* implement user lockout checking at login

* cleanup legacy patterns

* expose passwords in test_user

* test user lockout after bad attempts

* test user service

* bump alembic version

* save increment to database

* add locked_at to datetime transformer on import

* do proper test cleanup

* implement scheduled task

* spelling

* document env variables

* implement context manager for session

* use context manager

* implement reset script

* cleanup generator

* run generator

* implement API endpoint for resetting locked users

* add button to reset all locked users

* add info when account is locked

* use ignore instead of expect-error
2022-08-13 13:18:12 -08:00
Hayden
e5bf7bce17
fix: case-sensitive-password-reset (#1545)
* make password reset case insensitive

* update test to check for case insensitive
2022-08-09 21:10:00 -08:00
Michael Genson
f45e2587a0
feat: category and tag filters to recipe pagination route (#1508)
* fixed incorrect response model

* added category and tag filters

* moved categories and tags params to route and
changed to query array param

* type fixes

* added category and tag tests
2022-08-09 14:01:59 -08:00
Philipp Fischbeck
34cd6eb687
fix: validate OpenAPI spec (#1528)
* init api check test

* Fix openAPI issues

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-08-06 16:54:29 -08:00
Hayden
11478134a1
security: restrict backup file upload (#1522) 2022-08-02 12:53:58 -08:00
Hayden
13850cda1f
security: multiple reported CVE fixes (#1515)
* update out of date license

* update typing / refactor

* fix arbitrarty path injection

* use markdown sanatizer to prevent XSS CWE-79

* fix CWE-918 SSRF by validating url and mime type

* add security docs

* update recipe-scrapers

* resolve DOS from arbitrary url

* update changelog

* bump version

* add ref to #1506

* add #1511 to changelog

* use requests decoder

* actually fix encoding issue
2022-07-31 13:10:20 -08:00
Michael Genson
7f50071312
feat: advanced filtering API (#1468)
* created query filter classes

* extended pagination to include query filtering

* added filtering tests

* type improvements

* move type help to dev depedency

* minor type and perf fixes

* breakup test cases

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-07-09 20:57:09 -08:00
Michael Genson
2809cef3b1
fix: mealplan pagination (#1464)
* added pagination to get_slice route

* updated mealplan tests

* renamed vars to match pagination query
2022-07-02 09:44:01 -08:00
Michael Genson
cb15db2d27
feat: re-write get all routes to use pagination (#1424)
rewrite get_all routes to use a pagination pattern to allow for better implementations of search, filter, and sorting on the frontend or by any client without fetching all the data. Additionally we added a CI check for running the Nuxt built to confirm that no TS errors were present. Finally, I had to remove the header support for the Shopping lists as the browser caching based off last_updated header was not allowing it to read recent updates due to how we're handling the updated_at property in the database with nested fields. This will have to be looked at in the future to reimplement. I'm unsure how many other routes have a similar issue. 

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-06-25 11:39:38 -08:00
Michael Genson
292bf7068a
feat: added "last-modified" header to supported record types (#1379)
* fixed type error

* exposed created/updated timestamps to shopping list schema

* added custom route to mix in "last-modified" header when available in CRUD routes

* mixed in MealieCrudRoute to APIRouters

* added HEAD route for shopping lists/list-items

* replaced default serializer with FastAPI's
2022-06-21 09:41:14 -08:00
Hayden
5a053cdcd6
feat: mealplan-webhooks (#1403)
* fix type errors on event bus

* webhooks fields required for new implementation

* db migration

* wip: webhook query + tests and stub function

* ignore type checker error

* type and method cleanup

* datetime and time utc validator

* update testing code for utc scheduled time

* fix file cmp function call

* update version_number

* add support for translating "time" objects when restoring backup

* bump recipe-scrapers

* use specific import syntax

* generate frontend types

* utilize names exports

* use utc times

* add task to scheduler

* implement new scheduler functionality

* stub for type annotation

* implement meal-plan data getter

* add experimental banner
2022-06-17 13:25:47 -08:00
Michael Genson
754e77c9cb
feat: extend Apprise JSON notification functionality with programmatic data (#1355)
* Fixed incorrect generic deleted notification text

* Added custom "event_source" header for json notifs

* Added internal reference data to event notifs

* Added event listeners to shopping list items

* Fixed type issues

* moved JSON event source k:v pairs to message body

* added hook for all supported custom endpoints
fixed bug that excluded non-custom notification types

* created event_source class to replace loosely-typed dict

* fixed silent error when dispatching a null task

* moved url updates to static function

* added unit tests for event_source url manipulation

* removed array from event bus (it's unsupported)
2022-06-15 11:49:42 -08:00
Hayden
932f4a72df
refactor: remove depreciated repo call (#1370)
* ingredient parser hot fixes (float equality)

* remove `get` in favor of `get_one` & `multi_query`
2022-06-10 19:01:14 -08:00
Michael Genson
b904b161eb
fix: increased float rounding precision for CRF parser (#1369)
* increased float rounding precision for crf parser

* limited fractions to a max denominator of 32 to prevent weirdly specific values

* add test cases for 1/8 and 1/32

* add rounding to avoid more digits than necessary

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-06-10 18:18:31 -08:00