114 Commits

Author SHA1 Message Date
Michael Genson
d4de15ba1e
Merge branch 'mealie-next' into fix/translation-issues-when-scraping 2024-02-07 09:58:45 -06:00
RealFoxie
e686fa671c
Better bruteforce parsing for units (#3066)
* try to match units when brute parsing and no amount is matched

* brute parser: better handle multiple word food items

Also checks the case when a food might have been split in a unit + ingredient

* fix formatting

* add test cases for ingredient parsing that don't start with an amount

* parametrized tests and added ingredient data fixture

* fixed group_id ref in tests

* fixed test inputs

* add extra tests for units as third token

---------

Co-authored-by: Michael Genson <71845777+michael-genson@users.noreply.github.com>
2024-02-07 09:16:20 -06:00
Michael Genson
634b0590ed
Merge branch 'mealie-next' into fix/translation-issues-when-scraping 2024-02-04 13:20:44 -06:00
Michael Genson
7947aa99ae
fix: Migration Issue With Duplicate Labels (#3085)
* fixed eager queries

* test

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2024-01-31 13:37:12 -06:00
Michael Genson
e1cd2717d3
fix: Update Group Slug When Updating Group (#3084)
* added slug update hook to group updates

* added test

* force refresh if group slug changes

* added alert if something goes wrong
2024-01-30 18:41:37 +00:00
Michael Genson
4ae5c52de9 refactor to use bcrypt directly 2024-01-24 22:03:16 +00:00
Michael Genson
2a5997a968
fix: Foreign Key Violations During Backup Restore (#2986)
* added more test data

* added missing pytest id

* add fk validation to backup restore

* removed bad type imports

* actually apply the invalid fk filter and clean up types

* fix key name

* added log when removing bad rows

* removed unused import

* bumped info to warning
2024-01-16 16:12:20 -06:00
Michael Genson
14497b9b5e
Merge branch 'mealie-next' into fix/translation-issues-when-scraping 2024-01-10 11:35:28 -06:00
Michael Genson
8ca5a9454e
fix: Migration Failure On Postgres Due To Foreign Key Error (#2923)
* added test data to (hopefully) expose fk error

* added additional commits during migration

---------

Co-authored-by: Kuchenpirat <24235032+Kuchenpirat@users.noreply.github.com>
2024-01-08 11:01:44 +00:00
Michael Genson
7602c67449
fix: More Backup Restore Fixes (#2859)
* refactor normalized search migration to use dummy default

* changed group slug migration to use raw SQL

* updated comment

* added tests with anonymized backups (currently failing)

* typo

* fixed LDAP enum in test data

* fix for adding label settings across groups

* add migration data fixes

* fix shopping list label settings test

* re-run db init instead of just running alembic migration, to include fixes

* intentionally broke SQLAlchemy GUID handling

* safely convert between GUID types in different databases

* restore original test data after testing backup restores

* added missing group name update to migration
2024-01-03 04:19:04 +00:00
Michael Genson
e90f05d2dc
Merge branch 'mealie-next' into fix/translation-issues-when-scraping 2024-01-02 19:17:26 -06:00
Michael Genson
ca9f66ee24
feat: Remove OCR Support (#2838)
* remove ocr package

* remove tesseract

* remove OCR from app

* remove OCR from tests

* fix docs
2023-12-14 17:26:43 -09:00
Michael Genson
3a30b3216e fixed tests 2023-12-09 17:19:27 +00:00
Michael Genson
a8583c8e69 added backend translation support for plurals 2023-12-09 17:12:07 +00:00
Hayden
bc575ec5ae
feat: auto detect first login (#2722)
* 'hide' default email and password env variables

* first login API endpoint

* run code-generators

* frontend indicators for default username and pw

* remove old env variables from docs

* fix env set variable

* remove password from tests
2023-11-15 15:24:24 +00:00
Michael Genson
d440d51ffe
feat: plural foods and units, and aliases (#2674)
* added plural names and alias tables to foods/units

* updated models to include plural names and aliases

* updated parser to include plural and aliases

* fixed migrations

* fixed recursive models

* added plural abbreviation to migration

* updated parser and display prop

* update displays to use plurals

* fix display edgecase and remove print

* added/updated display tests

* fixed model bug and added parser tests

* added tests for aliases

* added new plural options to data management page

* removed unique constraint

* made base dialog more customizable

* added alias management to food and unit data pages

* removed unused awaits

* 🧹
2023-11-14 06:39:07 -09:00
Michael Genson
b153ddf858
feat: more shopping list enhancements (#2587)
* fix new position calculataion

* ensure consistent list item ordering

* fix recipe ref overflow on small screens

* added recipe ref elevation

* tweaked line height (for long notes)

* removed unused user dependency

* remove old shopping list items when there's >100

* 🤷

* cleaned up function generator

* fixed test

* fix potential type error

* made max position calc more efficient
2023-10-07 13:06:00 -08:00
Michael Genson
a98e863bca
fix: database unique constraints (#2594)
* fixed missing migration name

* added unique constraints to all m2m tables

* fixed bug trying to create duplicate tags

* added more unique constraints

* fixed duplicate seeder data

* updated tests

* fixed seed rollback error
2023-10-07 11:23:13 -08:00
Matthew Hill
4bd7bda60d
fix: Fix bugs with account locking (#2580)
* fix(security): reset login attempts after successful login

Enforce a maximum number of consecutive failed logins. Successfully logging in should reset the
count.

#2569

* fix(security): fix when user is unlocked

The user should be unlocked when locked_at is set, but the lock has expired.

#2569
2023-09-29 15:58:00 -08:00
Flightkick
bb9afd86c1
fix: Use reserved example.com as bogus instead of email.com domain. (#2551)
`email.com` is not a reserved domain, incorrect configuration could result in unintentional effects.
`example.com` is reserved by IANA for bogus purposes, see RFC 6761.
2023-09-23 07:56:34 -08:00
Michael Genson
2dfbe9f08d
feat: Improved Ingredient Matching (#2535)
* added normalization to foods and units

* changed search to reference new normalized fields

* fix tests

* added parsed food matching to backend

* prevent pagination from ordering when searching

* added extra fuzzy matching to sqlite ing matching

* added tests

* only apply search ordering when order_by is null

* enabled post-search fuzzy matching for postgres

* fixed postgres fuzzy search test

* idk why this is failing

* 🤦

* simplified frontend ing matching
and restored automatic unit creation

* tightened food fuzzy threshold

* change to rapidfuzz

* sped up fuzzy matching with process

* fixed units not matching by abbreviation

* fast return for exact matches

* replace db searching with pure fuzz

* added fuzzy normalization

* tightened unit fuzzy matching thresh

* cleaned up comments/var names

* ran matching logic through the dryer

* oops

* simplified order by application logic
2023-09-15 17:19:34 +00:00
Michael Genson
aec4cb4f31
feat: Advanced Query Filter Record Ordering (#2530)
* added support for multiple order_by strs

* refactored qf to expose nested attr logic

* added nested attr support to order_by

* added tests

* changed unique user to be function-level

* updated docs

* added support for null handling

* updated docs

* undid fixture changes

* fix leaky tests

* added advanced shopping list item test

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2023-09-14 14:09:05 +00:00
Michael Genson
095edef95e
feat: Improve Public URL Readability (#2482)
* added support for group slugs

* modified frontend to use links with group slug

* fixed test refs

* unused import

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2023-08-20 10:38:46 -08:00
Michael Genson
99372aa2b6
feat: Generalize Search to Other Models (#2472)
* generalized search logic to SearchFilter

* added default search behavior for all models

* fix for schema overrides

* added search support to several models

* fix for label search

* tests and fixes

* add config for normalizing characters

* dramatically simplified search tests

* bark bark

* fix normalization bug

* tweaked tests

* maybe this time?

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2023-08-20 10:30:21 -08:00
Jacob Corn
e1d3a247c7
feat: random sort option for front page (#2363)
* Add hook for random sorting

* Add random sorting to front page

* Add multiple tests for random sorting.

* Be extra sure that all recipes are returned.

* Too stable random. seed doesn't reach backend.

* add timestamp to useRecipeSearch

* Update randomization tests for timestamp seeding

* ruff cleanup

* pass timestamp separately in getAll

* remove debugging log items

* remove timestamp from address bar

* remove defaults from backend timestamps

* timestamp should be optional

* fix edge case: query without timestamp

* similar edge case: no timestamp in pagination

* ruff :/

* better edge case handling

* stabilize random search test w/more recipes

* better pagination seeding

* update pagination seed test

* remove redundant random/seed check

* Test for api routes to random sorting.

* please the typing gods

* hack to make query parameters throw correct exc

* ruff

* fix validator message typo

* black reformatting

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2023-05-29 16:56:20 -08:00
Jacob Corn
7e0d29afc7
feat: search tokenization, handling of quoted literal search, and postgres fuzziness (#2351)
* Creating postgres migration script and starting to set up to detect database

* non-working placeholders for postgres pg_tgrm

* First draft of some indexes

* non-working commit of postgres indexing

* Further non-working edits to db-centric fuzzy search

* update alembic for extensions

* More non-working setup

* Move db type check to init_db

* fix typo in db name check

* Add sqlite token search and postgres full text search

* reorder search to hit exact matches faster

* Add settings and docs for POSTGRES_LANGUAGE (full text search)

* Use user-specified POSTGRES_LANGUAGE in search

* fix fuzzy search typo

* Remove full text search and instead order by trigram match

* cleaner adding of indices, remove fulltext

* Cleanup old import of getting app settings

* Fix typo in index

* Fix some alembic fuzzy typos

* Remove diagnostic printing from alembic migration

* Fix mixed up commutator for trigram operator and relax criteria

* forgot to remove query debug

* sort only on name

* token and fuzzy search tests

* Refactor recipe search test to avoid rare random string cross-matches.

* Add ability to quote parts of search for exact match

* Remove internal punctuation, unless it's quoted for literal search

* Add tests for special character removal and literal search

* Remove the outer double quotes from searches, but leave internal single quotes alone.

* Update tests to avoid intra-test name collisions

* Fixing leftovers highlighted by lint

* cleanup linting and mypy errors

* Fix test cross-matching on dirty db (leftovers from bulk import)

* forgot to cleanup something when debugging mypy errors

* re-order pg_trgm loading in postgres

* address comments
2023-05-28 09:46:53 -08:00
Michael Genson
8e2d50054c
Fix: Query Filter Date Comparisons Are Off By One Date (#2389)
* fixed erroneous date -> datetime conversion

* added tests for date and datetime bounds
2023-05-11 22:28:14 -08:00
Michael Genson
5d87b7e411
feature: query filter support for common SQL keywords (#2366)
* added support for SQL keywords IS, IN, LIKE, NOT
deprecated datetime workaround for "<> null"
updated frontend reference for "<> null" to "IS NOT NULL"

* tests

* refactored query filtering to leverage orm

* added CONTAINS ALL keyword

* tests

* fixed bug where "and" or "or" was in an attr name

* more tests

* linter fixes

* TIL this works
2023-05-06 14:28:40 -08:00
Michael Genson
fe17922bb8
Feature: Global Timeline (#2265)
* extended query filter to accept nested tables

* decoupled timeline api from recipe slug

* modified frontend to use simplified events api

* fixed nested loop index ghosting

* updated existing tests

* gave mypy a snack

* added tests for nested queries

* fixed "last made" render error

* decoupled recipe timeline from dialog

* removed unused props

* tweaked recipe get_all to accept ids

* created group global timeline
added new timeline page to sidebar
reformatted the recipe timeline
added vertical option to recipe card mobile

* extracted timeline item into its own component

* fixed apploader centering

* added paginated scrolling to recipe timeline

* added sort direction config
fixed infinite scroll on dialog
fixed hasMore var not resetting during instantiation

* added sort direction to user preferences

* updated API docs with new query filter feature

* better error tracing

* fix for recipe not found response

* simplified recipe crud route for slug/id
added test for fetching by slug/id

* made query filter UUID validation clearer

* moved timeline menu option below shopping lists

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2023-04-25 09:46:00 -08:00
Sören
ccb0b43cef
feat: implement backup restoration from old db schemas (#2213)
* Remove some implicit lazy-loads from user serialization

* implement full backup restore across different database versions
2023-03-12 12:39:51 -08:00
Carter
7d9be67432
feat: LDAP Improvements and E2E testing (#2199)
* add option to enable starttls for ldap

* add integration test for ldap service

* document new, optional environment variable

* fix: support anonymous bind

* id and mail attributes in LDAP_USER_FILTER should be implied

* remove print statement
2023-03-12 12:36:32 -08:00
Sören
84d55eb920
fix: #2169 webhooks not firing (#2178)
* fix webhooks not firing due to missing session

* disable webhook test button because it doesnt do anything

* fix background task administration not working at all

* fix error in test
2023-02-26 10:27:22 -09:00
Carter
2e6ad5da8e
Feature: Add "Authentication Method" to allow existing users to sign in with LDAP (#2143)
* adds authentication method for users

* fix db migration with postgres

* tests for auth method

* update migration ids

* hide auth method on user creation form

* (docs): Added documentation for the new authentication method

* update migration

* add  to auto-form instead of having hidden fields
2023-02-26 10:12:16 -09:00
Michael Genson
6418a10428
fix: misused update over patch in last_update call (#2168)
* fixed mealplan timeline event task
fixed indentation to only look at one group at a time
changed grumpy update to happy patch

* updated pytest to catch this error

* I don't know how this got past the pre-commit
2023-02-23 13:10:47 -09:00
Michael Genson
a6c46a7420
Feature: Shopping List Label Section Improvements (#2090)
* added backend for shopping list label config

* updated codegen

* refactored shopping list ops to service
removed unique contraint
removed label settings from main route/schema
added new route for label settings

* codegen

* made sure label settings output in position order

* implemented submenu for label order drag and drop

* removed redundant label and tweaked formatting

* added view by label to user preferences

* made items draggable within each label section

* moved reorder labels to its own button

* made dialog scrollable

* fixed broken model

* refactored labels to use a service
moved shopping list label logic to service
modified label seeder to use service

* added tests

* fix for first label missing the tag icon

* fixed wrong mapped type

* added statement to create existing relationships

* fix restore test, maybe
2023-02-21 18:58:41 -09:00
Michael Genson
05e2566c35
fix: recipe scraper image cleaning (#2139)
* updated image cleaner
enabled image cleaner
added case for nested image dicts

* refactored image cleaner to return a list of urls
2023-02-19 15:43:52 -09:00
Sören
6a5f9d7f6b
feat: diacritic-insensitive search (#2132)
* add normalized columns and use them for search

* add migration to fill all normalized columns
2023-02-19 15:40:18 -09:00
Hayden
71f8c1066a
feat: server side search (#2112) (#2117)
* feat: server side search API (#2112)

* refactor repository_recipes filter building

* add food filter to recipe repository page_all

* fix query type annotations

* working search

* add tests and make sure title matches are ordered correctly

* remove instruction matching again

* fix formatting and small issues

* fix another linting error

* make search test no rely on actual words

* fix failing postgres compiled query

* revise incorrectly ordered migration

* automatically extract latest migration version

* test migration orderes

* run type generators

* new search function

* wip: new search page

* sortable field options

* fix virtual scroll issue

* fix search casing bug

* finalize search filters/sorts

* remove old composable

* fix type errors

---------

Co-authored-by: Sören <fleshgolem@gmx.net>
2023-02-11 21:26:10 -09:00
Sören
7d4a379f0d
feat: improve database indexing (#2104)
* add indices to all foreign keys and some fields that are used for ordering and filtering

* add missing migrations

* update migration orders

---------

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2023-02-11 10:40:53 -09:00
Carter
da60e56982
fix: user login and creation with LDAP (#2107)
* Corrected if statement to check if a results was returned by the LDAP search. And decoded the user_attributes from binary data to string

* removed trailing spaces

* Revert asserts in LDAP unit test back

Since an empty tuple is still a result, an user is created and the result should not be false.

* Simplified code

* Extended the LDAP implementation

* fix ldap authentication and user creation

* modified docs to include new LDAP environment variables

* update tests and linting

* add libldap-2.4-2 as runtime dependency for the api

---------

Co-authored-by: Erik Landkroon <eriklandkroon@gmail.com>
2023-02-11 10:16:33 -09:00
Michael Genson
5f7ac92c96
feat: timeline event for mealplans (#2050)
* added related user to mealplans

* made timeline event message actually optional

* added task to create events for mealplan recipes

* replaced fk constraint ops with bulk ops

* fixed event creation and adjusted query range

* indentation is hard

* added missing recipe id query filter

* added tests
2023-02-11 10:08:53 -09:00
Hayden
9e77a9f367
prs-fleshgolem-2070: feat: sqlalchemy 2.0 (#2096)
* upgrade sqlalchemy to 2.0

* rewrite all db models to sqla 2.0 mapping api

* fix some importing and typing weirdness

* fix types of a lot of nullable columns

* remove get_ref methods

* fix issues found by tests

* rewrite all queries in repository_recipe to 2.0 style

* rewrite all repository queries to 2.0 api

* rewrite all remaining queries to 2.0 api

* remove now-unneeded __allow_unmapped__ flag

* remove and fix some unneeded cases of "# type: ignore"

* fix formatting

* bump black version

* run black

* can this please be the last one. okay. just. okay.

* fix repository errors

* remove return

* drop open API validator

---------

Co-authored-by: Sören Busch <fleshgolem@gmx.net>
2023-02-06 18:43:12 -09:00
Michael Genson
2340ee5bfb
Dev Improvements (#2058)
* added ruff cache to git ignore

* moved venv to project and added interpreter path

* added dummy method to LDAP test

* removed unused setting
2023-01-28 15:49:09 -09:00
Sören
3415a9c310
Convert scraper to use async (#1915)
* add httpx depedency for async http requests

* rework scraper strategies to download recipe html asynchronously

* rework recipe_data_service to download recipe images asynchronously

* fix recipe_parser test, so it can use async results

* fix bulk import so that it also works with async scraper

* fix broken recipe_parser tests

* Fix issues found by scanners

* Add additional checks for ingredient and instruction count in test_create_by_url

* Revert changes in test recipe_data
Since we are checking ingredients and instructions in test_create_url now, these would fail with the stored html of recipe data

* Add explicit type annotation in recipe_data_service.largest_content_len

* Fix typo in annotation
2023-01-28 15:43:27 -09:00
Hayden
4d5550ad85
chore: mypy, ruff, extract, openapi, and recipes-scraper updates (#1968)
* bump ruff and mypy

* mypy fixes

* bump and fix openapi

* drop debug statement

* bump extruct

* bump recipes-scrapers
2023-01-01 13:47:27 -09:00
Michael Genson
46cc3898ab
feat: Improved Recipe Scaling Support and Shopping List Recipe Refactor (#1847)
* propogate scale changes to print view

* fixed incorrect variable reference

* refactored shopping list recipe routes
cleaned up existing logic
added support for recipe scaling

* updated current revision

* adding to shopping list respects UI recipe scale

* added field annotations

* added tests for recipe scaling

* made column nullable and set to 1 during migration
2022-12-30 13:47:35 -09:00
Michael Genson
a2dcdc1adf
feat: "I Made This" Dialog (#1801)
* added chef hat

* removed unnecessary log

* modified recipe and recipe timeline event schema
changed timeline event "message" -> "event_message"
added "last made" timestamp to recipe

* added "I made this" dialog to recipe action menu

* added missing field and re-ran code-gen

* moved dialog out of context menu and refactored
removed references in action menu and context menu
refactored dialog to be triggered by a button instead
added route to update recipe last made timestamp
added visual for last made timestamp to recipe header and title

* added sorting by last made

* switched event type to comment

* replaced alter column with pydantic alias

* added tests for event message alias
2022-11-13 14:12:53 -09:00
Hayden
89d0cae51d
refactor: rewrite cleaner functions for parsing recipe dicts (#1743)
* rewrite cleaner functions

* unify verbage

* try importing dep during check

* fix syntax

* allow override defaults

* satisfy mypy
2022-11-10 15:16:51 -09:00
Michael Genson
6ee64535df
feat: recipe timeline backend api (#1685)
* added recipe_timeline_events table to db

* added schema and routes for recipe timeline events

* added missing mixin and fixed update schema

* added tests

* adjusted migration revision tree

* updated alembic revision test

* added initial timeline event for new recipes

* added additional tests

* added event bus support

* renamed event_dt to timestamp

* add timeline_events to ignore list

* run code-gen

* use new test routes implementation

* use doc string syntax

* moved event type enum from db to schema

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>
2022-11-01 00:12:26 -08:00
Hayden
a8f0fb14a7
chore: upgrade pre-commit hooks (#1735)
* change pep585 hook to pyupgrade

* run pyupgrade + cleanup lint errors
2022-10-17 14:37:06 -08:00