Cutlery/searxng
1
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2026-02-23 03:30:07 -05:00
Code Issues Packages Projects Releases Wiki Activity
9,210 Commits 5 Branches 0 Tags
Commit Graph

559 Commits

Author SHA1 Message Date
github-actions[bot]
15133c2f25
[data] update searx.data - update_firefox_version.py (#5699) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2026-01-29 10:29:28 +01:00
github-actions[bot]
a7f15f4289
[data] update searx.data - update_wikidata_units.py (#5700) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2026-01-29 10:28:54 +01:00
github-actions[bot]
ac4b1275a3
[data] update searx.data - update_currencies.py (#5702) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2026-01-29 10:28:22 +01:00
github-actions[bot]
bf2e7f8fc5
[data] update searx.data - update_engine_descriptions.py (#5703) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2026-01-29 10:27:07 +01:00
github-actions[bot]
cdbf41bb9b
[data] update searx.data - update_ahmia_blacklist.py (#5701) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2026-01-29 10:24:33 +01:00
mg95
cf74e1d9e9
[fix] google: switch to using GSA for iPhone useragent 2026-01-11 16:21:01 +01:00
github-actions[bot]
a5c946a321
[data] update searx.data - update_external_bangs.py (#5607) 2025-12-30 08:29:05 +01:00
searxng-bot
9491b514c9 [data] update searx.data - update_engine_descriptions.py 2025-12-29 15:24:06 +01:00
searxng-bot
ce46f30739 [data] update searx.data - update_currencies.py 2025-12-29 13:48:16 +01:00
searxng-bot
65a95539f1 [data] update searx.data - update_ahmia_blacklist.py 2025-12-29 13:47:12 +01:00
searxng-bot
874dc3f5ea [data] update searx.data - update_firefox_version.py 2025-12-29 13:46:57 +01:00
searxng-bot
7941719371 [data] update searx.data - update_wikidata_units.py 2025-12-29 13:46:46 +01:00
github-actions[bot]
c954e71f87
[data] update searx.data - update_engine_descriptions.py (#5496) 2025-11-29 16:04:34 +01:00
searxng-bot
cb4a5abc8c [data] update searx.data - update_currencies.py 2025-11-29 14:54:09 +00:00
searxng-bot
07ff6e3ccc [data] update searx.data - update_wikidata_units.py 2025-11-29 09:00:05 +00:00
searxng-bot
cdaab944b4 [data] update searx.data - update_firefox_version.py 2025-11-29 08:58:56 +00:00
searxng-bot
6ecf32fd4a [data] update searx.data - update_ahmia_blacklist.py 2025-11-29 08:58:26 +00:00
Markus Heiser
9c2b8f2f93 [data] update searx.data - update_ahmia_blacklist.py 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-10-30 07:40:20 +01:00
github-actions[bot]
dce383881d
[data] update searx.data - update_firefox_version.py (#5388) 2025-10-29 07:15:11 +01:00
github-actions[bot]
1ebedcbc17
[data] update searx.data - update_wikidata_units.py (#5389) 2025-10-29 07:14:31 +01:00
github-actions[bot]
5d99877d8d
[data] update searx.data - update_currencies.py (#5390) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2025-10-29 07:13:53 +01:00
github-actions[bot]
adc1a2a1ea
[data] update searx.data - update_engine_descriptions.py (#5391) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2025-10-29 07:13:19 +01:00
Markus Heiser
33e798b01b
[fix] TrackerPatternsDB.clean_url: don't delete query argument from new_url (#5339) 
The query argument for URLs like:

- 'http://example.org?q='       --> query_str is 'q='
- 'http://example.org?/foo/bar' --> query_str is 'foo/bar'

is a *simple string* and not a key/value dict.  This string may only be removed
from the URL if one of the patterns matches.

BTW get_pretty_url(): keep such a *simple string* in the path element.

Closes: https://github.com/searxng/searxng/issues/5299

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-10-20 11:20:33 +02:00
searxng-bot
0f3ef5da59 [data] update searx.data - update_engine_traits.py 2025-09-30 16:48:59 +02:00
github-actions[bot]
77fd3ee534
[data] update searx.data - update_currencies.py (#5261) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2025-09-29 07:37:30 +02:00
github-actions[bot]
b8e60542d9
[data] update searx.data - update_engine_descriptions.py (#5262) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2025-09-29 07:36:51 +02:00
github-actions[bot]
5513549adc
[data] update searx.data - update_wikidata_units.py (#5259) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2025-09-29 07:35:10 +02:00
github-actions[bot]
cd2ea7c5f5
[data] update searx.data - update_ahmia_blacklist.py (#5257) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2025-09-29 07:34:38 +02:00
searxng-bot
38b80133f7 [data] update searx.data - update_firefox_version.py 2025-09-29 07:33:19 +02:00
Markus Heiser
18a58943cc [mod] ExpireCacheSQLite - implement .setmany() for bulk loading 
[1] https://github.com/searxng/searxng/issues/5223#issuecomment-3328597147

Suggested-by: Ivan G <igabaldon@inetol.net> [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-09-28 07:32:41 +02:00
Markus Heiser
62b0b3f697 [mod] debug-log the bootload of the CURRENCIES caches (CurrenciesDB) 
The time for the bootload is measured and recorded.  To have an eye on the
bootload time is motivated by: In [1] it can be seen that the bootload of the
CURRENCIES cache takes about 2/3 of the time required to initialize SearXNG::

    6.068 <module>  searx/webapp.py:1
    └─ 6.068 wrapper  pyinstrument/context_manager.py:52
       ├─ 5.538 init  searx/webapp.py:1373
       │  ├─ 4.822 initialize  searx/search/__init__.py:34
       │  │  ├─ 4.631 ProcessorMap.init  searx/search/processors/__init__.py:47
       │  │  │  ├─ 4.607 OnlineCurrencyProcessor.initialize  searx/search/processors/online_currency.py:55
       │  │  │  │  └─ 4.607 CurrenciesDB.init  searx/data/currencies.py:25
       │  │  │  │     ├─ 4.601 CurrenciesDB.load  searx/data/currencies.py:34
       │  │  │  │     │  ├─ 4.572 ExpireCacheSQLite.set  searx/cache.py:334

In the example, the CurrenciesDB.init call takes 4.6 seconds... on my laptop,
CurrenciesDB.init takes only 0.7 seconds.  The absolute numerical values depend
on external conditions, where I already find 4-5 seconds very long. Test::

    $ rm /tmp/sxng_cache_*.db*
    $ make run 2>&1 | grep "searx.data.CURRENCIES"
    DEBUG   searx.data : init searx.data.CURRENCIES
    DEBUG   searx.data : init searx.data.CURRENCIES added 9089 items in 0.7623255252838135 sec.

[1] https://github.com/searxng/searxng/issues/5223#issuecomment-3323083411

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-09-28 07:32:41 +02:00
Markus Heiser
81cbe0befe
[upd] pypi: Bump black from 24.3.0 to 25.9.0 (#5251) 
In 25.1.0 [2] an old bug has been fixed: "Docstring formatting does not apply to
module docstrings" [3].

[1] https://github.com/psf/black/blob/main/CHANGES.md#2590
[2] https://github.com/psf/black/blob/main/CHANGES.md#2510
[3] https://github.com/psf/black/issues/4094

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-09-26 12:35:57 +02:00
Markus Heiser
8f8343dc0d [mod] addition of various type hints / engine processors 
Continuation of #5147 .. typification of the engine processors.

BTW:

- removed obsolete engine property https_support
- fixed & improved currency_convert
- engine instances can now implement a engine.setup method

[#5147] https://github.com/searxng/searxng/pull/5147

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-09-18 19:40:03 +02:00
Markus Heiser
f24d85bc4b [mod] drop: from __future__ import annotations 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-09-03 13:37:36 +02:00
Markus Heiser
57b9673efb [mod] addition of various type hints / tbc 
- pyright configuration [1]_
- stub files: types-lxml [2]_
- addition of various type hints
- enable use of new type system features on older Python versions [3]_
- ``.tool-versions`` - set python to lowest version we support (3.10.18) [4]_:
  Older versions typically lack some typing features found in newer Python
  versions.  Therefore, for local type checking (before commit), it is necessary
  to use the older Python interpreter.

.. [1] https://docs.basedpyright.com/v1.20.0/configuration/config-files/
.. [2] https://pypi.org/project/types-lxml/
.. [3] https://typing-extensions.readthedocs.io/en/latest/#
.. [4] https://mise.jdx.dev/configuration.html#tool-versions

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Format: reST
 2025-09-03 13:37:36 +02:00
github-actions[bot]
fde4fac3ca
[data] update searx.data - update_external_bangs.py (#5158) 2025-08-29 07:07:46 +02:00
github-actions[bot]
a9263fd063
[data] update searx.data - update_wikidata_units.py (#5159) 2025-08-29 07:07:00 +02:00
github-actions[bot]
146cefe7fa
[data] update searx.data - update_firefox_version.py (#5160) 2025-08-29 07:05:08 +02:00
github-actions[bot]
2a7d26b714
[data] update searx.data - update_ahmia_blacklist.py (#5161) 2025-08-29 07:04:41 +02:00
github-actions[bot]
480914d736
[data] update searx.data - update_currencies.py (#5162) 2025-08-29 07:04:12 +02:00
github-actions[bot]
edc50768cc
[data] update searx.data - update_engine_traits.py (#5163) 2025-08-29 07:03:29 +02:00
github-actions[bot]
a519a13bfa
[data] update searx.data - update_engine_descriptions.py (#5164) 2025-08-29 07:02:31 +02:00
Markus Heiser
664aab0ec9
[fix] CI task "update_engine_traits.py" fails (#5069) 
* [fix] CI task "update_engine_traits.py" fails

To catch all problems with an HTTP request, the more general class
``httpx.HTTPError`` must be caught, for your test use::

    $ ./manage dev.env
    $ python ./searxng_extra/update/update_engine_traits.py

Closes: https://github.com/searxng/searxng/issues/5068

* [data] update searx.data - update_engine_traits.py
 2025-08-01 12:08:27 +02:00
github-actions[bot]
40b78ad06c
[data] update searx.data - update_wikidata_units.py (#5062) 2025-07-29 07:26:01 +02:00
github-actions[bot]
db83a39544
[data] update searx.data - update_external_bangs.py (#5061) 2025-07-29 07:25:26 +02:00
github-actions[bot]
bb3bea829d
[data] update searx.data - update_ahmia_blacklist.py (#5064) 2025-07-29 07:24:09 +02:00
github-actions[bot]
dc9ad0a493
[data] update searx.data - update_currencies.py (#5065) 2025-07-29 07:23:38 +02:00
github-actions[bot]
5db7b70dc7
[data] update searx.data - update_engine_descriptions.py (#5066) 2025-07-29 07:22:58 +02:00
github-actions[bot]
2ad35421d7
[data] update searx.data - update_firefox_version.py (#5063) 
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
 2025-07-29 07:22:21 +02:00
mggh0139
54a2b553f4
[fix] tracker pattern: let startup continue if url fetch fails (#5055) 
Use Python exception to prevent startup crash in case of fetch ClearURL
failure. Also add some logs.

Closes: https://github.com/searxng/searxng/issues/5054
 2025-07-28 07:03:01 +02:00