Notes on the developer environment with MISE have been added to the Quickstart
Guide. We migrated `.tool-versions` to replace it with `mise` [1], so we can
now remove `asdf` related documentation.
[1] https://github.com/searxng/searxng/pull/5253
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
A Connection object can be used as a context manager that automatically commits
or rolls back open transactions when leaving the body of the context manager.
If the connection attribute ``isolation_level`` is **not** ``None``, new
transactions are implicitly opened before ``execute()`` and ``executemany()``
executes SQL- INSERT, UPDATE, DELETE, or REPLACE statements [1].
The old implementation set ``isolation_level`` to ``None`` and thereby prevented
the context manager from opening and closing the transaction scopes.
[1] https://docs.python.org/3/library/sqlite3.html#sqlite3-transaction-control-isolation-level
[2] https://github.com/searxng/searxng/pull/5239#discussion_r2381416731
Reported-by: Ivan G <igabaldon@inetol.net> [2]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In [1] it can be seen that the bootload of the CURRENCIES cache takes about 2/3
of the time required to initialize SearXNG. Whatever the absolute durations may
be, an explicit bootload during the SearXNG initialization is not required, as
the bootload is already ensured by the API of the CACHE.
- ``CurrenciesDB.name_to_iso4217``
- ``CurrenciesDB.iso4217_to_name``
The fact that the bootload now occurs on-demand should improve the
initialization time of SearXNG.
[1] https://github.com/searxng/searxng/issues/5223#issuecomment-3323083411
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The time for the bootload is measured and recorded. To have an eye on the
bootload time is motivated by: In [1] it can be seen that the bootload of the
CURRENCIES cache takes about 2/3 of the time required to initialize SearXNG::
6.068 <module> searx/webapp.py:1
└─ 6.068 wrapper pyinstrument/context_manager.py:52
├─ 5.538 init searx/webapp.py:1373
│ ├─ 4.822 initialize searx/search/__init__.py:34
│ │ ├─ 4.631 ProcessorMap.init searx/search/processors/__init__.py:47
│ │ │ ├─ 4.607 OnlineCurrencyProcessor.initialize searx/search/processors/online_currency.py:55
│ │ │ │ └─ 4.607 CurrenciesDB.init searx/data/currencies.py:25
│ │ │ │ ├─ 4.601 CurrenciesDB.load searx/data/currencies.py:34
│ │ │ │ │ ├─ 4.572 ExpireCacheSQLite.set searx/cache.py:334
In the example, the CurrenciesDB.init call takes 4.6 seconds... on my laptop,
CurrenciesDB.init takes only 0.7 seconds. The absolute numerical values depend
on external conditions, where I already find 4-5 seconds very long. Test::
$ rm /tmp/sxng_cache_*.db*
$ make run 2>&1 | grep "searx.data.CURRENCIES"
DEBUG searx.data : init searx.data.CURRENCIES
DEBUG searx.data : init searx.data.CURRENCIES added 9089 items in 0.7623255252838135 sec.
[1] https://github.com/searxng/searxng/issues/5223#issuecomment-3323083411
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Mise en place config [1] does no longer support ``.tool-versions``
compatibility syntax, migrate to TOML ``mise.toml``.
In ``utils/lib_sxng_node.sh`` the node version was not updated, update to
24.3.0 (compare ``mise.toml``).
[1] https://mise.jdx.dev/configuration.html
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Issue was introduced by PR-5204 [1]
reported issue::
logger.error("init method of engine %s failed (%s).", eng_proc.engine.name)
TypeError: not enough arguments for format string
[1] https://github.com/searxng/searxng/pull/5204
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Docker buildx outputs the following error:
variable expansion is not supported for --from, define a new stage with FROM
using ARG from global scope as a workaround.
Also force BuildKit extension to be installed, legacy build is no longer
supported.
Closes https://github.com/searxng/searxng/issues/5219
We can leverage the immutable nature of containers to add additional
optimizations.
No debugging or tinkering inside containers, so stripping all unused symbols
inside `venv` should be fine. We are also going to compile the bytecode
ourselves to modify some parameters related to reproducibility.
With these small changes, we have reduced the `venv` layer size by 10MB~
We get some good stuff without uvloop, 13MB~ less of dependencies, 3
minutes of build time for armv7 saved, and we are one step closer to NT
compatibility. Although it's true that theoretically the raw performance
have worsened on network side (we only used uvloop for that), the latest
cpython versions have been improving on asyncio performance.
Revision of the Astrophysics Data System (ADS) engine / use of the result type
Paper as well as other typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Revision of the engine / use of the result type Paper as well as other
typifications.
The engine has been placed on inactive because no service is currently
available, or at least not known in the SearXNG community [1]
[1] https://github.com/searxng/searxng/issues/3610
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch adds a new result type: Paper
- Python class: searx/result_types/paper.py
- Jinja template: searx/templates/simple/result_templates/paper.html
- CSS (less) client/simple/src/less/result_types/paper.less
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Commit 8f8343d [1] introduced a bug in the network logic of SearXNG where stream
requests (such as the one from the image proxy) would fail because a wrapper was
returned instead of a response object with the correct attribute.
This is just a quick in place fix I implemented to get it working again. It
would be better to implement corresponding logic to give stream requests the
correct object.
[1] https://github.com/searxng/searxng/pull/5204
