Cutlery/searxng
1
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2025-11-01 11:07:08 -04:00
Code Issues Packages Projects Releases Wiki Activity

[mod] debug-log the bootload of the CURRENCIES caches (CurrenciesDB)

Browse Source 
The time for the bootload is measured and recorded.  To have an eye on the
bootload time is motivated by: In [1] it can be seen that the bootload of the
CURRENCIES cache takes about 2/3 of the time required to initialize SearXNG::

    6.068 <module>  searx/webapp.py:1
    └─ 6.068 wrapper  pyinstrument/context_manager.py:52
       ├─ 5.538 init  searx/webapp.py:1373
       │  ├─ 4.822 initialize  searx/search/__init__.py:34
       │  │  ├─ 4.631 ProcessorMap.init  searx/search/processors/__init__.py:47
       │  │  │  ├─ 4.607 OnlineCurrencyProcessor.initialize  searx/search/processors/online_currency.py:55
       │  │  │  │  └─ 4.607 CurrenciesDB.init  searx/data/currencies.py:25
       │  │  │  │     ├─ 4.601 CurrenciesDB.load  searx/data/currencies.py:34
       │  │  │  │     │  ├─ 4.572 ExpireCacheSQLite.set  searx/cache.py:334

In the example, the CurrenciesDB.init call takes 4.6 seconds... on my laptop,
CurrenciesDB.init takes only 0.7 seconds.  The absolute numerical values depend
on external conditions, where I already find 4-5 seconds very long. Test::

    $ rm /tmp/sxng_cache_*.db*
    $ make run 2>&1 | grep "searx.data.CURRENCIES"
    DEBUG   searx.data : init searx.data.CURRENCIES
    DEBUG   searx.data : init searx.data.CURRENCIES added 9089 items in 0.7623255252838135 sec.

[1] https://github.com/searxng/searxng/issues/5223#issuecomment-3323083411

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser 2025-09-24 12:23:32 +02:00 committed by Markus Heiser
parent 87bc97776a
commit 62b0b3f697
1 changed files with 7 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
searx/data/currencies.py
View File

@ -6,6 +6,7 @@ __all__ = ["CurrenciesDB"]
import typing as t import typing as t
import json import json
import pathlib import pathlib
import time
from .core import get_cache, log from .core import get_cache, log
@ -32,6 +33,7 @@ class CurrenciesDB:
# in /tmp and will be rebuild during the reboot anyway # in /tmp and will be rebuild during the reboot anyway
def load(self): def load(self):
_start = time.time()
log.debug("init searx.data.CURRENCIES") log.debug("init searx.data.CURRENCIES")
with open(self.json_file, encoding="utf-8") as f: with open(self.json_file, encoding="utf-8") as f:
data_dict: dict[str, dict[str, str]] = json.load(f) data_dict: dict[str, dict[str, str]] = json.load(f)
@ -39,6 +41,11 @@ class CurrenciesDB:
self.cache.set(key=key, value=value, ctx=self.ctx_names, expire=None) self.cache.set(key=key, value=value, ctx=self.ctx_names, expire=None)
for key, value in data_dict["iso4217"].items(): for key, value in data_dict["iso4217"].items():
self.cache.set(key=key, value=value, ctx=self.ctx_iso4217, expire=None) self.cache.set(key=key, value=value, ctx=self.ctx_iso4217, expire=None)
log.debug(
"init searx.data.CURRENCIES added %s items in %s sec.",
len(data_dict["names"]) + len(data_dict["iso4217"]),
time.time() - _start,
)
def name_to_iso4217(self, name: str) -> str | None: def name_to_iso4217(self, name: str) -> str | None:
self.init() self.init()