From c109e28686f86388d7e095218c0bcfbc499e868e Mon Sep 17 00:00:00 2001 From: yoni13 Date: Mon, 13 Jan 2025 00:44:22 +0800 Subject: [PATCH] DOCS --- docs/docs/features/ml-hardware-acceleration.md | 10 ++++++++++ docs/docs/install/environment-variables.md | 4 ++++ 2 files changed, 14 insertions(+) diff --git a/docs/docs/features/ml-hardware-acceleration.md b/docs/docs/features/ml-hardware-acceleration.md index fdf6149ed9..838b0abccc 100644 --- a/docs/docs/features/ml-hardware-acceleration.md +++ b/docs/docs/features/ml-hardware-acceleration.md @@ -10,6 +10,7 @@ You do not need to redo any machine learning jobs after enabling hardware accele ## Supported Backends - ARM NN (Mali) +- RKNN (Rockchip) - CUDA (NVIDIA GPUs with [compute capability](https://developer.nvidia.com/cuda-gpus) 5.2 or higher) - OpenVINO (Intel discrete GPUs such as Iris Xe and Arc) @@ -34,6 +35,15 @@ You do not need to redo any machine learning jobs after enabling hardware accele - The `hwaccel.ml.yml` file assumes an additional file `/lib/firmware/mali_csffw.bin`, so update accordingly if your device's driver does not require this file - Optional: Configure your `.env` file, see [environment variables](/docs/install/environment-variables) for ARM NN specific settings +#### RKNN + +- You must have a supported Rockchip SoC, only RK3566 and RK3588 are supported at this moment. +- Make sure you have the appropriate linux kernel driver installed + - This is usually pre-installed on the device vendor's Linux images +- RKNPU driver V0.9.8 or later must be available in the host server + - You may confirm this by running `cat /sys/kernel/debug/rknpu/version` to check the version +- Optional: Configure your `.env` file, see [environment variables](/docs/install/environment-variables) for RKNN specific settings + #### CUDA - The GPU must have compute capability 5.2 or greater. diff --git a/docs/docs/install/environment-variables.md b/docs/docs/install/environment-variables.md index 1f34b5c6d0..bdcd1b64c4 100644 --- a/docs/docs/install/environment-variables.md +++ b/docs/docs/install/environment-variables.md @@ -166,6 +166,10 @@ Redis (Sentinel) URL example JSON before encoding: | `MACHINE_LEARNING_ANN_TUNING_LEVEL` | ARM-NN GPU tuning level (1: rapid, 2: normal, 3: exhaustive) | `2` | machine learning | | `MACHINE_LEARNING_DEVICE_IDS`\*4 | Device IDs to use in multi-GPU environments | `0` | machine learning | | `MACHINE_LEARNING_MAX_BATCH_SIZE__FACIAL_RECOGNITION` | Set the maximum number of faces that will be processed at once by the facial recognition model | None (`1` if using OpenVINO) | machine learning | +| `MACHINE_LEARNING_RKNN` | Enable RKNN hardware acceleration if supported | `True` | machine learning | +| `MACHINE_LEARNING_RKNN_TEXTUAL_THREADS` | How many threads of RKNN runtime should be spinned up while infrencing textual model. | 1 | machine learning | +| `MACHINE_LEARNING_RKNN_VISUAL_THREADS` | How many threads of RKNN runtime should be spinned up while infrencing visual model. | 1 | machine learning | +| `MACHINE_LEARNING_RKNN_TEXTUAL_THREADS` | How many threads of RKNN runtime should be spinned up while infrencing textual model. | 1 | machine learning || `MACHINE_LEARNING_RKNN_FACIAL_THREADS` | How many threads of RKNN runtime should be spinned up while infrencing facial model. | 1 | machine learning | \*1: It is recommended to begin with this parameter when changing the concurrency levels of the machine learning service and then tune the other ones.