Complete API Reference
Python SDK, Environment Variables, and C ABI (ggml-ameva backend)
Python SDK — ameva_runtime
| Symbol | Signature / Type | Description |
ameva_runtime.list_backends() | () → list[str] | Returns the backends available in this build, e.g. ['vulkan', 'cpu'] or ['vulkan', 'opencl', 'cpu']. |
ameva_runtime.get_device_info(backend: str = 'vulkan') | (backend: str) → dict | Returns {"vendor": str, "device_name": str, "driver_version": str, "api_version": str, "total_vram_mb": int}. Raises AmevaBackendError if the backend is unavailable. |
ameva_runtime.run_llm(model_path: str, prompt: str, **kwargs) | (model_path, prompt, n_predict=128, n_threads=4, ctx_size=2048, backend='auto') → str | High-level inference call. Selects the best available backend automatically when backend='auto'. Returns the generated text string. |
ameva_runtime.vulkan | module | Low-level Vulkan-specific submodule. Exposes vulkan.dispatch_gemv(), vulkan.compile_kernel(), and vulkan.get_vk_device(). |
Environment Variables (Runtime Configuration)
| Variable | Values | Default | Description |
AMEVA_BACKEND | vulkan | opencl | cpu | vulkan | Forces a specific compute backend. Set to cpu to diagnose GPU path issues without recompiling. |
AMEVA_MMQ | 0 | 1 | 1 (where supported) | Enables integer dot-product GEMM kernel (ameva_mmq). Auto-disabled on Mali r38 and below which lack the extension. Measured 2.0–2.6× prompt speedup on S25/S20/A35. |
AMEVA_FUSE | 0 | 1 | 0 | Experimental operation fusion (RMS_NORM + gate merge). No statistically significant speedup measured across the 5-device fleet; disabled by default. |
AMEVA_CL_WAIT | spin | yield | block | spin | OpenCL completion wait strategy. spin is the default; block allows idle CPU cores to sleep but was observed to cause 2× slowdown on Mali when all CPU cores yield. |
AMEVA_PROFILE | 0 | 1 | 0 | Enables per-operation GPU timing output to stderr. Reports dispatch time, kernel selection, and buffer transfer cost per MUL_MAT op. |
AMEVA_CL_SOURCE_DIR | path | (built-in) | Path to a directory containing OpenCL kernel source files (*.cl). Used during development to override embedded kernels without recompiling the shared library. |
AMEVA_MMQB | 0 | 1 | 0 | Enables block-integer accumulation kernels (ameva_mmqb) for Q4_K. Measured 1.45× speedup on A53 OpenCL and 2.1–2.7× on A35 OpenCL. No benefit on A53 GEMV (memory-bound path). |
C ABI — ggml-ameva backend (libggml-ameva.so)
| Symbol | Signature | Description |
ameva_abi_check | (uint32_t host_ggml_version) → int | Called at load time by the loader. Returns 0 if the library's ggml ABI version is compatible with host_ggml_version. Returns non-zero and prints a diagnostic on mismatch; the loader must reject the library in this case. |
ggml_backend_ameva_init | () → ggml_backend_t | Initialises and returns the AMEVA ggml backend. Called once per process. Returns NULL if no compatible GPU is found; the caller must not treat NULL as a CPU fallback silently. |
ggml_backend_ameva_buffer_type | () → ggml_backend_buffer_type_t | Returns the buffer type for AMEVA-managed GPU memory allocations. |
ggml_backend_is_ameva | (ggml_backend_t backend) → bool | Type-checks a backend pointer. Use before calling AMEVA-specific functions to avoid undefined behaviour on non-AMEVA backends. |
Supported ggml Quantisation Types
| Type | GPU GEMV | GPU GEMM (tiled) | Notes |
| Q4_K_M / Q4_K_S | ✅ ameva_qmv | ✅ ameva_qgemm | Default for most production deployments. OpenCL block-int path available (AMEVA_MMQB=1). |
| Q5_K_M / Q5_K_S | ✅ ameva_qmv | ✅ ameva_qgemm | Short-loop fix applied (write rows out) to prevent 2× performance regression on OpenCL. |
| Q6_K | ✅ ameva_qmv | ✅ ameva_qgemm | Same short-loop fix as Q5_K. Memory-bound on generation path. |
| Q8_0 | ✅ ameva_qmv | ✅ ameva_qgemm | KV cache quantisation. GPU and CPU outputs are identical; no quality degradation. |
| F16 | ✅ ameva_qmv | ✅ (conv2d tiled path) | 5–7× faster than the GEMV-oriented path when dispatched as 1×1 CONV_2D. |
| F32 | ✅ ameva_qmv | ✅ ameva_qgemm | Reference precision path. Used for dequantisation comparison and parity testing. |