Complete API Reference

Python SDK, Environment Variables, and C ABI (ggml-ameva backend)

Python SDK — ameva_runtime

SymbolSignature / TypeDescription
ameva_runtime.list_backends()() → list[str]Returns the backends available in this build, e.g. ['vulkan', 'cpu'] or ['vulkan', 'opencl', 'cpu'].
ameva_runtime.get_device_info(backend: str = 'vulkan')(backend: str) → dictReturns {"vendor": str, "device_name": str, "driver_version": str, "api_version": str, "total_vram_mb": int}. Raises AmevaBackendError if the backend is unavailable.
ameva_runtime.run_llm(model_path: str, prompt: str, **kwargs)(model_path, prompt, n_predict=128, n_threads=4, ctx_size=2048, backend='auto') → strHigh-level inference call. Selects the best available backend automatically when backend='auto'. Returns the generated text string.
ameva_runtime.vulkanmoduleLow-level Vulkan-specific submodule. Exposes vulkan.dispatch_gemv(), vulkan.compile_kernel(), and vulkan.get_vk_device().

Environment Variables (Runtime Configuration)

VariableValuesDefaultDescription
AMEVA_BACKENDvulkan | opencl | cpuvulkanForces a specific compute backend. Set to cpu to diagnose GPU path issues without recompiling.
AMEVA_MMQ0 | 11 (where supported)Enables integer dot-product GEMM kernel (ameva_mmq). Auto-disabled on Mali r38 and below which lack the extension. Measured 2.0–2.6× prompt speedup on S25/S20/A35.
AMEVA_FUSE0 | 10Experimental operation fusion (RMS_NORM + gate merge). No statistically significant speedup measured across the 5-device fleet; disabled by default.
AMEVA_CL_WAITspin | yield | blockspinOpenCL completion wait strategy. spin is the default; block allows idle CPU cores to sleep but was observed to cause 2× slowdown on Mali when all CPU cores yield.
AMEVA_PROFILE0 | 10Enables per-operation GPU timing output to stderr. Reports dispatch time, kernel selection, and buffer transfer cost per MUL_MAT op.
AMEVA_CL_SOURCE_DIRpath(built-in)Path to a directory containing OpenCL kernel source files (*.cl). Used during development to override embedded kernels without recompiling the shared library.
AMEVA_MMQB0 | 10Enables block-integer accumulation kernels (ameva_mmqb) for Q4_K. Measured 1.45× speedup on A53 OpenCL and 2.1–2.7× on A35 OpenCL. No benefit on A53 GEMV (memory-bound path).

C ABI — ggml-ameva backend (libggml-ameva.so)

SymbolSignatureDescription
ameva_abi_check(uint32_t host_ggml_version) → intCalled at load time by the loader. Returns 0 if the library's ggml ABI version is compatible with host_ggml_version. Returns non-zero and prints a diagnostic on mismatch; the loader must reject the library in this case.
ggml_backend_ameva_init() → ggml_backend_tInitialises and returns the AMEVA ggml backend. Called once per process. Returns NULL if no compatible GPU is found; the caller must not treat NULL as a CPU fallback silently.
ggml_backend_ameva_buffer_type() → ggml_backend_buffer_type_tReturns the buffer type for AMEVA-managed GPU memory allocations.
ggml_backend_is_ameva(ggml_backend_t backend) → boolType-checks a backend pointer. Use before calling AMEVA-specific functions to avoid undefined behaviour on non-AMEVA backends.

Supported ggml Quantisation Types

TypeGPU GEMVGPU GEMM (tiled)Notes
Q4_K_M / Q4_K_S✅ ameva_qmv✅ ameva_qgemmDefault for most production deployments. OpenCL block-int path available (AMEVA_MMQB=1).
Q5_K_M / Q5_K_S✅ ameva_qmv✅ ameva_qgemmShort-loop fix applied (write rows out) to prevent 2× performance regression on OpenCL.
Q6_K✅ ameva_qmv✅ ameva_qgemmSame short-loop fix as Q5_K. Memory-bound on generation path.
Q8_0✅ ameva_qmv✅ ameva_qgemmKV cache quantisation. GPU and CPU outputs are identical; no quality degradation.
F16✅ ameva_qmv✅ (conv2d tiled path)5–7× faster than the GEMV-oriented path when dispatched as 1×1 CONV_2D.
F32✅ ameva_qmv✅ ameva_qgemmReference precision path. Used for dequantisation comparison and parity testing.