Benchmarks & Profiling
Deterministic latency, throughput, and hardware acceleration benchmarks measured on physical Android ARM64 devices.
1. Measured LLM Inference on Devices (Qwen2.5-0.5B-Instruct, GGUF Q4_K_M)
Measured on physical Samsung Galaxy testbed using llama-completion of the v2.8.0 llama bundle with 4 threads. Prompt processing evaluates ~370 tokens concurrently; token generation evaluates 64 tokens sequentially.
| Device & SoC | Default Route | Prompt Processing (CPU / GPU t/s) | Token Generation (CPU / GPU t/s) |
|---|---|---|---|
| Galaxy S25 (Snapdragon 8 Elite, Adreno 830) | GPU | 30-58 / 1,010-1,081 t/s | 16-49 / 36-77 t/s |
| Galaxy S20 (Snapdragon 865, Adreno 650) | GPU | 60 / 181-197 t/s | 26-27 / 23.6-24.4 t/s (32.9 @ 587MHz) |
| Galaxy A35 (Exynos 1380, Mali-G68 MP5) | GPU | 109-110 / 190 t/s | 35-37 (CPU) / 21.5 t/s (GPU) |
| Galaxy S21 (Exynos 2100, Mali-G78 MP14) | CPU | 115-119 / 227 t/s | 46-47 t/s (CPU) / 22.8 t/s (GPU) |
| Galaxy A53 (Exynos 1280, Mali-G68 MP4) | CPU | 55 / 67 t/s | 22-24 t/s (CPU) / 14.5 t/s (GPU) |
2. Empirical Real-Device STT Benchmarks (Whisper Large-v3-Turbo Q5_0, 548MB)
Test Device: Samsung Galaxy A35 5G (Exynos 1380, Mali-G68 MP5, 8GB RAM, Android 16). Audio input: John F. Kennedy 1-minute benchmark audio (jfk_1min.wav).
| Execution Mode | Target Hardware | Elapsed Time | GPU Clock / Load | CPU Utilization | Speedup |
|---|---|---|---|---|---|
| CPU NEON Mode (4 threads) | Cortex-A78 x4 cores | 816.48s (13m 36s) | 0% (Idle) | 291% (Active) | Baseline |
| Vulkan GPU Mode (Mali Quirk) | Mali-G68 MP5 | 360.60s (6m 00s) | 949 MHz (100%) | 20~30% (Low) | 2.26x (56% time reduction) |
3. AMUDA (AMEVA-Compute) Android SDK v0.3.0 In-Process Benchmarks
In-process Android AAR SDK (no daemon, no shell, runs embedded inside client apps).
| Engine / API | Hardware Backing | Measured Metric | Physical Testbed | Status |
|---|---|---|---|---|
| AmevaSenseVoice | CPU / ONNX Runtime (sherpa-onnx v1.13.8) | Warm RTF: 0.063 ~ 0.068 (S21) Warm RTF: 0.31 ~ 0.37 (A35/A53) |
Galaxy S21, A35, A53 | Verified (15x Real-Time) |
| Auto Backend-Select | Pre-flight GEMV Benchmarking | CPU: 6.77ms • Vulkan: 3.39ms • OpenCL: 2.92ms | Galaxy A35 (Mali-G68) | Auto-Selected OpenCL |
| AmevaSpeech | Vulkan STT Backend (whisper.cpp) | GPU MatMul (f16, q5_0) Dispatch | Galaxy A35, A53 | Verified |
| AmevaDiffusion | stable-diffusion.cpp (SDXS 683MB GGUF) | 512x512 1-Step Distilled txt2img | Galaxy A35 | Verified |
| AmevaLlm.reset() | KV Cache In-Memory Erasure | Latency: < 0.01ms (No reload) | All Devices | Active |
4. Silicon & Driver Defect Resolution (Ground Truth)
-
ARM Mali-G68 Valhall Integer Truncation Infinite Loop Elimination:
loadstride_b = gl_WorkGroupSize.x * LOAD_VEC_B / BK = 16 * 1 / 32 = 0정수 나눗셈 절삭 버그로 인한 무한 루프(GPU Hang / TDR Reset)를 감지하고, Medium MatMul(_m, 작업그룹 크기 128,loadstride_b = 4)을 강제 적용하여 25/25 전 레이어 오프로딩을 안정화하였습니다. -
Qualcomm Adreno 830 JIT 컴파일러 레지스터 할당 버그 격리:
Snapdragon 8 Elite 환경에서 Specialization Constant
NUM_COLS >= 3시 발생하는VK_ERROR_UNKNOWN (-13)컴파일 결함을 분석하고,mul_mat_vec_max_cols = 2경계를 강제하여 음성 입력 시 4,401ms 안정 구동을 달성하였습니다.