We are looking for a senior-level engineer to focus on high-performance model inference across PC and Android platforms. The role centers on optimizing LLM/multimodal models for low latency and efficient memory use, implementing C++ runtimes, applying advanced acceleration techniques, and collaborating closely with research teams to bring optimized inference solutions into production environments.
Key Responsibilities
? Design and implement optimized model inference pipelines for PC (x86/AMD/Intel) and Android (ARM).
? Apply quantization, operator/kernal fusion, memory optimization, and runtime scheduling techniques.
? Work with at least one major inference stack: llama.cpp, Qualcomm AI SDKs (QNN/QAIRT/QSDK) or MTK Neuro Pillot; better to have experience with OpenVINO, Ryzen AI, and other inference SDKs.
? Profile and tune CPU/GPU/NPU performance using industry-standard profiling tools.
? Collaborate with model researchers to translate new methods into efficient runtime implementations.
Required Qualifications
? Master’s degree or above, with 3+ years experience in model inference, runtime engineering, or performance optimization.
? Strong C++ programming skills; familiarity with Android NDK/JNI is a plus.
? Solid understanding of transformer architectures, inference mechanisms, and acceleration methods.
? Hands-on experience with at least one of: llama.cpp, Qualcomm AI SDKs (QNN/QAIRT/QSDK) or MTK Neuro Pillot; better to have experience with OpenVINO, Ryzen AI, and other inference SDKs.
? Ability to read English technical papers and documentation; English communication preferred.
Preferred
? Experience with ONNX Runtime, TVM, XNNPack, or mobile performance tools.
? Contributions to open-source inference or optimization frameworks.