Просмотр вакансии

We are looking for LLM/ ML Infrastructure engineer experienced with Rust/C++ and CUDA for remote work.

Our client is building a decentralized AI infrastructure focused on running and serving ML models directly on user-owned hardware (on-prem / edge environments).

A core component of the product is a proprietary “capsule” runtime for deploying and running ML models. Currently, some components rely on popular open-source solutions (e.g., llama.cpp). Still, the strategic goal is to replace community-driven components with in-house ML infrastructure to gain complete control over performance, optimization, and long-term evolution.

In parallel, the company is developing:

• its own network for generating high-quality, domain-specific datasets,

• fine-tuned compact models for specialized use cases,

• a research track focused on ranking, aggregation, accuracy improvements, and latency reduction.

The primary target audience is B2B IT companies.

The long-term product vision is to move beyond generic code generation and focus on high-performance, hardware-aware, and efficiency-optimized code generation.

ML Direction

1. Applied ML Track (Primary focus for this role)

• Development of ML inference infrastructure

• Building and evolving proprietary runtime capsules

• Porting and implementing ML algorithms on a custom architecture

• Low-level performance optimization across hardware platforms

2. Research Track

• ML research with published papers

• Improvements in answer quality and inference efficiency

• Experiments with aggregation, ranking, and latency reduction

This position is primarily focused on the applied ML / engineering track.

Role

This is a strongly engineering-oriented ML role focused on inference, performance, and systems-level implementation rather than model experimentation.

Approximately 90% of the work is hands-on coding and optimization.

You will

• Implement ML algorithms from research papers into production-ready code

• Port existing ML inference algorithms to the company’s proprietary architecture

• Develop and optimize inference

• Optimize performance, memory usage, and latency

• Integrate and adapt open-source ML solutions (LLaMA, VLMs, llama.cpp, etc.)

• Contribute to the foundational architecture of the ML platform

Key Responsibilities

Inference Infrastructure Development:

Design and implementation of a cross-platform engine for ML model inference

Development of low-level components in Rust and C++ with focus on maximum performance

Creation and integration of APIs for interaction with the inference engine

Performance Optimization:

Implementation of modern optimization algorithms: Flash Attention, PagedAttention, continuous batching

Development and optimization of CUDA kernels for GPU-accelerated computations

Profiling and performance tuning across various GPU architectures

Optimization of memory usage and model throughput

Model Operations:

Implementation of efficient model quantization methods (GPTQ, AWQ, GGUF)

Development of memory management system for working with large language models

Integration of support for various model architectures (LLaMA, Mistral, Qwen, and others)

We are waiting from you

• Strong proficiency in Rust or C++

• Hands-on experience with GPU / hardware acceleration, including:

  • CUDA, AMD or Metal (Apple Silicon)

• Solid understanding of:

  • LLM principles

  • core ML algorithms

  • modern ML approaches used in production systems

• Ability to read ML research papers and implement them in code

• Ability to write clean, efficient, highly optimized code

• Interest in systems-level ML and low-level performance optimization

• High level of autonomy:

  • take existing algorithms from research or open-source,

  • understand them deeply,

  • adapt and integrate them into a new architecture

• Fruent English

What The Company Offers

• Remote-first setup (work from anywhere)

• Dubai working hours

• High level of ownership and autonomy

• Flat structure

• Salary in cryptocurrency

• An opportunity to create a great product that will break the AI market