Technical Director, Large-Scale AI Model Inferencing
San Jose, California, United States
$219k–$351k a yearOn-siteLead & above
Job details
- San Jose, California, United States
- On-site
- $219k–$351k a year (posted by the company)
- Full-time · Lead & above
- Memory Solutions Lab (MSL)
- Posted 2 months ago
Sponsorship history
Past filings by Samsung Semiconductor in public government data. Not a promise of sponsorship.
- Filed 24 H-1B LCAs in FY2025
Full job description
Please Note:
To provide the best candidate experience amidst our high application volumes, each candidate is limited to 10 applications across all open jobs within a 6-month period.
Advancing the World’s Technology Together
Our technology solutions power the tools you use every day--including smartphones, electric vehicles, hyperscale data centers, IoT devices, and so much more. Here, you’ll have an opportunity to be part of a global leader whose innovative designs are pushing the boundaries of what’s possible and powering the future.
We believe innovation and growth are driven by an inclusive culture and a diverse workforce. We’re dedicated to empowering people to be their true selves. Together, we’re building a better tomorrow for our employees, customers, partners, and communities.
What You’ll Do
Inference is becoming a memory-bandwidth business. As models scale past what any single GPU can hold — KV caches grow with context, MoE expert weights spill beyond HBM, and new architectures change the rules of what "model state" even means — the winners will be the companies that treat memory as the core product of AI inference, not an afterthought.
We are looking for a Hands-on Principal Engineer who combines deep, first-principles knowledge of AI model architectures (dense Transformers, Mixture-of-Experts, State Space Models, and hybrids) with production-scale inference expertise, to own the requirement for full-stack AI memory solutions at scale — spanning GPU HBM, host DRAM, CXL-attached memory pools, and NVMe/SSD tiers and Samsung Cognos, AI memory software that moves model state intelligently across them.
This person will be the technical authority who connects model behavior to memory-system design: someone who can explain why an MoE router's activation pattern dictates an LRU expert cache policy, why a Mamba state cache breaks the assumptions of PagedAttention, and why disaggregated prefill/decode changes the required memory bandwidth per token by an order of magnitude — and then build the products that exploit those facts.
Level: Principal Engineer
Team: Memory Solutions Lab / Data Fabric Solutions
Reports to: Chief Technologist, Memory Solutions Lab
Location: Daily onsite presence preferred at our San Jose office/headquarters in alignment with our Flexible Work policy; remote/hybrid option available.
Job ID: 43027
Model Architecture Expertise — The Foundation
- Serve as expert on how different model families consume and move memory, and translate that into memory-product requirements:
