Staff Software Engineer, Ray Data
San Francisco
$240k–$270k a yearOn-siteLead & above
Job details
- San Francisco
- On-site
- $240k–$270k a year (posted by the company)
- Full-time · Lead & above
- Engineering
- Posted 2 months ago
Sponsorship history
Past filings by Anyscale in public government data. Not a promise of sponsorship.
- Filed 3 H-1B LCAs in FY2025
Full job description
About Anyscale:
At Anyscale, we're on a mission to democratize distributed computing and make it accessible to software developers of all skill levels. We’re commercializing Ray, a popular open-source project that's creating an ecosystem of libraries for scalable machine learning. Companies like OpenAI, Uber, Spotify, Instacart, Cruise, and many more, have Ray in their tech stacks to accelerate the progress of AI applications out into the real world.
With Anyscale, we’re building the best place to run Ray, so that any developer or data scientist can scale an ML application from their laptop to the cluster without needing to be a distributed systems expert.
Proud to be backed by Andreessen Horowitz, NEA, and Addition with $250+ million raised to date.
About Ray Data Team:
Ray Data is a Python-native data processing engine and a one-stop shop for all AI data processing needs. Ray Data provides performant, first-class integration with cutting-edge AI frameworks using both multimodal and structured data.
The Ray Data team develops and maintains Ray Data, building the underlying distributed data processing infrastructure that powers modern AI workloads. We are a team of engineers passionate about solving challenging problems in distributed systems, data processing, and performance at scale. We are looking for exceptional engineers to build, optimize, and scale Ray Data for increasingly complex AI workloads, including multimodal data processing and large-scale batch inference.
Learn more about how Ray Data enables scalable multimodal AI workloads in Architecting Multimodal Data Pipelines That Scale with Ray.
As part of this role, you will:
- Design, build, and improve the core systems that power Ray Data, with a focus on performance, scalability, and reliability.
- Design and optimize distributed execution across different stages of data pipelines in heterogeneous environments.
- Build data loading and processing solutions for production training and inference workloads.
- Solve challenging problems in distributed execution, scheduling, resource management, data partitioning, fault tolerance, and performance optimization.
- Make system-level architectural decisions and reason through tradeoffs in areas such as resource allocation, execution models, batch vs. streaming workloads, and consistency and availability.
- Work with customers and new-age AI-native companies to understand and solve challenges in scaling their AI workloads.
