Senior Software Engineer - Distributed Inference

NVIDIA • California, California • 7h ago

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It's a unique legacy of innovation that's fueled by great technology-and amazing people. Today, we're tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what's never been done before takes vision, innovation, and the world's best talent. As an NVIDIAN, you'll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.

We are now looking for a Senior System Software Engineer to work on user facing tools for Dynamo Inference Server! NVIDIA is hiring software engineers for its GPU-accelerated deep learning software team, and we are a remote friendly work environment. Academic and commercial groups around the world are using GPUs to power a revolution in deep learning, enabling breakthroughs in problems from LLM, image classification to speech recognition to natural language processing. We are a fast-paced team building tools and software to make the design and deployment of new deep learning models easier and accessible to more data scientists.

What you'll be doing:

Build and maintain distributed model management systems, including Rust-based runtime components, for large-scale AI inference workloads.

Implement inference scheduling and deployment solutions on Kubernetes and Slurm, while driving advances in scaling, orchestration, and resource management.

Collaborate with infrastructure engineers and researchers to develop scalable APIs, services, and end-to-end inference workflows.

Create monitoring, benchmarking, automation, and documentation processes to ensure low-latency, robust, and production-ready inference systems on GPU clusters.

What we need to see:

Bachelor's, Master's, or PhD in Computer Science, ECE, or related field (or equivalent experience).

6+ years of professional systems software development experience.

Strong programming expertise in Rust (with C++, Python as a plus).

Deep knowledge of distributed systems, runtime orchestration, and cluster-scale services.

Hands-on experience with Kubernetes, container-based microservices, and integration with Slurm.

Proven ability to excel in fast-paced R&D environments and collaborate across functions.

Ways to stand out from the crowd:

Experience with inference-serving frameworks (e.g., Dynamo Inference Server, TensorRT, ONNX Runtime) and deploying/managing LLM inference pipelines at scale.

Contributions to large-scale, low-latency distributed systems (open-source preferred) with proven expertise in high-availability infrastructure.

Strong background in GPU inference performance tuning, CUDA-based systems, and operating across cloud-native and hybrid environments (AWS, GCP, Azure).

NVIDIA has continuously reinvented itself over three decades. Our invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI - the next era of computing. We are widely considered to be the leader of AI computing, and one of the technology world's most desirable employers. We have some of the most forward-thinking and committed people in the world working for us. If you're creative and autonomous, we want to hear from you!

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5.

You will also be eligible for equity and benefits (https://www.nvidia.com/en-us/benefits/) .

Applications for this job will be accepted at least until August 30, 2025.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

IT & Engineering

Description We're seeking a skilled Systems Engineer to join our team and help build resilient, secure, and scalable IT infrastructure. If you're passionate about virtualization, networking, and disaster recovery, this i...

1d ago

Go to Apply

Software Engineer

CACI International • Virginia, Virginia • 1d ago

IT & Engineering

Software Engineer Job Category: Information Technology Time Type: Full time Minimum Clearance Required to Start: Secret Employee Type: Regular Percentage of Travel Required: Up to 10% Type of Travel: Continental US The O...

1d ago

Go to Apply

Network Engineer - Wireless - Secret Clearance

MILVETS Systems Technology, Inc. • Virginia, Virginia • 1d ago

IT & Engineering

Network Engineer - Wireless - Secret Clearance NNSY, Portsmouth, VA Full Time Experienced Share ( https://milvetssystemstechnologyinc.applytojob.com/app/share/yoIHN7VD38 ) Job Title: Network Engineer - Wireless - Secret...

1d ago

Go to Apply

IT Cybersecurity Specialist (INFOSEC)

Washington Headquarters Services • Virginia, Virginia • 1d ago

IT & Engineering

Summary This position serves as an IT Cyber Security Specialist in the Washington Headquarters Services (WHS) Information and Technology Management Office, as such the incumbent is responsible for overseeing, managing, c...

1d ago

Go to Apply

Associate Analyst, Data-Remote in California

Molina Healthcare • California, California • 3d ago

IT & Engineering

JOB DESCRIPTION Job Summary Designs and implements processes and solutions associated with a wide variety of data sets used for data/text mining, analysis, modeling, and predicting to enable informed business decisions....

3d ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.