Roles / AI Performance Benchmarking Engineer

AI Performance Benchmarking Engineer

AIS-009v1

AI/ML Silicon Software Stack & Co-DesignSoftwareIC3–IC6Individual contributor

Also known as: MLPerf Engineer; AI Benchmark Engineer; Competitive Performance Analyst

See this role on the AI accelerator lifecycle or on the SoC lifecycle

Summary

Produces the defensible, reproducible performance numbers that the company publishes and that customers use to choose silicon, covering MLPerf submissions, competitive analysis and internal performance tracking.

A day in the life

Work in progressNo day-in-the-life description has been written for this role yet. If you do this work, we would welcome yours. Suggest a source or a correction

Responsibilities

Project

Work done for a product or program.

Instrument power and thermal measurement so performance-per-watt claims survive third-party scrutiny.
Triage and bisect performance regressions to the specific compiler, kernel, runtime or driver commit responsible.
Maintain reproducibility artefacts including container images, seeds, exact model checkpoints and system BIOS settings.
Support marketing and product management with defensible claim language and technical review of published material.
Model projected performance for unreleased silicon and validate projections against first measured results.
Define benchmark acceptance gates that must pass before a software release ships.

Function

Work that keeps the discipline running: methods, tools, suppliers and people.

Prepare, tune and submit MLPerf Training, Inference, Client and Storage results under MLCommons rules and audit review.
Build automated benchmark harnesses that run the tracked model suite nightly across every supported configuration.
Run competitive benchmarking on rival accelerators and produce apples-to-apples comparison methodology.
Measure and report throughput, latency percentiles, tokens per second per watt, and total cost of ownership metrics.

Major tasks

Project

Produce and hand over the role's key deliverables
- Audited MLPerf submission packages and result logs
- Automated nightly benchmark harness and performance dashboard
- Competitive benchmarking report with documented methodology
- Performance-per-watt and TCO analysis for target workloads
- Release performance gate criteria and sign-off reports
Develop the softwareGeneral
- Design to the hardware specification
- Write and review the code
- Test on models, emulation or silicon
Release and supportGeneral
- Package and document releases
- Fix customer-reported defects

Function

Maintain the platformGeneral
- Keep build, test and release tools working
- Track toolchain and operating system changes
Raise qualityGeneral
- Run code reviews
- Extend automated tests

Works with

Kernel engineers
compiler engineers
runtime engineers
product marketing
product management
hardware performance architects
MLCommons working groups

Typical backgrounds

BS/MS in Computer Engineering, Computer Science or Physics; 3–10 years in performance engineering, systems benchmarking or HPC operations.

Demand signal

0 open listings right now
0 Talent Wafer employer inquiries

Counts live listings tagged to this role, and employer interest in candidates from Talent Wafer runs for this role (all runs to date). Hub activity only, not the wider market.

Comp tier (within semis)

3 / 5

Remote feasibility

Medium

Semis core vs. portable

Semis-primary, transfers

Geographic hotspots

Santa Clara, Austin, Portland, Bangalore, Shanghai, Hsinchu, Munich, Tel Aviv

Certifications / standards training

—

Transfers to (industries)

Cloud infrastructure capacity planning, HPC procurement and national labs, storage and networking vendors, technology analyst firms

Risk if vacant

The company cannot publish credible competitive numbers or catch performance regressions, so claims are challenged publicly and releases silently lose throughput.

AI Performance Benchmarking Engineer

Summary

A day in the life

Responsibilities

Project

Function

Major tasks

Project

Function

Works with

Register or sign in to read on