Senior Software Engineer, AI Operations, GPS
Scale AI · Doha, Qatar · 1mo ago
Role Overview
As a Senior Software Engineer, AI Ops at Scale AI, you will own the long-term technical health, performance, and stability of AI solutions deployed across our strategic public sector partners.
While our Delivery Teams build and launch new use cases, you are the technical steward ensuring these deployments operate with Operational Excellence. You will bridge software engineering, MLOps, and client governance, managing tiered SLAs, tracking model drift, and executing maintenance protocols that protect both system integrity and operational margins.
Key Responsibilities
- Handover Gate & Onboarding: Act as the technical gatekeeper during the formal transition from Delivery to Maintenance. Conduct deep-dive reviews to ensure baseline code, prompts, and architecture meet strict maintainability and documentation standards before sign-off.
- Tiered SLA & Incident Management: Own technical response and resolution targets across multi-tiered service models (from Business-Hours Essential to 24/7 Mission-Critical). Lead Incident Governance, Root Cause Analysis (RCA), and P1/P2 mitigations within strict active support windows.
- AI Lifecycle Governance: Monitor production model performance, latency, and data drift. Manage prompt configuration repositories to maintain behavioral consistency and perform regression testing when LLM providers update underlying endpoints.
- Request Classification & Technical Scope: Operationalize the boundary between Routine Maintenance (In-Scope) and System Evolution (Out-of-Scope). Assess incoming client requests and run comparative benchmarking on new AI models.
- Automation & Reliability Engineering: Eliminate operational toil by engineering self-healing data pipelines, automated RAG indexing syncs, and telemetry tooling. Influence upstream "Delivery" teams to adopt architectural patterns that simplify ongoing maintenance.
- Client Technical Interface: Serve as the senior technical point of contact for government and enterprise IT leads. Translate technical AI concepts (data drift, prompt versioning, API deprecation) into clear business impacts for non-technical stakeholders.
Ideally you'd have
- Background: 5+ years in Software Engineering, MLOps, SRE, or Forward Deployed Engineering in heavy data or production AI environments.
- Technical Stack: Advanced proficiency in Python, SQL, REST/gRPC APIs, and cloud architecture (AWS, Azure, or GCP). Hands-on experience with MLOps tooling, vector databases, and LLM orchestration frameworks (e.g., LangChain, LlamaIndex).
- AI Governance Expertise: Practical understanding of prompt version control, model benchmarking against evaluation datasets, RAG pipeline mechanics, and data drift detection.
- Engineering Mindset: A drive to build systematic, automated fixes rather than applying temporary patches. Strong grasp of CI/CD for machine learning pipelines.
- Client Acumen & Boundary Control: Strong technical communication skills with the ability to manage client expectations, defend operational boundaries (Maintenance vs. Evolution), and advise on long-term system roadmaps.
For those applying based in Qatar: Residency and employment in Qatar requires certain permissions (visa and permits) issued by the Qatari authorities. As part of the application process, candidates will be asked to provide personal information, including residency status and nationality, which is required for visa processing. This information is collected solely for immigration compliance purposes and will be not used as a criterion in any hiring decision unless such use is lawfully permitted. Visa issuance is at the discretion of the Qatari authorities. If you are successful in your application, you will be required to provide the documentation requested by Scale and the authorities to obtain the necessary permissions for you to live and work in Qatar, and Scale will work with successful candidates to support the visa application process.
PLEASE NOTE: Our policy requires a 90-day waiting period before reconsidering candidates for the same role. This allows us to ensure a fair and thorough evaluation of all applicants.
About Us:
At Scale, our mission is to develop reliable AI systems for the world's most important decisions. Our products provide the high-quality data and full-stack technologies that power the world's leading models, and help enterprises and governments build, deploy, and oversee AI applications that deliver real impact. We work closely with industry leaders like Meta, Ernst & Young, Mayo Clinic, Time Inc., the Government of Qatar, and U.S. government agencies including the Army and Air Force. We are expanding our team to accelerate the development of AI applications.
We believe that everyone should be able to bring their whole selves to work, which is why we are proud to be an inclusive and equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability status, gender identity or Veteran status.
We are committed to working with and providing reasonable accommodations to applicants with physical and mental disabilities. If you need assistance and/or a reasonable accommodation in the application or recruiting process due to a disability, please contact us at accommodations@scale.com. Please see the United States Department of Labor's Know Your Rights poster for additional information.
We comply with the United States Department of Labor's Pay Transparency provision.
PLEASE NOTE: We collect, retain and use personal data for our professional business purposes, including notifying you of job opportunities that may be of interest and sharing with our affiliates. We limit the personal data we collect to that which we believe is appropriate and necessary to manage applicants’ needs, provide our services, and comply with applicable laws. Any information we collect in connection with your application will be treated in accordance with our internal policies and programs designed to protect personal data. Please see our privacy policy for additional information.