Associate Director, Software Engineering (Guardrail Platform Control Plane & Observability Track)

Some careers have more impact than others.

If you’re looking for a career where you can make a real impression, join HSBC and discover how valued you’ll be.

We are currently seeking an experienced professional to join our team in the role of Associate Director, Software Engineering (Guardrail Platform Control Plane & Observability Track).

Business: CTO Platform (AI Platforms)

Job ID: 53442

Location: Guangzhou / Xi'an

Principal responsibilities

1. Design and build the Guardrail Control Plane to manage AI safety policies, detector configurations, evaluation workflows, and enforcement strategies across enterprise AI applications.

2. Develop scalable and extensible guardrail platform capabilities including policy lifecycle management, detector registry, configuration management, versioning, rollout/rollback, and governance workflow, and runtime decision management.

3. Design and impllement system architecture and APIs to enable seamless integration between Guardrail Control Plane, Runtime Engine, AI Gateway, RAG, Agent, and other AI platform components.

4. Build AI platform observability capabilities providing end-to-end visibility across AI workloads, including request lifecycle tracking, safety decisions, model interactions, performances metrics, cost insight and operational analytics.

5. Design and implement LLM tracing solutions using OpenTelemerty / OpenInference to capture AI execution flows including prompts, models calls, retrieval, embedding, tool calls, guardrail checks, and final responses.

6. Develop AI safety evaluation and improvement frameworks including datasets management, regression testing pipeline, LLM- as-a-jedge evaluation, human feedback workflows, and production quality monitoring.

7. Own hands-on engineering delivery including system design, coding, testing, trobleshooting, performance optimization, and production operation of scalable Guardrail Service.

Requirements

1. Bachelor’s degree or above in Computer Science or related discipline, with 8+ years of professional software engineering experience with strong hands-on experience building production-grade backend platforms and distributed systems.

2. Proven experience designing and implementing platform systems such as API platforms, workflow platforms, governance platforms, or internal developer platforms.

3. Strong system design skills with experience in service architecture, domain modeling, API design, configuration-driven systems, version management, and extensibility patterns.

4. Strong hands-on programming experience with Python (FastAPI) and/or Go, with ability to independently design, implement, and operate backend services.

5. Experience building configuration management systems, policy engines, workflow systems, rule engines, or platform control planes.

6. Experience designing observability platforms including distributed tracting, telemetry pipelines, metrics, dashboards, alerting systems and operational analytics.

7. Hands-on experience with LLM observability concepts including prompt tracing, model tracing, retrieval tracing, tool-call tracing, agent workflow tracing, and AI quality monitoring.

8. Experience with OpenTelemetry, OpenInference, or similar tracing frameworks.

9. Experience integrating AI observability platforms such as Arize Phoenix, Langfuse, LangSmith, MLflow, or similar tools.

10. Experience with Kubernetes/Docker and at least one public cloud platform (AWS/GCP/Azure).

Tech Stack

Python / Go, FastAPI

Kubernetes / Docker

Redis / PostgreSQL

Kafka / PubSub

WebSocket / SSE / gRPC

OpenTelemetry

GCP / AWS / Azure

You’ll achieve more when you join HSBC.

HSBC is an equal opportunity employer committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and, opportunities to grow within an inclusive and diverse environment. We encourage applications from all suitably qualified persons irrespective of, but not limited to, their gender or genetic information, sexual orientation, ethnicity, religion, social status, medical care leave requirements, political affiliation, people with disabilities, color, national origin, veteran status, etc., We consider all applications based on merit and suitability to the role.

Personal data held by the Bank relating to employment applications will be used in accordance with our Privacy Statement, which is available on our website.

***Issued By HSBC Software Development (GuangDong) Limited***