Senior Product Manager for AI Observability
LSEG · London, United Kingdom · 4mo ago
Role Profile
As part of theLSEG AI, we are hiring aSeniorProduct Manager– AI Observabilityto own the strategy, design and rollout of telemetry systems thatmonitor,measureandanalysehow AI models, MCPs, and AI-enabled features behave across all LSEG products.
This roleis responsible fordefining how we collect, structure,analyseand act on AI-specific telemetry signals—including prompt patterns, model performance, MCP call usage, latency, error conditions, cost metrics, userbehavioursignals, and system reliability.
You will develop a telemetry foundation that supports:
AI governance & risk
model performance tracking
cost efficiency
user experienceoptimisation
operational reliability
auditability
and the long-term evolution of our AI platform.
Key Responsibilities & Accountabilities
Telemetry & Observability Strategy
Define the end-to-endtelemetry vision and roadmapfor LLMs, MCPs, vector stores, embeddings, inferencelayersand AI-powered user experiences.
Establish astandardisedtelemetry schemafor capturing prompts, tool calls, model responses, durations, errors, confidence signals, and quality indicators across business lines.
Partner with platform engineering to ensure instrumentation is consistent,scalableand compliant.
Signal Design & Data Architecture
Identifykey signalsrequiredfor:
Quality & reliability
Latency & throughput
Cost tracking &optimisation
MCP usage patterns
User workflow insights
Failure pattern detection
Guardrail and safety event monitoring
Define retention rules, PII considerations,anonymisationand usage policies in partnership with AI Governance and Compliance.
Platform & Tooling
Own requirements for dashboards,monitoringtools, model comparison views, anomaly detection alerts, and performance scorecards.
Design systems that allow product and engineering teams toself-serveinsights about AI model behavior and MCP interactions.
Partner with engineering to build pipelines that supportreal-time and batch analytics.
Cross-Functional Collaboration
Collaborate with product owners across LSEG toinstrumenttheir AI features consistently.
Partner with AI Evaluation PM (previousrole), Model Risk, GSSR, Legal and Compliance to align telemetry with governance frameworks.
Work closely with Engineering and SRE teams to drive observability improvements and reliability engineering for AI systems.
Optimisation& Insights
Identifycost inefficiencies across model and MCP usage, and drive recommendations to improve ROI.
Surface workflow-level insights on how customers use AI features—informing product roadmaps.
Partner with product leaders across divisions to understand how telemetry can improve customer experience,reliabilityand performance.
Governance & Standards
Define andmaintainAI telemetry standards and best practices across all LSEG divisions.
Contribute to LSEG’s AI governance and Responsible AI initiatives byprovidingdata-driven insights on model behavior and user impact.
Ensure telemetry systems support auditability,complianceand explainability requirements.
Skills & Competencies
Required
Experience in product management witha strong foundationinobservability, telemetry, data platforms, monitoring, or SRE / DevOps-driven products.
Understanding ofLLMs, embeddings, vector search, MCP tools, and AI inference workflows.
Deep familiarity withlogging, tracing, metrics, and event-based telemetry systems.
Ability to define data schemas, signal taxonomies, aggregationstrategiesand data contracts.
Strong analytical skills and ability to derive insights from large-scale system telemetry.
Experience working with senior engineering, data science,riskand governance stakeholders.
Preferred
Exposure tofinancial data, analyticssystemsand enterprise-scale data workflows.
Familiarity with BI tools, metrics stores, distributedtracingand monitoring stacks.
Understanding ofcloud infrastructure, serverless runtimes, APIgatewaysand model hosting architectures.
Ability to drive cross-functional alignment andestablishgroup-wide standards.
Measures of Success
Platform Reliability & Performance
Reduction in AI system error rates, latencyspikesand operational incidents.
Stability and transparency of MCP usage across product lines.
Quality and coverage of telemetry signals across all AI-enabled products.
Insight Generation & Adoption
Adoption of telemetry dashboards and self-service tools across divisions.
Number ofidentifiedand resolved model or MCP reliability issues driven by telemetry.
Level of insight generated to influence productprioritisationand AI model choices.
Operational Efficiency & Cost
Improved cost transparency and cost-saving impact via telemetry-drivenoptimisation.
Reduction in unnecessary model invocation or inefficient tool usage.
Governance & Compliance
Contribution to Responsible AI frameworks through robust, auditable telemetry.
Improved explainability and traceability of AI features for internal/external stakeholders.
Career Stage:
ManagerLondon Stock Exchange Group (LSEG) Information:
Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.
LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.
Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership, Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.
Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.
We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyone’s race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.
You will be part of a collaborative and creative culture where we encourage new ideas. We are committed to sustainability across our global business and we are proud to partner with our customers to help them meet their sustainability objectives. Our charity, the LSEG Foundation provides charitable grants to community groups that help people access economic opportunities and build a secure future with financial independence. Colleagues can get involved through fundraising and volunteering.
LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.
Please take a moment to read this privacy notice carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what it’s used for, and how it’s obtained, your rights and how to contact us as a data subject.
If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility to ensure that candidates applying to LSEG are aware of this privacy notice.