Full Stack Engineer, Charles River Development, Assistant Vice President

Senior Production Reliability Engineer

We are seeking a highly technicalSeniorProduction Reliability Engineer toleadourlocalExcellenceinEngineering team. This isnot a software development role: its focus isthe reliability, diagnosability and operational effectiveness of complex, largely non-cloud production systems.The successful candidate will combinedeep knowledgeof system architecture,data flows and databases with advanced debugging, monitoring and troubleshooting skills.They will lead difficult investigations through to root cause, drive AI-enabled automation and operational improvement, and provide technical direction and mentoring to the local team. Infrastructure, capacity and performance are important supporting areas, but cloud-platform engineering is not the primary remit.

Key Responsibilities:

Advanced Incident Investigation & Problem Management

• Lead complex, high-impact and multi-system production investigations from triage and containment through recovery, root cause and corrective action.

• Diagnose issues across applications, services, operating systems, databases, messaging and networks using logs, traces, metrics, configuration, queries and controlled reproduction.

• Retain end-to-end ownership while coordinating effectively with software engineering, product, infrastructure and customer-facing teams.

AI, Automation & Operational Improvement

• Champion practical use of AI and automation to accelerate incident investigation, correlate telemetry, surface likely causes and recommend next actions.

• Automate diagnostics, health checks, evidence collection, recovery activities and recurring operational workflows using scripts, tools and governed AI-assisted solutions.

• Build searchable operational knowledge from incidents, runbooks, architecture information and known errors, and measure reductions in manual effort and time to recovery.

• Improve system supportability by influencing logging, telemetry, diagnostic interfaces and non-functional engineering requirements.

System Architecture, Data Flows & Databases

• Maintain a deep understanding of end-to-end architecture, component responsibilities, dependencies and critical business flows.

• Trace transactions and data across APIs, services, message brokers, batch processes and databases to isolate defects and performance bottlenecks.

• Diagnose database query, locking, contention, indexing, connectivity, data-quality and throughput issues, and identify resilience gaps and operational risks.

Local Team Leadership& Operational Readiness

• Provide day-to-day technical leadership for the local team, setting priorities, allocating investigations and maintaining high standards of ownership and follow-through.

• Coach and mentor engineers in architecture, debugging, observability, automation and effective incident practice, while fostering collaboration with global teams.

Observability,Monitoring& Reliability

• Help develop actionable observability across services, databases, interfaces and infrastructure using platforms such as Dynatrace.

• Help direct improvements to dashboards, alerts and diagnostic views covering service health, dependencies, latency, throughput, saturation and errors.

• Help define meaningful indicators and thresholds, reduce alert noise and use telemetry to shorten diagnosis and recovery times.

Skills, Experience & Qualifications Required:

• 4–7 years of progressively responsible experience in site reliability, production engineering, application support, systems engineering or a similarly deep technical operations role, ideally in financial services.

• Proven ability to lead complex investigations across distributed systems using structured diagnosis and root-cause analysis.

• Strong understanding of system architecture, APIs, microservices, messaging, data flows and multi-tier failure modes.

• Strong relational database and SQL skills, including query performance, locking, indexing, connectivity and data integrity.

• Hands-on experience with observability platforms such as Dynatrace and with logs, metrics, traces, dashboards and alerts.

• Strong commitment to leveraging AI and automation safely and pragmatically to improve diagnostics, operations and reliability outcomes.

• Scripting capability in PowerShell, Python, Bash or SQL; working knowledge of Java, C#, Kubernetes, Kafka and API gateways is advantageous, but regular feature development is not expected.

• Understanding of performance, capacity, resilience, networking and operating-system fundamentals; on-premises or hybrid experience is strongly valued, while Azure familiarity is useful but secondary.

• Experience providing technical leadership, prioritising team activity, mentoring engineers and coordinating across engineering, product, infrastructure and customer-facing teams.