AI Analytics Manager

At eBay, we're more than a global ecommerce leader — we’re changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We’re committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work — every day. We're in this together, sustaining the future of our customers, our company, and our planet.

Join a team of passionate thinkers, innovators, and dreamers — and help us connect people and build communities to create economic opportunity for all.

Analytics is undergoing a fundamental shift—from traditional business intelligence and data science toward AI-native, conversational analytics products.

We are building a new generation of analytics experiences where business and product teams can interact with data through natural language, diagnose performance, uncover root causes, and take action with the support of AI agents.

We are looking for a technical, product-minded Data Scientist / AI practitioner to help build these experiences, with a particular focus on analytics chatbots, AI orchestration, retrieval and grounding, agentic workflows, and evaluation.

If you are motivated to move beyond traditional analysis into building intelligent systems that combine LLMs, enterprise data, analytical context, and business workflows, this is an opportunity to help shape how eBay interacts with data in the AI era.

About the Role

In this role, you will design and develop the intelligence layer behind AI-powered analytics chatbots and agentic analytics products.

You will work across the end-to-end conversational analytics stack—including LLM orchestration, intent understanding, retrieval, context engineering, tool calling, SQL/data access, analytical reasoning, response generation, and evaluation.

A key part of the role will be designing systems that can translate natural-language questions into reliable analytical workflows: identifying the right data and business context, retrieving relevant knowledge, invoking analytical tools, reasoning across results, and returning grounded and explainable answers.

You will partner closely with Analytics, Data Engineering, Product, and Engineering teams, as well as Legal, Privacy, Security, and Responsible AI, to move AI capabilities from experimentation into reliable enterprise products.

What You’ll Do

Build Conversational & Agentic Analytics Systems

• Develop AI-powered analytics chatbots and conversational interfaces that allow users to explore business performance using natural language.

• Design multi-step agentic workflows that can interpret user intent, retrieve context, query data, perform analytical reasoning, and generate grounded responses.

• Build orchestration flows connecting LLMs with SQL engines, analytics APIs, semantic layers, knowledge bases, and other analytical tools.

• Design tool-calling patterns and agent routing strategies that determine which data sources, analytical workflows, or specialized agents should handle a request.

• Develop structured response patterns supporting analytical outputs such as narratives, tables, charts, follow-up questions, and recommended actions.

• Translate ambiguous analytical questions into deterministic and AI-assisted workflows that are reliable enough for enterprise decision-making.

Design the Retrieval & Context Layer

• Build retrieval architectures across structured and unstructured analytics knowledge, including metric definitions, schemas, SQL examples, dashboards, analytical documentation, historical analyses, and business context.

• Design RAG and context-engineering workflows using techniques such as semantic search, embeddings, metadata filtering, hybrid retrieval, reranking, and dynamic context assembly.

• Develop strategies for retrieving structured context—including relevant schemas, tables, dimensions, metrics, and example SQL—to support accurate data querying.

• Optimize knowledge organization, chunking, deduplication, indexing, and retrieval strategies for analytical use cases.

• Design mechanisms to maintain and continuously update domain-specific analytics knowledge as definitions, schemas, and business logic evolve.

Develop AI Orchestration & Analytical Reasoning

• Design LLM orchestration pipelines spanning intent classification, planning, retrieval, tool selection, query generation, execution, reasoning, and response synthesis.

• Develop single-agent and multi-agent architectures, including specialized agents for different analytical domains or tasks.

• Build workflows for natural-language-to-SQL, query validation, data retrieval, and grounded interpretation of query results.

• Implement model routing and fallback strategies based on task complexity, latency, accuracy, context-window requirements, and cost.

• Design conversation-state and context-management approaches for multi-turn analytical interactions.

• Implement guardrails, validation layers, structured outputs, error handling, and fallback behaviors to improve system reliability.

Build Evaluation & Continuous Improvement Systems

• Develop evaluation frameworks for accuracy, groundedness, retrieval quality, SQL correctness, analytical reasoning, latency, cost, and user usefulness.

• Build golden evaluation datasets containing representative user questions, expected queries, analytical outputs, and business context.

• Create automated regression testing for prompts, retrieval configurations, models, and orchestration changes.

• Evaluate individual components of the chatbot pipeline—including retrieval, tool selection, SQL generation, reasoning, and final-answer quality.

• Analyze user interactions, feedback, failure patterns, and agent traces to identify opportunities for product and system improvements.

• Build proactive and reactive feedback loops that continuously improve analytics knowledge, prompts, retrieval, and agent behavior.

Productionize AI Analytics Capabilities

• Develop Python-based services and reusable components supporting conversational analytics and agent workflows.

• Partner with engineering teams to integrate AI capabilities with APIs, data services, authentication systems, caching layers, and enterprise infrastructure.

• Support production considerations including observability, tracing, latency optimization, scalability, reliability, and cost management.

• Prototype new LLM, retrieval, and agent architectures and help transition successful approaches into production.

• Establish reusable patterns and technical standards that enable conversational analytics capabilities to scale across multiple business domains.

Partner Cross-Functionally

• Work closely with Analytics and domain teams to encode metric definitions, analytical workflows, business logic, and domain knowledge into AI systems.

• Partner with Product and Engineering teams to translate analytical use cases into scalable product capabilities.

• Work with Legal, Privacy, Security, and Responsible AI partners to ensure AI analytics products are compliant, secure, explainable, and safe.

• Balance rapid experimentation with production reliability, analytical accuracy, governance, latency, and cost efficiency.

Required Qualifications

• Experience building LLM applications, AI-powered analytics products, conversational AI systems, or agentic workflows.

• Strong Python skills and hands-on experience developing AI application logic, APIs, data workflows, or backend services.

• Strong understanding of LLM orchestration, including tool/function calling, multi-step workflows, context management, and structured outputs.

• Experience building RAG and retrieval systems across structured and/or unstructured enterprise data.

• Experience integrating LLM applications with SQL databases, analytical APIs, data platforms, or query engines.

• Strong understanding of prompt and context engineering, including system prompts, tool prompts, dynamic context construction, and prompt iteration.

• Experience designing LLM evaluations, golden datasets, automated quality checks, and regression testing.

• Strong understanding of analytics, business metrics, data models, and how users investigate and consume data-driven insights.

• Ability to translate ambiguous business questions into well-defined analytical workflows and technical system designs.

• Familiarity with privacy, security, safety, and governance requirements for enterprise AI systems.

Desired Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Science, Statistics, Machine Learning, Analytics, Information Systems, or a related technical field, or equivalent practical experience.

• 7+ years of experience across data science, analytics engineering, machine learning, AI application development, or related technical roles.

• Hands-on experience with agent frameworks and LLM application libraries such as LangChain, LangGraph, LlamaIndex, or equivalent technologies.

• Experience with vector databases, embeddings, semantic search, hybrid retrieval, and reranking.

• Experience with natural-language-to-SQL systems, semantic layers, metadata catalogs, or analytics query engines.

• Experience building or integrating FastAPI or similar API-based services for AI applications.

• Familiarity with containerized and cloud-native environments such as Docker and Kubernetes.

• Experience with LLM observability, tracing, caching, model routing, and inference cost optimization.

• Experience designing multi-agent systems or domain-specific agent architectures.

• Experience developing enterprise-scale AI, analytics, or business intelligence products.

• Experience partnering with Legal, Privacy, Security, or Responsible AI teams to productionize AI capabilities.

What Success Looks Like

• Analytics chatbots reliably translate natural-language questions into grounded analytical workflows and accurate answers.

• The orchestration layer effectively coordinates retrieval, data querying, analytical tools, specialized agents, and LLM reasoning.

• Retrieval systems consistently provide the right business definitions, schemas, metrics, SQL examples, and analytical context for each request.

• AI-generated responses are trusted because they are grounded in source data, explainable, reproducible, and aligned with business definitions.

• Evaluation frameworks detect regressions across retrieval, SQL generation, reasoning, and final-answer quality before they reach users.

• New analytical domains can be onboarded through reusable knowledge, retrieval, workflow, and evaluation patterns rather than bespoke chatbot development.

• The platform can iterate quickly while maintaining enterprise standards for accuracy, reliability, security, compliance, latency, scalability, and cost.

Additional Details

The base pay range for this position is expected in the range below:

Base pay offered may vary depending on multiple individualized factors, including location, skills, and experience. The total compensation package for this position may also include other elements, including a target bonus and restricted stock units (as applicable) in addition to a full range of medical, financial, and/or other benefits (including RRSP eligibility, various paid time off benefits, such as PTO and parental leave). Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

This job posting relates to an existing vacancy within eBay.

eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, and disability, or other legally protected status. If you have a need that requires accommodation, please contact us at talent@ebay.com. We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBay's commitment to ensuring digital accessibility.

We use cookies to enhance your experience and may use AI tools for administrative tasks in the hiring process. To learn how we handle your personal data and use AI responsibly, please visit our Talent Privacy Notice, Privacy Center and AI Hiring Guidelines.