Palantir Data Pipeline & Metadata Lead
Guidehouse · Remote · New York, United States +1 · 8d ago
Job Family:
Data Science & Analysis
Travel Required:
Clearance Required:
What You Will Do:
This role serves as a senior technical leader responsible for establishing and scaling the data pipeline and metadata foundation that enables enterprise analytics, AI-driven capabilities, and system integration. The ideal candidate combines deep, demonstrable Palantir (including AIP) expertise, strong data architecture and governance experience, and the ability to lead teams and grow capabilities in complex, regulated environments.
Roles & Responsibilities
Lead the design, development, and governance of enterprise data pipelines and metadata frameworks within Palantir Foundry and integrated data platforms.
Serve as the technical and functional lead for data ingestion, transformation, and metadata management across structured and unstructured data sources.
Define and enforce metadata standards, data models, ontologies, and data dictionaries to enable scalable search, analytics, and cross-system integration.
Oversee implementation of end-to-end data pipelines, including ingestion, validation, transformation, and delivery into downstream platforms (e.g., Palantir, Databricks).
Establish and govern data quality, validation, and exception handling processes, ensuring completeness, accuracy, and traceability of data assets.
Ensure alignment of pipelines and metadata with enterprise architecture, system-of-record requirements, and integration patterns.
Ensure pipelines effectively support document-based ingestion workflows, including integration with OCR/ICR outputs and downstream metadata extraction processes.
Enable AI-ready data foundations, supporting downstream capabilities such as semantic search, entity resolution, and advanced analytics.
Partner with data science and engineering teams to ensure data pipelines and metadata support AI/ML use cases and analytical workflows.
Drive data governance and lifecycle management, including schema versioning, lineage tracking, auditability, and compliance with security and privacy requirements.
Oversee integration across platforms (e.g., AWS, Databricks, Palantir), ensuring scalable, secure, and reliable data exchange.
Lead and mentor teams in a matrixed, cross-functional environment, providing technical direction and quality oversight.
Engage with senior stakeholders to define data strategy, prioritize initiatives, and translate business needs into technical solutions.
Operate within an Agile delivery model, overseeing backlog prioritization, technical design reviews, and iterative delivery across workstreams.
What You Will Need:
U.S. Citizenship required and ability to obtain and maintain a Public Trust clearance.
Bachelor’s degree
EIGHT (8) or more years of experience in data engineering, data architecture, or platform integration, with increasing leadership responsibility.
Demonstrated, hands-on expertise in Palantir Foundry (required), including:
Designing and implementing production-grade data pipelines
Developing ontologies, data models, and relationship mappings
Integrating data across multiple enterprise systems
Demonstrated experience with Palantir AIP (required), including enabling AI-driven workflows (e.g., search, analytics, or decision-support use cases).
Proven experience leading enterprise-scale Palantir implementations, including architecture, delivery, and governance.
Strong experience designing and managing large-scale data pipelines in cloud environments (AWS preferred).
Experience integrating Palantir with Databricks and/or Spark-based platforms for advanced data processing and analytics.
Expertise in metadata management and data governance, including:
Data dictionaries and controlled vocabularies
Data lineage and traceability
Schema versioning and change management
Experience implementing data quality frameworks, including validation rules, exception handling, and reconciliation processes.
Proficiency in Python, SQL, and/or other relevant languages for data pipeline development.
Experience designing systems that support AI/ML and analytics use cases, including unstructured data and document processing pipelines (e.g., OCR/ICR outputs).
Experience delivering complex solutions in an Agile environment.
What Would Be Nice To Have:
Experience growing or leading Palantir capabilities and engagements within the federal sector, particularly in health or human services domains.
Strong leadership and communication skills, with experience managing cross-functional and matrixed teams.
Strong understanding of data lake/lakehouse architectures (e.g., AWS S3, Databricks).
Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field (Master’s preferred).
Experience designing AI-ready data architectures, including support for semantic search, entity resolution, and advanced analytics.
Experience with enterprise ML and data platforms (e.g., AWS SageMaker, Databricks Machine Learning) and integration with data pipelines.
Familiarity with OCR/ICR pipelines and document intelligence workflows, including integration of extracted text and metadata into downstream data platforms.
Experience implementing data governance frameworks in regulated environments, including auditability and compliance controls.
Prior experience working with federal agencies or highly regulated environments.
Experience in a consulting environment, including leading client engagements and supporting business development.
Experience supporting training, user enablement, and scaling Palantir/data capabilities across organizations.
Familiarity with graph-based data models, ontology-driven applications, and relationship analytics.
Relevant certifications (e.g., Palantir Foundry, AWS, Databricks) or equivalent demonstrated expertise.
What We Offer:
Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace.
Benefits include:
Medical, Rx, Dental & Vision Insurance
Personal and Family Sick Time & Company Paid Holidays
Parental Leave
401(k) Retirement Plan
Group Term Life and Travel Assistance
Voluntary Life and AD&D Insurance
Health Savings Account, Health Care & Dependent Care Flexible Spending Accounts
Transit and Parking Commuter Benefits
Short-Term & Long-Term Disability
Tuition Reimbursement, Personal Development, Certifications & Learning Opportunities
Employee Referral Program
Corporate Sponsored Events & Community Outreach
Care.com annual membership
Employee Assistance Program
Supplemental Benefits via Corestream (Critical Care, Hospital Indemnity, Accident Insurance, Legal Assistance and ID theft protection, etc.)
Position may be eligible for a discretionary variable incentive bonus
About Guidehouse
Guidehouse is an Equal Opportunity Employer–Protected Veterans, Individuals with Disabilities or any other basis protected by law, ordinance, or regulation.
Guidehouse will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of applicable law or ordinance including the Fair Chance Ordinance of Los Angeles and San Francisco.
If you have visited our website for information about employment opportunities, or to apply for a position, and you require an accommodation, please contact Guidehouse Recruiting at 1-571-633-1711 or via email at RecruitingAccommodation@guidehouse.com. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodation.
All communication regarding recruitment for a Guidehouse position will be sent from Guidehouse email domains including @guidehouse.com or guidehouse@myworkday.com. Correspondence received by an applicant from any other domain should be considered unauthorized and will not be honored by Guidehouse. Note that Guidehouse will never charge a fee or require a money transfer at any stage of the recruitment process and does not collect fees from educational institutions for participation in a recruitment event. Never provide your banking information to a third party purporting to need that information to proceed in the hiring process.
If any person or organization demands money related to a job opportunity with Guidehouse, please report the matter to Guidehouse’s Ethics Hotline. If you want to check the validity of correspondence you have received, please contact recruiting@guidehouse.com. Guidehouse is not responsible for losses incurred (monetary or otherwise) from an applicant’s dealings with unauthorized third parties.
Guidehouse does not accept unsolicited resumes through or from search firms or staffing agencies. All unsolicited resumes will be considered the property of Guidehouse and Guidehouse will not be obligated to pay a placement fee.