Senior Software Engineer - Data Acquisition
Woven by Toyota · Ann Arbor, United States · 8h ago
TEAM
Our Cloud and Data Engineering organization is working on accelerating autonomous driving by providing access to petabytes of data collected from our large fleet of autonomous and non-autonomous vehicles. Efficient, fast and cost-effective access to data at large scale is key to tackle the hardest problems in AD/ADAS, from developing the Machine Learning (ML) models for perception and prediction of human driving patterns, to increasing the sophistication of our validation and simulation by identifying rare and interesting real-world driving situations. The Lakehouse ingestion platform developed by the Data Acquisition Engineering team is a fundamental building block for developing and testing modern AD/ADAS products that will impact millions of customers.
The primary goal of our Lakehouse system and Fleetnik mobile app is to seamlessly ingest, enrich, and monitor fleet vehicle data as it flows into the cloud. A key goal for this role is delivering data-collection edge filtering infrastructure that supports data validation and scenario detection. Our pipelines are based on industry standard frameworks deployed to AWS. We engineer large-scale data acquisition pipelines that process hundreds of terabytes daily from numerous global ingestion sites. Our data acquisition products leverage industry-standard frameworks deployed on AWS, utilizing Java, Golang, Python, and JavaScript. We believe strongly in automation and testing to ensure delivery of robust and correct systems. We are a distributed team, working in Japan, the UK and the US.
WHO ARE WE LOOKING FOR?
The Data Acquisition team is looking for engineers who are passionate about and enable the next generation of automotive software development. The right candidate will have excellent communication skills, solid coding skills, broad knowledge of software engineering in areas such as Data Infrastructure and Warehouses, Data Ingestion, Distributed Databases, Compute Frameworks, Stream Processing, Observability and Build Infrastructure.
RESPONSIBILITIES
-
Lead the design, build, maintenance, optimization and support of large scale, cloud-native data ingestion, data storage, data processing and data serving systems.
-
Architect and scale large production real-time systems written and maintained in Python and C++.
-
Understand the complex data requirements of modern AD/ADAS platforms and tailor our data ecosystem to these needs.
-
Work closely with other Data Infrastructure, Site Reliability and Vehicle software platform engineers on high-impact projects to create innovative solutions to problems in the self-drive space.
-
Mentor junior engineers in their day to day work and drive best practices across the organization.
-
Contribute to the long term strategy for several of our systems and products.
MINIMUM QUALIFICATIONS
-
Bachelor's degree in Computer Science, a related field, or equivalent practical experience.
-
5+ years of experience with data structures/algorithms and professional software engineering in one or more programming languages (e.g., Python, Go, Java).
-
Extensive development experience with autonomous driving and/or data collection using C++ under embedded environments.
-
3+ years designing and building data-intensive, concurrent, scalable applications.
-
Experience with cloud-based (e.g. AWS, GCP) microservice architecture, event-driven, distributed architectures.
-
2+ years writing testable, modular code in Python.
-
Business-level proficiency in English speaking, reading and writing (e.g., technical documents, software documentation).
NICE TO HAVES
-
3+ years of experience with data platforms, data pipelines, workflow orchestration, batch processing, and/or distributed databases.
-
2+ years writing testable, modular code in Golang.
-
2+ years writing testable, modular code in Java.
-
Experience working as a Site Reliability Engineer.
-
Experience with Terraform, Docker, cloud-native technologies, networking and Kubernetes in production.
-
Experience designing, deploying, and maintaining multi-region and/or multi-cloud systems.
-
Experience working in a fast-paced environment, collaborating across teams and disciplines.
-
Experience with data governance, data privacy and security.
-
Business-level proficiency in Japanese.