We are looking for an experienced, opinionated, and highly technical Staff Data Engineer to define the long-term direction of our Data Platform.
This role is ideal for a senior data leader who combines deep hands-on expertise with strong architectural judgment, enjoys mentoring others, and takes full ownership of complex, high-scale data systems.
You will drive platform architecture, set engineering standards, and lead the design and delivery of scalable batch and streaming data solutions on AWS.
Key Responsibilities:
Technical Leadership & Architecture:
Own and evolve the overall data platform architecture-scalability, reliability, security, and maintainability.
Lead the long-term strategic direction of the platform, balancing performance, cost, and operational excellence.
Introduce and drive adoption of modern data paradigms: lakehouse (Iceberg), event-driven pipelines, schema-aware processing.
Data Lake & Architecture Ownership:
Design, model, and evolve the Data Lake architecture, including:
Storage layout and data organization
Data formats and table design (e.g., Iceberg)
Batch and streaming ingestion patterns
Schema governance and lifecycle policies
Define and promote best practices for data modeling, partitioning, and data quality.
Ensure the Data Lake supports analytics, ML workloads, and operational systems at scale.
Platform Engineering:
Design and build high-scale ETL/ELT pipelines leveraging Apache Spark (EMR/Glue) and AWS-native services.
Optimize production-grade pipelines using S3, Athena, Kinesis, Lambda, Step Functions, and EKS.
Lead the rollout of modern patterns such as lakehouse architectures and event-driven data pipelines.
Security & Governance:
Ensure alignment with AWS security best practices-IAM, encryption, auditing, and compliance frameworks.
Partner with security and governance teams to support regulated and sensitive data environments.
Mentorship & Collaboration:
Serve as a technical mentor to data engineers; elevate team capabilities.
Lead architecture reviews and cross-team design discussions.
Work closely with Data Science, ML Engineering, Backend, and Product teams to deliver end‑to‑end data solutions.
This role is ideal for a senior data leader who combines deep hands-on expertise with strong architectural judgment, enjoys mentoring others, and takes full ownership of complex, high-scale data systems.
You will drive platform architecture, set engineering standards, and lead the design and delivery of scalable batch and streaming data solutions on AWS.
Key Responsibilities:
Technical Leadership & Architecture:
Own and evolve the overall data platform architecture-scalability, reliability, security, and maintainability.
Lead the long-term strategic direction of the platform, balancing performance, cost, and operational excellence.
Introduce and drive adoption of modern data paradigms: lakehouse (Iceberg), event-driven pipelines, schema-aware processing.
Data Lake & Architecture Ownership:
Design, model, and evolve the Data Lake architecture, including:
Storage layout and data organization
Data formats and table design (e.g., Iceberg)
Batch and streaming ingestion patterns
Schema governance and lifecycle policies
Define and promote best practices for data modeling, partitioning, and data quality.
Ensure the Data Lake supports analytics, ML workloads, and operational systems at scale.
Platform Engineering:
Design and build high-scale ETL/ELT pipelines leveraging Apache Spark (EMR/Glue) and AWS-native services.
Optimize production-grade pipelines using S3, Athena, Kinesis, Lambda, Step Functions, and EKS.
Lead the rollout of modern patterns such as lakehouse architectures and event-driven data pipelines.
Security & Governance:
Ensure alignment with AWS security best practices-IAM, encryption, auditing, and compliance frameworks.
Partner with security and governance teams to support regulated and sensitive data environments.
Mentorship & Collaboration:
Serve as a technical mentor to data engineers; elevate team capabilities.
Lead architecture reviews and cross-team design discussions.
Work closely with Data Science, ML Engineering, Backend, and Product teams to deliver end‑to‑end data solutions.
Requirements:
Technical Expertise:
Advanced experience with Apache Spark (EMR, Glue, PySpark).
Deep expertise in AWS data ecosystem: S3, EMR, Glue, Athena, Lambda, Step Functions, Kinesis.
Strong understanding of Data Lake and Lakehouse architectures.
Experience building scalable batch and streaming pipelines.
Hands-on experience with Infrastructure as Code (Terraform / CDK / SAM).
Python as a primary programming language (TypeScript is a plus).
Leadership Mindset:
Opinionated yet pragmatic; able to defend architectural trade-offs.
Strategic thinker capable of translating long-term vision into actionable roadmaps.
Strong end‑to‑end ownership mentality, from design to production operations.
Passionate about automation, simplicity, and scalable engineering.
Excellent communicator capable of explaining complex decisions to diverse stakeholders.
Nice to Have:
AWS certifications (Solutions Architect, Data Engineer).
Experience supporting ML pipelines or AI-driven analytics.
Familiarity with data governance, data mesh, or self‑service data platforms.
Experience working in regulated, security‑sensitive, or law‑enforcement domains.
Technical Expertise:
Advanced experience with Apache Spark (EMR, Glue, PySpark).
Deep expertise in AWS data ecosystem: S3, EMR, Glue, Athena, Lambda, Step Functions, Kinesis.
Strong understanding of Data Lake and Lakehouse architectures.
Experience building scalable batch and streaming pipelines.
Hands-on experience with Infrastructure as Code (Terraform / CDK / SAM).
Python as a primary programming language (TypeScript is a plus).
Leadership Mindset:
Opinionated yet pragmatic; able to defend architectural trade-offs.
Strategic thinker capable of translating long-term vision into actionable roadmaps.
Strong end‑to‑end ownership mentality, from design to production operations.
Passionate about automation, simplicity, and scalable engineering.
Excellent communicator capable of explaining complex decisions to diverse stakeholders.
Nice to Have:
AWS certifications (Solutions Architect, Data Engineer).
Experience supporting ML pipelines or AI-driven analytics.
Familiarity with data governance, data mesh, or self‑service data platforms.
Experience working in regulated, security‑sensitive, or law‑enforcement domains.
This position is open to all candidates.
