We are now hiring a talented, self-driven and passionate Senior Data Engineer to build and maintain optimized and highly available data pipelines that facilitate deeper analysis and reporting.
What Youll Do:
Design, develop, and maintain scalable data pipelines that integrate data from multiple sources, including APIs, databases, streaming platforms, edge computing devices, cloud services, and on-premise systems.
Design reliable and business-critical data processing workflows for batch and real-time use cases.
Develop data access services and APIs that enable efficient communication between edge devices, cloud infrastructure, and on-premise environments.
Design and optimize storage solutions for structured, semi-structured, and high-dimensional sensor data to support AI model training, inference, and analytics.
Strong understanding of distributed systems and scalable data processing architectures.
Build and maintain scalable streaming data pipelines for low-latency processing and event-driven architectures.
Analyze existing data architecture, storage models, and processing workflows, and continuously improve performance, scalability, reliability, and maintainability.
Optimize cloud infrastructure and data storage costs while maintaining high availability and low-latency access.
Collaborate closely with Software, AI, Algorithms, DevOps, and Product teams to translate business requirements into scalable technical solutions.
Design monitoring, observability, and operational processes for data platforms.
What Youll Do:
Design, develop, and maintain scalable data pipelines that integrate data from multiple sources, including APIs, databases, streaming platforms, edge computing devices, cloud services, and on-premise systems.
Design reliable and business-critical data processing workflows for batch and real-time use cases.
Develop data access services and APIs that enable efficient communication between edge devices, cloud infrastructure, and on-premise environments.
Design and optimize storage solutions for structured, semi-structured, and high-dimensional sensor data to support AI model training, inference, and analytics.
Strong understanding of distributed systems and scalable data processing architectures.
Build and maintain scalable streaming data pipelines for low-latency processing and event-driven architectures.
Analyze existing data architecture, storage models, and processing workflows, and continuously improve performance, scalability, reliability, and maintainability.
Optimize cloud infrastructure and data storage costs while maintaining high availability and low-latency access.
Collaborate closely with Software, AI, Algorithms, DevOps, and Product teams to translate business requirements into scalable technical solutions.
Design monitoring, observability, and operational processes for data platforms.
Requirements:
Bachelors degree in Computer Science, Engineering, Mathematics, or a related quantitative field.
5+ years of professional experience in Data Engineering or a related role.
Strong experience designing and implementing large-scale data pipelines using orchestration frameworks such as Apache Airflow, Prefect, or similar.
5+ years of software development experience, including at least 2 years of Python development.
Strong knowledge of relational and NoSQL databases such as PostgreSQL, MySQL, MongoDB, Elasticsearch/OpenSearch, ClickHouse, or similar technologies.
Experience designing and implementing streaming and event-driven data architectures using technologies such as AWS Kinesis, Amazon SQS, RabbitMQ, Kafka, or similar messaging systems.
Experience designing REST APIs and backend services (FastAPI or similar frameworks).
Experience working with AWS cloud services (S3, EC2, Lambda, CloudWatch, IAM, etc.).
Experience with Git, Docker, CI/CD pipelines, and modern software engineering practices.
Excellent communication and collaboration skills with engineering, AI, and Product teams.
Self-driven, innovative, and continuously looking for ways to improve systems and processes.
Great to Have:
Experience with Kubernetes and container orchestration.
Experience with distributed computing platforms and distributed data processing systems.
Experience building ML data pipelines supporting training and inference workloads.
Experience working with large-scale sensor, IoT, or time-series data.
Experience with monitoring and observability tools such as Grafana, Prometheus, ELK, Kibana, or OpenSearch.
Experience working in edge computing or hybrid cloud environments.
Bachelors degree in Computer Science, Engineering, Mathematics, or a related quantitative field.
5+ years of professional experience in Data Engineering or a related role.
Strong experience designing and implementing large-scale data pipelines using orchestration frameworks such as Apache Airflow, Prefect, or similar.
5+ years of software development experience, including at least 2 years of Python development.
Strong knowledge of relational and NoSQL databases such as PostgreSQL, MySQL, MongoDB, Elasticsearch/OpenSearch, ClickHouse, or similar technologies.
Experience designing and implementing streaming and event-driven data architectures using technologies such as AWS Kinesis, Amazon SQS, RabbitMQ, Kafka, or similar messaging systems.
Experience designing REST APIs and backend services (FastAPI or similar frameworks).
Experience working with AWS cloud services (S3, EC2, Lambda, CloudWatch, IAM, etc.).
Experience with Git, Docker, CI/CD pipelines, and modern software engineering practices.
Excellent communication and collaboration skills with engineering, AI, and Product teams.
Self-driven, innovative, and continuously looking for ways to improve systems and processes.
Great to Have:
Experience with Kubernetes and container orchestration.
Experience with distributed computing platforms and distributed data processing systems.
Experience building ML data pipelines supporting training and inference workloads.
Experience working with large-scale sensor, IoT, or time-series data.
Experience with monitoring and observability tools such as Grafana, Prometheus, ELK, Kibana, or OpenSearch.
Experience working in edge computing or hybrid cloud environments.
This position is open to all candidates.
