This role will report to the Team manager- R&D.
Why is this role so important ?
we are a data-focused company, and data is the heart of our business. As a data server engineer, you will work in one of the companys core teams and design and implement complex, high-scale systems that retrieve and analyze data from around the internet.
You will be responsible for:
Scraping – Building and maintaining high scale web-scraping operations.
Processing – Creating complex data pipelines that process and enrich our raw data and serve it to different consumers through various databases and data-lakes.
Engineering – Designing and building large scale, high quality systems to support our data infrastructure and production systems
As a Data-Server Engineer in Data Extraction, you will:
Write complex PySpark applications that process, manipulate and validate large datasets, in high quality and under strict SLA.
Develop dockerized high-performance microservices.
Take a major part in designing and implementing complex high scale systems using a large variety of technologies.
Work in a team with talented and independent engineers with whom youll be able to share knowledge and expertise.
learn new technologies, handle a multitude of systems.
Implement solutions in AWS cloud environment.
Take part in monitoring and debugging data pipelines and high scale services in order to identify and fix bottlenecks and failures.
Has at least 5+ years of server-side software development experience in one or more general purpose programming languages (C#, Java, Python, GO, Scala, etc.) – Must.
Has Experience with Big Data technologies such as Hadoop, Spark, Databricks and Airflow.
Worked in a cloud environment such as AWS or GCP, and is familiar with its different services, as well as docker and Kubernetes
Has experiences with web scraping technologies or familiar with web technologies and the internets internals.
Holds a BSc degree in Computer Science or equivalent practical experience.
Is comfortable taking on challenges and learning new technologies, including new coding languages.
Is a team player and likes giving and receiving constructive feedback
Can effectively prioritize tasks, multitask and work independently
Has good communication skills and enjoys helping and supporting team members as well as people from other groups
