Data Engineer with Python, AWS

We are looking for an experienced Data Engineer to join our Tech team. You will be responsible for building our data pipelines and you will be also participating in setting the right Infrastructure and developing the needed tools for our Analytics.

If you have excellent data engineering skills, strong problem-solving skills, and a passion for

developing applications utilizing cutting edge technologies, then we would like to meet you.

 

Job Location:

It can be in our Dubai office or remotely with periodic travel to Dubai

 

Responsibilities

 

  • Gather and address technical and design requirements.
  • Architect our data platform & develop real-time apps ingesting large volume of data and processing messages real-time.
  • Handle all our data pipeline and contribute towards our data strategy and its execution.
  • Architecting, building, testing & managing our data platform infrastructure from data collection all the way to data processing & data storage.
  • Develop, customize and manage integration tools, monitoring tools, databases, warehouses, and analytical systems.
  • Support the developers through training and transfer of knowledge.
  • Participate in the entire application life-cycle mainly focusing on coding, debugging,

and testing.

 

Requirements

 

  • BSc degree in Computer Science/Computer Engineering. Masters is a plus.
  • Strong competencies in algorithms and software architecture.
  • 3+ years of experience in Python.
  • 2+ years of experience as a Data Engineer.
  • Strong experience in real-time data processing and data ingestion.
  • Strong experience in batch data processing and data pipelines.
  • Strong work experience in AWS Kinesis Services and building apps utilizing it is a must.
  • 3+ years of experience in developing APIs (REST & SOAP protocols).
  • 2+ years of experience in using AWS cloud and its resources/services.
  • Strong experience in Relational Databases (MySQL, SQL Server or PostgreSQL) and non-relational databases (Mongo DB or Cassandra. etc.)
  • Strong knowledge in Continuous Integration & Continuous Deployment (CI/CD) utilizing Docker containers.
  • Strong knowledge in Pandas & NumPy libraries is a big plus.
  • Strong knowledge in workflow orchestration tools (Airflow ...etc.) is a plus.
  • Experience in Java is a plus.
  • Previous experience in automated testing including unit testing & UI testing is a plus.
  • Strong in-depth understanding of the entire web development process (design, development, and deployment)