Associate Data Engineer

Job Overview

– Collaborate with customer to gather requirements and to understand their business processes.

– Create and maintain optimal data pipeline architecture,

– Assemble large, complex data sets that meet functional / non-functional business requirements.

– Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

– Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using

Spark, SQL and Azure ‘big data’ technologies.

– Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and

other key business performance metrics.

– Use tools to deploy and monitor the performance of the systems in production.

Demonstrate knowledge and real-world experience on Big data technologies. A successful history of manipulating, processing and extracting value from large disconnected datasets.

– Advanced working SQL knowledge and experience working with relational databases, query

authoring (SQL) as well as working familiarity with a variety of databases.

– Strong analytic skills related to working with unstructured datasets.

– Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

– We are looking for a candidate with > 2+ years of experience in a Data Engineer role. They should also have experience using the following software/tools:

○ Experience with big data tools: Hadoop, Spark, Kafka, etc.

○ Experience with relational SQL databases such as Oracle, SQL Server, MySQL or Postgres.

○ Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

○ Experience with stream-processing systems: Storm, Spark-Streaming, etc.

○ Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

○ Some experience on visualization tools such as PowerBI/Tableau or other BI tools will be



– Degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field

– 2+ years of relevant experience.

– Negotiable salary.

Job Detail
  • Max Gross Salary (USD)1,000 - 1,500
Shortlist Never pay anyone for job application test or interview.