Looking for a Data Engineer to work within a thriving Data Services department that offers a motivating, flexible and fast-moving environment. The candidate will be responsible for building and optimizing data pipelines and datasets that would serve different teams within the enterprise varying from business functions, technical teams and fellow data analysts and data scientists. The candidate will help in identifying data quality issues and put preventive measures to ensure higher confidence in data. The Data Engineer will help in improving the data platform that will support internal and external needs. The ideal candidate is very passionate about Data Engineering and is self-directed and comfortable to work with ambiguity of requirements to purpose and implement optimal solutions to support the needs of internal teams, systems, and external products and partners.

Key Responsibilities:

• Work with different business stakeholders and technical teams to gather requirements.

• Architecture and implement optimal data pipelines.

• Redesign and improve current processes to eliminate manual processes and ensure timely delivery of data.

• Prepare scalability plans for smoother expansion when needed.

• Build data models that ensure wholesome view of the company’s needed analytics.

• Work with Data Analysts to prepare Ad-Hoc or permanent batch/real-time data pipelines to enable further analysis or machine learning requirements.

• Implement quality checks and monitoring schemes to ensure data quality and successful completion of data pipelines.

• Design and build data platforms.

Key Requirements:

• 3+ years or more of experience working as a Business Intelligence / Data Engineering role.

• Strong problem solving and root cause analysis skills.

• Advanced knowledge of variety Relational Database Engines of a large scale.

• Ability to write advanced SQL queries.

• Enterprise-level knowledge of designing and implementing data pipelines.

• Ability to work with scripting languages such as Python for data manipulation and data pipelines orchestration.

• Ability to build complex data models for Reporting or Functional purposes.

• Solve performance issues in terms of data injection or data retrieval.

• Candidate should have experience and familiarity working with the following tools:

o DBT (data build tool) experince.

Relational Database Management Systems, such as: SQL Server, Postgresql.

o Real-time OLAP data stores, such as: Kudu, ClickHouse, Apache Pinot

o Distributed File Systems, Object stores, such as: Hadoop, minio

o Distributed Data Processing Frameworks, such as: Apache Spark

o Distributed query engines, such as: Presto

o Stream processing tools, such as: Apache Kafka, Apache Flink, Apache Druid.

o NoSQL databases.

o Dataflow management tools, such as: Apache Nifi, Alteryx, SSIS.

o API integration with external sources.

o Object-oriented/Functional programming in: Python, Java, Scala, etc.

o Data Visualization tools, such as: Power Bi, Metabase, Apache Superset.