Senior Data Engineer - Immediate joiner only

About the Role

We are looking for a Mid-Level Data Engineer to support the development and operation of an industrial data platform The role involves building and maintaining scalable data pipelines, integrating industrial and enterprise data sources, and ensuring high data quality and reliability across the platform. The engineer will work closely with cloud, platform, and domain teams in an agile delivery environment.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines

• Ingest data from industrial systems, databases, APIs, and files

• Process and manage time‑series, asset, and event-based data

• Develop data pipelines using Azure Databricks / Apache Spark

• Orchestrate workflows using Azure Data Factory

• Store, manage, and optimize data in Azure Data Lake Storage (ADLS)

• Prepare and publish curated datasets for ingestion into Cognite

• Implement data transformation, validation, and enrichment logic

• Monitor pipelines, troubleshoot failures, and perform root cause analysis

• Ensure data quality, reliability, and performance in production environments

• Collaborate with platform, cloud, and domain teams in Agile delivery

• Participate in code reviews, sprint planning, and technical discussions

• Maintain technical documentation for pipelines and data flows

Mandatory Skillsets

• 3-5 years of hands-on experience in Data Engineering

• Strong hands-on experience with Python (pandas, scripting) Strong SQL skills (complex joins, aggregations, performance tuning) Experience with Apache Spark / Databricks

• Extensive experience with Azure cloud services for data engineering (Azure Data Factory, Azure Data Lake Storage (ADLS), Azure data services)

• Solid understanding of data warehousing concepts and cloud-based implementations

• Strong knowledge of ETL/ELT architectures and best practices

• Expertise in dimensional modelling, star schemas, and data mart design

• Experience handling large-scale datasets with performance optimization techniques

• Strong analytical thinking and problem-solving skills

• Familiarity with Git and version control

Preferred Skillsets

• Experience with Databricks and Delta Lake architecture

• Familiarity with data lake and lakehouse architectures

• Knowledge of data warehouse migration strategies from on-prem to cloud

• Exposure to real-time or streaming technologies

• Experience with workflow orchestration tools

• Understanding of data quality frameworks and data governance tools

• Familiarity with data virtualization concepts

• Cloud certifications (AWS / Azure / Databricks) are a plus

• Exposure to Cognite Data Fusion or similar industrial data platforms

• Experience in oil \& gas, energy, manufacturing, or industrial domains

• Knowledge of time‑series and IoT data processing

• Experience with CI/CD pipelines