Senior Data Engineer (ETL Developer)

Job Description

Experience:

6\+ Years

Role:

Senior Data Engineer

Location:

Coimbatore- Work from Office

Employment

Type:

Full-time

Availability:

Immediate Joiner Preferred

Role Summary

We are seeking an experienced Data Engineer with 6\+ years of experience in designing, developing, and maintaining scalable data pipelines and data warehouse solutions. The ideal candidate should have strong expertise in ETL/ELT, Python, dbt, Snowflake, workflow orchestration, job scheduling, and Git.

Key Responsibilities

• Design, develop, test, and maintain scalable ETL/ELT data pipelines.

• Build and manage enterprise data warehouse solutions using Snowflake.

• Develop reusable data transformation models using dbt.

• Write efficient and maintainable Python programs for data ingestion, transformation, validation, and automation.

• Extract and integrate data from databases, APIs, flat files, and other structured or semi-structured sources.

• Develop dimensional data models using concepts such as:

• Star and snowflake schemas

• Fact and dimension tables

• Slowly Changing Dimensions

• Surrogate keys

• Incremental and historical data loads

• Create and maintain ETL jobs, schedules, dependencies, alerts, and recovery procedures.

• Work with orchestration and scheduling tools such as Apache Airflow, Control-M, Azure Data Factory, AWS Glue, or similar platforms.

• Implement data-quality checks, reconciliation processes, audit controls, and exception handling.

• Monitor pipeline performance and troubleshoot failed or delayed data-processing jobs.

• Optimize Snowflake queries, storage, warehouses, and overall pipeline performance.

• Use Git for source-code management, branching, merging, pull requests, code reviews, and release management.

• Support CI/CD processes for deploying data pipelines and dbt models across environments.

• Collaborate with business analysts, data analysts, reporting teams, architects, and other stakeholders to understand data requirements.

• Prepare and maintain technical documentation, data mappings, lineage, and operational support procedures.

• Participate in Agile ceremonies, sprint planning, estimation, peer reviews, and production-support activities.

Required Skills and Qualifications

• 6\+ years of professional experience in data engineering, ETL development, or data warehousing.

• Strong understanding of ETL/ELT processes and data warehousing concepts.

• Hands-on experience with Python for data processing and automation.

• Strong hands-on experience with dbt, including:

• Models and materializations

• Sources and seeds

• Tests and documentation

• Macros and Jinja

• Incremental models

• Snapshots

• Hooks and packages

• Strong experience with Snowflake, including:

• Databases, schemas, tables, and views

• Virtual warehouses

• Stages and file formats

• Snowpipe

• Streams and Tasks

• Time Travel and zero-copy cloning

• Query and warehouse performance optimization

• Role-based access control

• Advanced SQL skills, including joins, CTEs, window functions, stored procedures, and query optimization.

• Experience creating, scheduling, monitoring, and troubleshooting ETL jobs.

• Experience with workflow orchestration tools such as Apache Airflow or an equivalent platform.

• Strong working knowledge of Git, including branching strategies, merge conflict resolution, pull requests, and code reviews.

• Experience working with structured and semi-structured data formats such as CSV, JSON, XML, Avro, and Parquet.

• Knowledge of data-quality frameworks, logging, monitoring, reconciliation, and error-handling practices.

• Good understanding of software development life cycle, Agile methodologies, and production-support processes.

• Strong analytical, troubleshooting, communication, and problem-solving skills.

Preferred Qualifications

• Experience with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform.

• Experience implementing CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, Azure DevOps, or similar tools.

• Familiarity with BI and reporting tools such as Power BI, Tableau, or Looker.

• Understanding of data governance, metadata management, lineage, security, and access control.

• Relevant Snowflake, dbt, cloud, or data-engineering certifications would be an advantage.

Expected Competencies

• Ability to independently design and deliver end-to-end data pipelines.

• Strong ownership of data accuracy, reliability, and performance.

• Ability to analyse complex data issues and provide sustainable solutions.

• Effective collaboration with technical and business stakeholders.

• Ability to mentor junior engineers and contribute to engineering best practices.