Available for Opportunities
Raj Shekar Bale
GCP Data Engineer
Effective GCP Data Engineer with 4+ years of hands-on experience in designing, implementing, and improving strong data ingestion pipelines and ETL/ELT workflows using Google Cloud Platform (GCP) services. Proven expertise in Big Query, Google Cloud Storage (GCS), Dataflow, Composer, Data Fusion, and Dataproc, with a strong proficiency in Python for scripting, debugging, and automation tasks. Adept at migrating legacy data platforms to flexible cloud-native architectures, reducing processing times by up to 30%, and enhancing system scalability and operational efficiency. Committed to collaborating with crossfunctional teams, ensuring stringent SLA adherence, and maintaining infrastructure-as-code using Terraform and Git for continuous integration and delivery.
0+
Roles
0+
Skills
Experience
AI/ML Computational Science Analyst
Accenture Solutions Pvt. Ltd.
- Designed and implemented strong, flexible, and secure data ingestion pipelines and ETL/ELT workflows on Google Cloud Platform (GCP), ensuring high availability and reliability for mission-critical data operations.
- Developed and improved complex batch and real-time ETL workflows using BigQuery, Google Cloud Dataflow, Pub/Sub, and Apache Beam, enabling critical analytics and business intelligence (BI) initiatives.
- Integrated diverse enterprise systems, including SAP HANA, Oracle, and JDA, into cloud-based data ingestion pipelines using Google Cloud Storage (GCS) and BigQuery, helping with reliable data movement and transformation.
- Designed and implemented partitioned and clustered BigQuery tables, significantly improving query performance by up to 40% and reducing storage costs, while actively monitoring production data pipelines to ensure stability and swift issue resolution.
- Collaborated effectively with cross-functional business and analytics teams to define data requirements, deliver well-structured data models, and provide practical findings supporting strategic decision-making processes.
- Managed parameter configurations, job orchestration via Composer, and metadata for complex data ingestion pipelines, ensuring data governance and operational consistency in GCP.
- Maintained deployment scripts and infrastructure configurations in GCP using Terraform for infrastructure-as-code (IaC), matching DevOps best practices and ensuring reproducible environments.
- Performed rigorous debugging and resolved runtime data issues independently for critical data pipelines, maintaining data integrity and minimizing disruptions to downstream consumers.
Packaged App Development Associate
Accenture Solutions Pvt. Ltd.
- Contributed significantly to modernizing data platforms by successfully migrating legacy workloads to Google Cloud Platform (GCP), using GCS, BigQuery, and Google Cloud Dataflow to construct flexible, cloud-native data ingestion pipelines.
- Transformed traditional server-based ETL processes into fully serverless architectures using GCP services like Dataflow and Cloud Functions, achieving significant improvements in system scalability and operational efficiency.
- Redesigned and improved data ingestion pipelines, in advance addressing performance bottlenecks and subsequently reducing overall data processing time by up to 30% across multiple critical data workflows.
- Improved and tuned BigQuery workloads for enhanced query performance and cost-effectiveness, implemented GCS lifecycle configurations for efficient storage management, and improved Dataflow worker use to raise pipeline reliability and reduce operational costs.
- Monitored daily job runs and production workflows diligently to pre-empt and minimize Service Level Agreement (SLA) breaches, collaborating closely with release teams to ensure smooth and controlled deployments, helped by Git for version control.
- Solidified a strong foundation in cloud-based data engineering principles, performance tuning methodologies, and the development of highly flexible data ingestion pipelines across diverse GCP environments.
- Contributed to small enhancements within the GCP data framework, using Python to implement new features and improve existing data processing utilities, sharing improvements with the core development team.
- Used Git for rigorous version control best practices, managing code repositories, helping collaborative development, and maintaining clear historical records of all data pipeline code and infrastructure configurations.
Skill Proficiency
Google Cloud Platform (GCP): GCS75%
BigQuery75%
Dataflow75%
Composer75%
Data Fusion75%
Dataproc75%
Pub/Sub75%
Cloud Shell75%
Cloud SDK75%
IAM75%
Programming & Scripting: Python75%
SQL75%
PySpark75%
Bash75%
Big Data Technologies: Apache Spark75%
Hadoop75%
Kafka75%
HDFS75%
Databases: MySQL75%
Oracle75%
PostgreSQL75%
Hive75%
BigQuery75%
ETL Tools: Apache NiFi75%
Apache Airflow (Composer)75%
Data Fusion75%
Data Visualization: Qlik Sense75%
Power BI75%
Looker75%
File Formats: CSV75%
Parquet75%
JSON75%
ORC75%
Avro75%
DevOps & IaC Tools: Jenkins75%
Terraform75%
Git75%
GitHub75%
Bitbucket75%
Jira75%
IDEs: Visual Studio Code75%
Education
B. Tech - Electronics and Communication Engineering (ECE)
Swarnandhra Institute of Engineering and Technology
06/2017 — 08/2021