Blog

Hello  Partners,

Looking for :
1. Strong in Terraform
2. ⁠Data engg experience
3. ⁠Strong in CI/CD pipelines
4. ⁠Experience setting up data transfer pipelines
5. ⁠Good experience on  GKE/GCP

Key Responsibilities:

Design, develop, and optimize data pipelines and workflows using GCP’s Data Engineering tools (e.g., Dataflow, Pub/Sub, BigQuery, Cloud Composer).
Architect, deploy, and manage infrastructure as code using Terraform, ensuring scalable and secure GCP environments.
Implement and maintain CI/CD pipelines to automate deployment and testing of data pipelines and cloud infrastructure.
Collaborate with cross-functional teams to design solutions that align with business objectives and ensure data availability, integrity, and performance.
Monitor and troubleshoot issues in GCP environments, ensuring high availability and reliability of data services.
Define and enforce best practices for Terraform usage across projects.

Mandatory Skills and Qualifications:

7+ years of experience in data engineering, cloud infrastructure, and CI/CD processes.
Hands-on expertise with Terraform in GCP environments, including creating modules, managing state, and scaling infrastructure.
In-depth knowledge of BigQuery for data storage, querying, and optimization.
Proficiency in GCP Data Engineering tools such as Dataflow, Pub/Sub, Cloud Data Fusion, and Cloud Composer.
Strong experience with CI/CD tools (e.g., Jenkins, GitLab CI/CD, or Cloud Build) and DevOps best practices.
Excellent coding skills in Python, SQL, and shell scripting for data processing and automation.

Preferred Skills and Qualifications:

GCP Certified (Professional Data Engineer or equivalent).
Experience with data modeling, ETL processes, and data governance.
Familiarity with Apache Beam, Spark, or other distributed data processing frameworks.
Strong problem-solving skills and the ability to optimize large-scale systems.

–.