Blog

GCP Data Engineer

Phoenix, AZ (hybrid)

12+Months

9+years of exp is a must

Technical Skills:

·         6+ years of experience in Data Engineering with an emphasis on Data Warehousing and Data Analytics.
·         4+ years of experience in design and build of salable data pipelines that deal with extraction, transformation, and loading.
·         2+ years hands on experience on GCP Cloud data implementation projects (Dataflow, DataProc, Cloud Composer, Big Query, Cloud Storage, GKE, Airflow, etc.).
·         4+ years of experience with Python, Scala with working knowledge on Notebooks.
·         4+ years experience in leading cloud platforms.
·         At least 2 years of experience in Data governance and Metadata Management.
·         Ability to work independently, solve problems, update the stake holders.
·         Analyze, design, develop and deploy solutions as per business requirements.
·         Strong understanding of relational and dimensional data modeling.
·         Experience in DevOps and CI/CD related technologies.
·         Excellent written, verbal communication skills, including experience in technical documentation and ability to communicate with senior business managers and executives.
·         Experience working across multiple time-zones and ability to get things done.

Roles & Responsibilities:

·         Take direction from Tech leads and be responsible for delivery, leading off-shore team
·         Knowledge of GCP stack
·         Monitors the Data Lake constantly and ensures that the appropriate support teams are engaged at the right times.
·         Design, build and test scalable data ingestion pipelines, perform end to end automation of ETL process for various datasets that are being ingested.
·         Determine best way to extract application telemetry data, structure it, send to proper tool for reporting (Kafka, Splunk).
·         Create reports to monitor usage data for billing and SLA tracking.
·         Work with business and cross-functional teams to gather and document requirements to meet business needs.
·         Provide support as required to ensure the availability and performance of ETL/ELT jobs.
·         Provide technical assistance and cross training to business and internal team members.
·         Collaborate with business partners for continuous improvement opportunities.