Blog

Hi,

Role: Senior Data Engineer

Location: Atlanta, GA / Frisco, TX / Overland Park, KS

Job Description:

We are seeking a Senior Data Engineer to lead a team of data engineers in an Onshore-Offshore model, collaborating closely with customer SMEs. This role requires a strong technical background in Azure services, data engineering, and SQL. The ideal candidate should be self-driven, have extensive experience with data pipelines, and be highly focused on automation and optimization.

Key Responsibilities:

• Data Pipeline Development: Design, build, and deploy efficient and reliable data pipelines to move and transform data across systems.
• Optimization & Maintenance: Optimize existing data pipelines and ensure ongoing maintenance of source-related data pipelines.
• Data Quality: Develop and implement comprehensive data quality checks to ensure the integrity and accuracy of data.
• Data Warehousing & Products: Build and deploy scalable, high-performance data warehouses and data products.
• Collaboration: Work closely with leadership, engineers, product managers, and data scientists to understand data needs and deliver on requirements.
• Framework Enhancement: Improve existing data movement frameworks by adding new features and enhancing performance.
• Automation Focus: Implement automation across processes to improve efficiency and reduce manual efforts.
• On-Call Support: Participate in on-call shifts to provide support for the team when necessary.

Required Skills:

1. Azure Services: 2+ years of experience with Azure services, including Azure Data Factory.
2. Azure DataBricks & Python: 5+ years of experience working with Azure DataBricks and Python for data processing and transformation.
3. SQL & Snowflake: Expertise in SQL and Snowflake, with a focus on building and optimizing complex queries and data storage.
4. Data Modeling & Analysis: 5+ years of experience in data modeling and analysis.
5. Test Automation: Strong experience with automating data testing processes.
6. CI/CD & DevOps: Experience with CI/CD pipeline implementation and DevOps practices using tools such as GitLab.
7. Agile Methodology: Familiarity with Agile fundamentals, along with prior experience using Jira for project management.

Technical Skill Matrix:

• Azure Services: 2+ years
• Azure DataBricks: 5+ years
• Python: 5+ years
• Azure Data Factory: 2+ years
• GitLab: 2+ years
• Data Modeling & Analysis: 5+ years
• SQL Queries / PL/SQL Procedures: 5+ years
• Snowflake: 2+ years

Thanks & Regards

Rajesh Kumar G

[email protected]

linkedin.com/in/raje-sh-56696321b