Blog

Hello 

Here are the Requirement for the below Position !! 

Job Title : Technical Databricks Lead

Location : [Charlotte, NC, 28246],[Cincinnati, OH, 45270] (Onsite,)

Experience : 13 to 17 Years

Work Authorization : GC, GC EAD, USC, H4EAD

Tax Term : C2C

Tier : Working with PV

End Client : Kroger

Note :- Above 13+ with last 3-4 years as Technical Lead (No Relocation)

Custom screening – sub directly to job, previous scores don't transfer

Top 3 Skills: Databricks Certified Technical Lead, Python, Data Domain experience

Project: Supporting Kroger’s transition from MS Azure to Databricks; early-stage setup of data domains

Soft Skills: Strong leadership, clear communication across agile on/offshore teams, transparency, and adherence to timelines

Requirements

• 10+ years of experience in data engineering, big data, or API development, with 3+ years in a leadership role
• Proven experience leading product-centric data engineering initiatives in an agile delivery environment
• Expertise in Azure Databricks, Apache Spark, Azure SQL, and other Microsoft Azure services
• Strong programming skills in Python, Scala, and SQL for data processing and API development
• Experience building and managing APIs (REST, GraphQL, gRPC) and microservices
• Hands-on experience with Azure Data Factory (ADF), Azure Synapse Analytics, and Delta Lake
• Proficiency in CI/CD pipelines, Infrastructure-as-Code tools like Terraform or Bicep
• Experience implementing data security and compliance for sensitive HR/employee data (encryption, access control, auditing)
• Strong problem-solving skills with a focus on performance tuning, security, and cost optimization
• Experience with containerization (Docker, Kubernetes) and event-driven architecture (preferred)
• Exposure to Informatica for ETL/ELT and data integration
• Excellent communication and leadership skills in a fast-paced environment
• Willingness to transition from contract to full-time employment

Preferred Qualifications

• Microsoft Certified: Azure Solutions Architect Expert or Databricks Certified Data Engineer/Architect
• Experience with agile methodologies such as Scrum or SAFe
• Familiarity with machine learning workflows in Azure Databricks
• Knowledge of Azure API Management and Event Hub for API integration
• Experience with Informatica PowerCenter or IICS
• Hands-on experience with Oracle HCM database models and APIs
• Background in HR analytics and workforce data integration

Core Responsibilities:

• Develop and maintain the HR data domain in a secure, compliant, and efficient manner aligned with industry best practices
• Lead the development of a data engineering team responsible for building scalable, high-performance data solutions, APIs, and microservices in Azure Databricks, Azure SQL, and Informatica
• Ensure the highest levels of security and privacy for sensitive HR data across all systems and integrations
• Bring deep expertise in big data processing, data architecture, cloud migrations, API development, data security, and agile methodologies within the Azure ecosystem

Key Responsibilities:

Azure Databricks & Big Data Architecture:

• Design and implement scalable data pipelines and architectures using Azure Databricks
• Optimize ETL/ELT workflows to ensure efficient data processing, storage, and retrieval
• Leverage Apache Spark, Delta Lake, and Azure-native services to build high-performance data solutions
• Apply best practices in data governance, security, and compliance across Azure environments
• Troubleshoot and fine-tune Spark jobs to improve performance and reduce costs

Azure SQL & Cloud Migration:

• Lead migration of workloads from Azure SQL to Azure Databricks, ensuring a seamless transition
• Design and implement scalable ETL/ELT pipelines from Azure SQL to Databricks Delta Lake
• Optimize Azure SQL queries and indexing strategies pre-migration to enhance post-migration performance
• Maintain data governance, security, and compliance throughout the migration lifecycle
• Use Azure Data Factory (ADF), Informatica, and Databricks to orchestrate automated migration workflows
• Enable integration of migrated data with APIs, ML models, and BI tools
• Establish post-migration performance monitoring and cost optimization strategies

API & Services Development:

• Design and build RESTful APIs and microservices for efficient data access and integration
• Implement secure and scalable API frameworks to expose data processing capabilities
• Utilize GraphQL, gRPC, or streaming APIs for real-time data consumption needs
• Integrate APIs with Azure data lakes, warehouses, Oracle HCM, and other enterprise systems
• Apply API performance tuning, monitoring, and security practices including OAuth, JWT, and Azure API Management

HR Data Domain & Security:

• Build and manage a scalable, secure, and well-governed HR data architecture
• Implement RBAC, encryption, and data masking to protect sensitive employee data
• Ensure compliance with data privacy regulations such as GDPR, CCPA, and HIPAA
• Design and implement audit logging and monitoring for data access and modifications
• Collaborate with HR and security teams to define access permissions, retention policies, and data anonymization strategies
• Enable secure APIs and data-sharing mechanisms for HR analytics while maintaining employee privacy
• Integrate Oracle HCM data structures within the Azure Databricks ecosystem

Product-Centric & Agile Delivery:

• Drive a product-focused approach to data engineering aligned with user needs and business outcomes
• Operate within an agile delivery framework using Scrum or Kanban methodologies
• Collaborate with product managers and stakeholders to define use cases and prioritize backlog items
• Foster a continuous improvement culture using feedback loops and data-driven decision-making
• Implement DevOps and CI/CD pipelines for rapid and iterative delivery of data solutions

Leadership & Collaboration:

• Provide technical leadership and mentorship to data engineers and developers
• Collaborate with business stakeholders, HR teams, product managers, and architects to translate requirements into data solutions
• Advocate for automation and Infrastructure-as-Code practices using tools like Terraform and Bicep
• Cultivate a culture of innovation, collaboration, and continuous learning within the team

Stay current with advancements in Azure Databricks, Azure SQL, Informatica, Oracle HCM, and other cloud technologies

Regards,

Ritika Negi

Technical Recruiter

Ritika@vsgbusinesssolutions.com

VSG Business Solutions Inc

3240 East State ST Ext Suite 203

Hamilton, New Jersey 08619