Blog

Hello,

Trust this email finds you well.

I am Akshay Kumar from Global Applications Solution, writing this email to share few exciting Job Opportunity with one of our premium clients.

If you are interested, please share resume on my email – [email protected]

Job Title: Data Engineering Lead

Experience: 14 years or above
Location:

·       Mississauga, ON

·       Montreal, QC

·       Iselin, NJ

(3 Days Hybrid from Day 1)

Must-Have Skills:

·       Snowflake

·       Databricks

·       SQL

·       Python

·       ADF

·       PySpark

·       Data Warehouse Concepts

Job Description:

We are seeking skilled Data Engineering Leads and Data Engineers with expertise in Python, PySpark, SQL, and data warehousing for enterprise-level systems. This position involves working in a cloud-based environment, handling large data volumes, and collaborating with business users and analysts, especially in the banking and capital markets industry. Strong database fundamentals, along with PySpark coding expertise, are crucial.

Major Responsibilities:

·       Data Pipeline Development:
Build and optimize efficient data pipelines for ingesting, transforming, and loading data from various sources while ensuring data quality and integrity.

·       Databricks Spark Program Design:
Design, develop, and deploy Spark programs within the Databricks environment to process and analyze large-scale datasets.

·       Data Integration & Modeling:
Experience with Delta Lake, Data Warehousing (DWH), and Data Modeling, including Dimensional Data Modeling.

·       Cloud Data Architectures:
Design and implement cloud-based data solutions. Experience with various cloud data warehouses (e.g., Snowflake, Redshift, BigQuery, Synapse).

·       Stream and Event-Based Data Processing:
Work with event-based and streaming technologies to ingest and process structured, semi-structured, and unstructured data.

·       Databricks Performance Optimization:
Optimize Databricks jobs for performance and scalability to handle big data workloads. Monitor, troubleshoot, and resolve issues or bottlenecks in Databricks jobs.

·       Data Security & Governance:
Implement best practices for data management, security, and governance within the Databricks environment.

·       SQL and Python Development:
Write complex SQL queries and develop Python programs for large dataset processing. Experience with stored procedures and reverse-engineering existing processes.

·       Code Reviews and Best Practices:
Perform code reviews to ensure alignment with requirements, optimal execution patterns, and adherence to established coding standards.

Skills and Experience Required:

·       Python: 5+ years of hands-on coding experience.

·       SQL Development: 5+ years of experience developing solutions with SQL Server or similar databases for large datasets.

·       ETL Pipelines & Databricks PySpark: 5+ years of experience developing and deploying ETL pipelines using Databricks and PySpark.

·       Cloud Data Warehousing: Experience with any cloud data warehouse such as Snowflake, Synapse, Redshift, or BigQuery.

·       Data Warehousing Expertise: Experience in OLTP, OLAP, Data Modeling (Facts, Dimensions), and ETL processes.

·       Cloud Data Platform Migration: Previous experience leading cloud data platform migrations with strong architectural and design skills.

·       Cloud Architecture & Analytics: Experience in cloud-based data architectures, messaging, and analytics.

·       Cloud Certifications: Certifications in cloud technologies are a plus (e.g., AWS, Azure, GCP).

·       Performance Tuning: Strong experience in code optimization and performance tuning for scalable data solutions.

Thanks & Regards,
Akshay Kumar
Talent Acquisition- Recruitment
Global Applications Solution LLC(USA)
#1 Meadowlands Plaza, Suite 216. East Rutherford, NJ 07073.
Office USA: +1 201-578-7586| WhatsApp: +91 7562 863220
📧[email protected]
https://www.linkedin.com/in/itsakshaykrs/
www.globalapplications.com