Greeting,
Hope you are doing great!
Please find below job description & let me know if it matches with your job profile. Please share your updated resume copy along with your contact information.
Role : Lead Databricks Engineer
Location : Iselin NJ / New York, NY : Hybrid onsite (2 Days)
Hire type : Contract
Must have : Expert in Databricks , Pyspark , python
Job description:-
- Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent work experience.
- 5+ years of experience in data engineering with Databricks and Spark.
- Proficient in SQL and Python and Pyspark.
- Experience with Azure Databricks Medallion Architecture with DLT, Iceberg.
- Financial/Corporate Banking context would be a plus.
- Experience with data integration and ETL tools, such as Azure Data Factory .
- Experience with Azure cloud platform and services.
- Experience with data warehouse and data lake concepts and architectures.
- Good to have experience with big data technologies, such as Kafka, Hadoop, Hive, etc.
- Strong analytical and problem-solving skills.
- Excellent communication and teamwork skills.
Responsibilities
- Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed.
- Optimize data pipelines for performance, scalability, and reliability.
- Ensure data quality and integrity throughout the data lifecycle.
- Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs.
- Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations.
- Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management.
- Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community.
- Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies.
Regards,
Lohit Tiwari
Sibitalent Corp.
101 E. Park Blvd., Suite 600
Plano, TX – 75074
Phone: 972-736-8310
Email: [email protected]
URL: www.sibitalent.com