|
Hi,
I hope you’re doing well!
Please look at the requirements below, let us know of your interest, and send us your updated resume to [email protected]
Role: Sr Data Engineer
Location: Remote-Fremont CA
Job Description:
- Design and Implement Scalable Data Pipelines:
- Build robust data pipelines to efficiently extract, transform, and load (ETL) data from diverse sources into Meta’s data warehouse.
- Ensure that data processing pipelines are scalable and maintainable to handle large volumes of data.
- Develop Data Processing and Analytics Applications:
- Utilize programming languages such as Python, Java, and SQL to create and maintain applications that process and analyze large datasets.
- Implement data models and ensure data is ready for analysis by data scientists and analysts.
- Collaborate with Cross-Functional Teams:
- Work closely with data scientists, analysts, and business stakeholders to understand their data requirements and deliver tailored solutions.
- Design and implement data-driven solutions that meet business needs, ensuring the solutions are aligned with the company’s goals.
- Optimize Data Access and Retrieval Performance:
- Apply performance optimization techniques such as caching, indexing, and other strategies to improve data access and retrieval times.
- Ensure that data retrieval processes are efficient and cost-effective.
- Ensure Data Quality and Integrity:
- Implement data validation and testing processes to ensure the accuracy and reliability of data at all stages of the pipeline.
- Use automated testing to ensure continuous data quality.
- Stay Up-to-Date with Emerging Technologies:
- Keep current with the latest advancements in data engineering, big data technologies, and best practices.
- Continuously improve systems and solutions by integrating innovative technologies and techniques.
Requirements:
- Strong Programming Skills:
- Proficiency in Python, Java, and SQL for data processing and application development.
- Experience with Data Pipeline Tools:
- Familiarity with data pipeline orchestration tools like Apache Airflow, Luigi, or Dataswarm for automating and scheduling workflows.
- Big Data Technologies:
- Experience working with Hadoop, Spark, Hive, or other big data technologies to process and analyze large datasets at scale.
- Cloud Computing Platforms:
- Familiarity with cloud platforms such as AWS, GCP, or Azure to build and deploy data infrastructure and manage cloud-based data storage solutions.
- Problem-Solving and Independence:
- Strong analytical and problem-solving skills with the ability to work independently and tackle complex technical challenges.
- Communication and Collaboration:
- Excellent communication skills to collaborate effectively with data scientists, analysts, and cross-functional teams.
- Ability to clearly articulate technical concepts to non-technical stakeholders.
Key Skills and Technologies:
- Programming: Python, Java, SQL
- Data Pipeline Tools: Apache Airflow, Luigi, Dataswarm
- Big Data: Hadoop, Spark, Hive
- Cloud Platforms: AWS, GCP, Azure
- Performance Optimization: Caching, Indexing
- Data Quality: Data validation, Testing
With Regards
kishore Reddy
|
