Blog

Job Description

We are seeking a skilled AWS Data Engineer with experience building and maintaining data pipelines for data warehousing and machine learning applications in AWS. In this role, you will analyse the data preparation and integration requirements and create the pipelines to clean/transform and ingest the data in preparation for visualization/querying and use in machine learning algorithms (eg. recommendation solutions). You will collaborate closely with various application stakeholders as well as your frontend developers and data scientists to help integrate these APIs into various applications, while ensuring the security and scalability of our backend services.

Responsibilities

• API Development and Maintenance: Design, develop, and maintain robust backend APIs to expose AI/ML capabilities, ensuring high performance and reliability.
• Data Pipeline Management: Develop and optimize data pipelines and processes to ensure efficient and seamless data flow, supporting the integration of AI/ML models.
• Cross-functional Collaboration: Collaborate closely with frontend developers and data scientists to integrate backend APIs into applications, ensuring cohesive and efficient functionality.
• Security and Scalability: Implement best practices to ensure the security, scalability, and resilience of backend services, protecting data integrity and supporting growth.
• Performance Optimization: Monitor and optimize the performance of backend services, identifying and addressing bottlenecks and ensuring smooth operation under varying loads.
• Code Quality and Standards: Maintain high standards of code quality and documentation, adhering to industry best practices and contributing to continuous improvement initiatives.
• Testing and Debugging: Develop and execute comprehensive test plans to identify and resolve defects, ensuring the robustness and reliability of backend services.
• Version Control and Deployment: Utilize version control systems and continuous integration/continuous deployment (CI/CD) pipelines to manage codebase changes and streamline deployment processes.
• Technical Documentation: Create and maintain detailed technical documentation for backend APIs, ensuring clarity and accessibility for all stakeholders.
• Continuous Learning: Stay updated with the latest trends, tools, and technologies in backend development and AI/ML, applying new knowledge to enhance system capabilities and performance.