Blog

We are seeking a highly skilled and motivated Senior Software Engineer to join our team for a data transformation and mapping project. The successful candidate will work on designing, building, and optimizing large-scale data pipelines and transformation processes within the Azure cloud ecosystem. This role is critical to ensuring seamless data flow, transformation, and integration, which are vital for our organization's data-driven initiatives.

The ideal candidate will have deep technical expertise in Azure Synapse, SQL, Azure Data Factory, and Python, with experience designing scalable and efficient ETL (Extract, Transform, Load) processes. Bonus points for candidates with experience in financial regulatory environments and an understanding of financial data standards and compliance requirements.

Required Qualifications:

• 5+ years of experience in software engineering or data engineering roles, with a focus on data transformation and mapping.

• Extensive experience in Azure Cloud Services, including Azure Synapse Analytics, Azure Data Factory, and other related tools.

• Proficiency in SQL for data querying, transformation, and optimization.

• Strong Python programming skills for scripting and automation in data workflows.

• Experience building and optimizing ETL pipelines for large-scale data integration.

• Proven ability to design and implement cloud-based data architecture and processing systems.

• Strong problem-solving skills with attention to detail and a proactive attitude toward troubleshooting and optimizing data workflows.

• Ability to work in an agile environment, managing multiple priorities and collaborating effectively with cross-functional teams.

Preferred Qualifications:

• Financial regulatory experience or familiarity with financial data standards and compliance requirements.

• Knowledge of data governance, security, and compliance best practices.

• Experience working with big data technologies or distributed systems within a cloud environment.

• Certification(s) in Azure Data Engineering, Azure Solutions Architecture, or relevant fields.

Key Responsibilities:

• Data Transformation and Mapping: Lead the design, development, and implementation of complex data transformation and mapping pipelines using Azure Data Factory, Azure Synapse, and SQL.

• Azure Cloud Development: Architect and implement cloud-native solutions leveraging the Azure stack, ensuring security, scalability, and performance.

• ETL Pipelines: Develop, monitor, and optimize ETL processes for data ingestion and transformation, ensuring the data is transformed and mapped accurately between systems.

• SQL Expertise: Write and optimize complex SQL queries for efficient data extraction, transformation, and loading in Azure Synapse.

• Data Integration: Collaborate with cross-functional teams to integrate and align data across multiple systems and platforms.

• Automation: Leverage Python to automate data workflows, improve data processing efficiency, and ensure the reliability of data pipelines.

• Data Quality Assurance: Ensure data quality and integrity throughout the data transformation process by implementing appropriate testing and validation techniques.

• Documentation and Best Practices: Create and maintain documentation, and advocate for best practices in data engineering, ensuring compliance with internal and external data handling standards.

• Financial Regulatory Data: (Bonus) If applicable, design and implement solutions that comply with financial regulations and reporting standards (e.g., SEC, FINRA, etc.), ensuring data mapping aligns with compliance requirements.