Blog

Hi,

Hope you are doing well.

I have a Job Position for Big Data Engineer. Find below the Job Description and Let me know if you are interested.

Job Description:

Job Title :       Big Data Engineer  

Location :      Hybrid to Malvern, PA

End Client :    Vanguard | Working with Prime Vendor

Qualifications

• Proficiency in Python programming
• Strong expertise in SQL, Presto, HIVE, and Spark
• Knowledge of trading and investment data
• Experience in big data technologies such as Spark and developing distributed computing applications using PySpark
• Experience with libraries for data manipulation and analysis, such as Pandas, Polars and NumPy Understanding of data pipelines, ETL processes, and data warehousing concepts
• Strong experience in building and orchestrating data pipelines
• Experience in building APIs Write, maintain, and execute automated unit tests using Python
• Follow Test-Driven Development (TDD) practices in all stages of software development
• Extensive experience with key AWS services/components including EMR, Lambda, Glue ETL, Step Functions, S3, ECS, Kinesis, IAM, RDS PostgreSQL, Dynamodb, Timeseries database, CloudWatch Events/Event Bridge, Athena, SNS, SQS, and VPC
• Proficiency in developing serverless architectures using AWS services
• Experience with both relational and NoSQL databases
• Skills in designing and implementing data models, including normalization, denormalization, and schema design
• Knowledge of data warehousing solutions like Amazon Redshift
• Strong analytical skills with the ability to troubleshoot data issues
• Good understanding of source control, unit testing, test-driven development, and CI/CD
• Ability to write clean, maintainable code and comprehend code written by others
• Strong communication skills
• Proficiency in data visualization tools and ability to create visual representations of data, particularly using Tableau

click here