Job Title: Senior Data Pipeline Engineer (AWS)
Department/Group: Enterprise Data Services
We are looking for Senior Data Pipeline Engineers (multiple positions) for ongoing projects, within the Data Services and ML/AI Group.
Senior Data Pipeline Engineers (AWS)
• 4+ years of experience in big data engineering, with a focus on Spark and SQL.
• 4+ Years Experience in developing Large-scale ETL Pipelines using AWS services not limited to IAM, Glue, Lambda, S3, Athena, SNS, SQS, DynamoDB, RDS, EMR, ECS, Route 53 and Redshift.
• Highly skilled in Python, PostgreSQL/RDS/Redshift, and writing complex SQL queries and analysis of data correlations.
• Design, construct, and manage Data Lake environments including Data ingestion, staging, Data quality monitoring, and business modeling, Data Schema Normalization, etc.
• Develop, construct, secure, test and maintain architectures (databases and large-scale processing systems)
• Design, develop and implement technical solutions in SQL environment (databases, SQL, data marts, integrations)
• Experience in tools like Jenkins, GIT and/or AWS CodePipeline/CodeBuild/CodeDeploy
• Experience building data-driven unit test suites for data platforms and modeling highly dimensional datasets
• Strong functional understanding of Sagemaker, Jupyter and other Data Science tools on AWS
PREFERRED SKILLS
• Experience designing and building big data architectures using tools such as Hadoop, Hive, or Kafka.
Notice Period – Imm / 30 days.