AWS ETL Cloud Data Engineer

Job Description AWS ETL Cloud Data Engineer

Responsibilities:

Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties Spark/Python on Glue, DMS, S3, Athena, RDS-PostgreSQL, Airflow, Lambda, Code Commit, Code Pipeline, Code build, etc.

Design and build production ETL data pipelines from ingestion to consumption within a big data architecture, using DMS, DataSync & Glue.

Understand the existing applications(including on-premise Cloudera Datalake) and infrastructure architecture.

Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.

Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.

Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.

Collaborate with development, infrastructure and data center teams to define Continuous Integration and Continuous Delivery processes in accordance with industry standards.

Work on hybrid Datalake.

Work closely with multiple stakeholders to ensure high standards are maintained.

Mandatory Skill-set

Bachelors Degree in Computer Science, Information Technology or other relevant fields

5+ years of work experience with ETL, Data Modelling, Data Architecture to build Datalake. Proficient in ETL optimization, designing, coding, and tuning big data processes using Pyspark.

3+ yrs of extensive experience in working on AWS platform using core services like AWS Athena, Glue Pyspark, RDS-PostgreSQL, S3 & Airflow(for orchestration).

Kshama

Registration No. / Unique Entity Number: 199801439D

Disclaimer:The company is committed to ensuring the privacy and security of your information. By submitting this form, you consent to the collection, processing, and retention of the information you provide. The data collected (which may include your contact details, educational background, work experience and skills) will be used solely for the purpose of evaluating your qualifications for the position you're applying for. Your data will be stored securely and retained for the duration necessary to fulfill our hiring process. If you are not selected for the position, your data will be kept on file for a limited period in case future opportunities arise. You have the right to access, correct, or delete your data at any time by contacting us at Quess Singapore | A Leading Staffing Services Provider in Singapore (quesscorp.sg)