We are seeking an experienced Data Engineer to join our engineering team. The ideal candidate will have extensive experience with AWS, Python, REST APIs, and Snowflake, as well as a strong background in GitLab-based CI/CD practices. This role will be responsible for architecting scalable solutions, mentoring team members, and driving technical excellence in our cloud-native environment for Big Data solutions. We are unable to provide any kind of visa sponsorship for this role.
Key Responsibilities
Technical Leadership
- Architect and design scalable, high-performance solutions using AWS services and Python
- Lead the development and maintenance of REST APIs, ensuring they meet performance and security standards
- Optimize data pipelines and analytics processes using Snowflake
- Drive best practices in code quality, testing, and documentation
DevOps and Infrastructure
- Have familiarity with GitLab-based CI/CD pipelines
- Deployment experience using Infrastructure as Code (IaC) practices using tools like AWS CloudFormation or Terraform
- Ensure high availability and disaster recovery for our AWS-based systems
Team Leadership
- Mentor junior and mid-level developers, fostering a culture of continuous learning
- Collaborate with product managers to translate business requirements into technical specifications
- Lead code reviews and architectural discussions
Performance and Security
- Implement and maintain security best practices for cloud-native applications
- Conduct performance tuning of Python applications and AWS services
- Optimize systems and applications for performance and efficiency in big data platforms
Required Skills & Qualifications
Technical Skills
- 10+ years of software development experience, with at least 5 years focused on Python and AWS
- Expert-level knowledge of Python, including asyncio, type hinting, and modern Python practices
- Extensive experience with AWS services, particularly Lambda, ECS, S3, and DynamoDB
- Proficiency in designing and developing RESTful APIs
- Strong understanding of Snowflake architecture and SQL optimization techniques
- Advanced Git skills and experience with GitLab CI/CD pipelines
Cloud and Data
- AWS Certified Solutions Architect or equivalent practical experience
- Experience with serverless architectures and event-driven design in AWS
Leadership and Soft Skills
- Proven track record of leading development teams and mentoring junior developers
- Excellent communication skills, able to explain complex technical concepts to non-technical stakeholders and create documentation and diagrams
- Strong problem-solving abilities and decision-making skills in high-pressure environments
Preferred Qualifications
- Experience with other AWS services like SQS, SNS, Batch, and Step Functions
- Familiarity with GraphQL APIs and webhooks
- Knowledge of ML/AI services in AWS (SageMaker, Bedrock, etc.)
- Contributions to open-source Python projects
