Job Description
At its core, this is a principal software architect position - with prior working experience as a software developer implementing, e.g., modern distributed systems, cloud-based data lakes, and open-source data engineering platforms. The successful candidate will play a lead role in the Data Engineering and Analytics team of Scoot's Information Technology Division.
Key Responsibilities
- Provide technical leadership and guidance for data engineers in Scoot. Drive the overall technical vision and roadmap for evolving Scoot's distributed data management systems and data analytics.
- Be an effective implementer and technical mentor for the team in the following core activities:Design and develop new systems architecture for data engineering services and their ecosystem, including distributed databases (relational, columnar, graph, in-memory) cloud DevOps with IaC (Infrastructure-as-Code) and other big data technologies.
Maintenance and evolution of existing Google Cloud data warehouse/data lake systems.
Design data models for mission-critical and high-volume near-real-time data build idempotent/atomic production data pipelines to make data ingestion robust and fault tolerant. - Assist in the stakeholder management and resolve resource conflicts within or between agile teams. Lead projects involving high level of coordination among departments and business areas.
- Any relevant ad-hoc duties.
Requirements
- BS degree in Computer Science or a related discipline is required. Advanced degrees in Computer Science (PhD, MS) are highly desirable.
- 7 years or more relevant industry experience in the following technical areas:
- At least intermediate-level knowledge and experience with Google Cloud and its best practices. AWS cloud experience is a plus.
- Advanced programming skills in Python. Conversant with data structure, algorithm design/analysis, and SQL. Substantial exposure on functional/object-oriented programming using a modern programming language (such as Scala, Java, C#, and TypeScript/JavaScript) is a plus.
- Exposure in the use of workflow/map-reduce and stream processing systems for big-data processing, such as Spark and Kafka.
- In-depth understanding of modern ETL methodologies. Experience in building data pipelines using modern data ingestion platforms, including open-source ones.
- Experience in the deployment of object stores and data warehouses - such as Amazon S3, and Google GCS and BigQuery.
- Experience in modern software development is a plus (such as in web frontend UI and backend API microservice).
- Proven experience in technical leadership. Capable of mentoring a data engineering team in delivering on multiple competing priorities with little supervision. Seasoned resource estimation, planning, and negotiation skills to work with diverse stakeholders.
- Prior experience with managing software development teams using Agile/Scrum/Kanban methodology is a plus.
