Design, develop, optimize, and maintain data architecture and pipelines that adhere to ETL principles and business goals.
Solve complex data problems to deliver insights that help the organization achieve its goals.
Code in Python with tools like Apache Spark to build a multi-cluster data warehouse.
Interact with other technology teams to define, prioritize, and ensure smooth deployments for other operational components.
Advise, consult, mentor, and coach other data and analytics professionals on data standards and practices.
Foster a culture of sharing, reuse, design for scale stability, and operational efficiency of data and analytical solutions.
Codify best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases to facilitate data capturing and management.
Strong experience in mapping attributes, data profiling, data cleansing, and technical data quality etc.
Strong experience in Ansi SQL and in-depth knowledge with structured, semi-structured & unstructured data.
Must have good knowledge of data lake and working experience of migration projects in cloud with providers like AWS or Microsoft AZURE or GCP.
Good to have experience in working with No SQL, Spark SQL using AWS Glue, EMR, and columnar data store.
Good to understand data security features like data masking, data encryption, role based & fine grain access control mechanisms etc.
Qualifications:
5+ years of relevant experience in data engineering/analytics space.
Expertise in SQL and data analysis and strong hands-on expertise with at least one programming language: Python.
Strong knowledge in one or more of the following big data tools: Hive, Hadoop Impala, Spark, Kafka.
Strong expertise in ETL, reporting tools, data governance, data warehousing, and hands-on experience.
Experience developing solutions for cloud computing services and infrastructure.
Experience developing and maintaining data warehouses in big data solutions.
Up to date on industry trends within the analytics space from a data acquisition processing, engineering, and management perspective.
Experience in agile development.
Strong people skills, specifically in collaboration and teamwork.
High level of curiosity, creativity, and problem-solving capabilities.
