The Data & Machine Learning Engineer plays a hybrid role that focuses on developing a modern Lakehouse platform, building data pipelines, and deploying machine learning models at scale. This position blends the responsibilities of a Data Engineer and Machine Learning Engineer to ensure robust data infrastructure and effective deployment of ML models. The ideal candidate will collaborate with cross-functional teams, working closely with Data Analysts, Business Analysts, and DevOps engineers to deliver reliable, scalable, and efficient data and ML solutions.
Key Responsibilities:
- Build, automate, and optimize ETL/ELT pipelines to ensure smooth data flow and availability across the Lakehouse platform.
- Partner with analysts to design scalable data storage solutions optimized for analytics and ML applications.
- Ensure data quality by implementing monitoring mechanisms to guarantee consistency, reliability, and compliance.
- Continuously monitor ML models, retrain and adjust them as necessary to maintain high performance.
- Design and implement CI/CD pipelines for seamless model deployment and ensure reliable delivery of models into production environments.
- Provide guidance and mentorship to junior engineers, supporting their growth and enhancing project delivery capabilities.
Skills & Experience:
- Strong teamwork skills, able to align data solutions with business needs, and communicate effectively across technical and non-technical teams.
- Expertise in analyzing data-driven insights and translating them into technical solutions.
- Proficient in designing and implementing data pipelines using Databricks and Spark.
- Advanced skills in SQL and Python for querying, modeling, and transforming data efficiently.
- Expertise in deploying machine learning models and creating APIs for serving predictions.
- Skilled in processing large-scale datasets and supporting machine learning workflows on Databricks.
- Experience in building CI/CD pipelines for automated deployment, testing, and validation of data pipelines and ML models using GitHub and GitHub Actions.
- Proven ability to monitor ML model performance, track key metrics, and maintain accuracy.
- Experience with automating cloud infrastructure using tools like Terraform for consistent deployment and scaling.
- Basic knowledge of Power BI for visualizing model performance and metrics.
Qualifications Required:
- 5+ Years in Data Engineering: Solid experience building and maintaining data infrastructure, including best practices in data pipeline development.
- 3+ Years in Machine Learning Engineering: Demonstrated experience managing the lifecycle of ML models from deployment to monitoring and retraining.
- Business & Data Transformation: Proven track record in supporting business transformation and process improvements by leveraging data-driven solutions.
- Agile Methodologies: Comfortable working in Agile environments with the ability to adapt and respond quickly in fast-paced settings.
To apply, simply click the Apply button or send your updated profile to
EA Licence No.:18S9405 / EA Reg. No.:R1330864
Percept Solutions is expanding and actively seeking talented individuals. We encourage applicants to follow Percept Solutions on LinkedIn at https://www.linkedin.com/company/percept-solutions/to stay informed about new opportunities and events.
