At a glance
- Experience
- 7+ years experience
- Location
- Bangalore, Karnataka
About the role
- Brillio is seeking a QA Architect to design and implement automated testing strategies for AI and data science outputs.
- The role involves developing validation frameworks, collaborating with data scientists, and ensuring accuracy across machine learning models and pipelines.
Responsibilities
- Design and implement automated testing strategies for AI and data science outputs, ensuring accuracy and reliability across models and pipelines
- Develop and maintain robust evaluation and validation frameworks for backend and frontend components, leveraging statistical and machine learning techniques
- Collaborate with data scientists and engineers to define test cases, hypotheses, and statistical metrics for model assessment and improvement
- Integrate advanced statistical tests such as T-Test, Z-Test, and regression analyses into automated QA workflows to validate model performance
- Utilize tools like Great Expectations, Evidently AI, and specific machine learning frameworks to monitor, track, and report on model drift, anomalies, and forecast accuracy
- Optimize testing processes for scalability and efficiency using Python, PySpark, R, and related technologies in large-scale data environments
- Configure and manage testing infrastructure using platforms such as KubeFlow and BentoML to streamline deployment and evaluation cycles
- Troubleshoot and resolve issues in automated testing pipelines, driving continuous improvement and high-quality deliverables
Requirements
- Advanced proficiency in Python and PySpark for test automation and statistical analysis
- Expertise in statistical testing methods including Hypothesis Testing, T-Test, Z-Test, and Regression (Linear, Logistic)
- Strong experience with machine learning frameworks such as TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, and MXNet
- Hands-on knowledge of Great Expectations and Evidently AI for data validation and monitoring
- Proficiency in SAS and SPSS for statistical computing and analysis
- Deep understanding of probabilistic graph models and classification algorithms including Decision Trees and SVM
- Experience with forecasting techniques including Exponential Smoothing, ARIMA, and ARIMAX
- Familiarity with distance metrics such as Hamming, Euclidean, and Manhattan Distance
- Advanced skills in R and R Studio for statistical modeling and QA scripting
- Experience configuring testing platforms such as KubeFlow and BentoML
Preferred qualifications
- Experience automating evaluation pipelines for AI/ML in production environments
- Expertise in integrating QA processes with CI/CD workflows and cloud-native architectures
- Knowledge of emerging ML testing tools and frameworks beyond industry standards
- Ability to develop custom statistical metrics for model evaluation
- Experience with QA automation for distributed systems at scale
Skills
- Python
- PySpark
- TensorFlow
- PyTorch
- Sci-Kit Learn
- CNTK
- Keras
- MXNet
- Great Expectations
- Evidently AI
- SAS
- SPSS
Show all 15 skills
- R
- KubeFlow
- BentoML
