Commit to quality: AI-enhanced testing in open source

"Commit to quality: AI-enhanced testing in open source," presented by Shelley Lambert (IBM), explores how large-scale open source projects are applying AI to improve testing, triage, and release quality. This session was recorded at Open Community Experience 2026 (OCX26) in Brussels, Belgium, as part of the Open Community for AI.

This session examines how AI techniques can be applied across the software testing lifecycle in large-scale open source projects, using the Adoptium and AQAvit initiatives as a case study.

The testing context involves high-volume pipelines, with over 17 million tests executed per release cycle. At this scale, manual analysis is infeasible, requiring automated approaches for test execution, triage, and reporting.

The approach focuses on augmenting key stages of the testing lifecycle. In triage, tools such as Commit Hunter identify likely fault-inducing changes using a combination of rule-based strategies and AI-assisted analysis. In pre-commit stages, Glitch Witcher applies defect prediction techniques to assess the likelihood of introducing errors based on historical code patterns.

Performance testing is addressed through projects like Perf Savvy, which integrates benchmarking data and explores AI-driven detection of regressions using historical test results. Additionally, retrieval-augmented generation (RAG) techniques are used to improve access to project-specific knowledge, enabling more effective querying of test frameworks, documentation, and historical data.

The session also explores opportunities for AI-assisted test generation and selection, using combinatorial analysis to reduce redundant test execution while maintaining coverage. Across these use cases, the focus remains on model-agnostic, data-driven approaches that leverage existing project artefacts rather than relying on proprietary tooling.

Key topics covered
- AI in software testing
- large-scale test automation
- Adoptium and AQAvit
- commit analysis and fault detection
- defect prediction models
- performance regression detection
- RAG for developer workflows
- test generation and selection
- combinatorial testing
- CI/CD test analytics

Why this matters
At scale, software quality depends on the ability to prioritise, analyse, and act on large volumes of test data. AI-assisted workflows enable teams to improve triage, reduce redundant testing, and surface actionable insights, making testing pipelines more efficient and maintainable.

About OCX26
Open Community Experience 2026 is the Eclipse Foundation’s flagship event, held in Brussels, Belgium. It brings together developers, architects, and industry leaders to explore open source technologies across domains including AI, automotive, tooling, and cloud systems, with a focus on practical implementation. Learn more at https://www.ocxconf.org/

Chapters
00:00 introduction and testing scale
00:49 Adoptium and AQAvit overview
03:46 testing lifecycle stages
05:28 AI-assisted testing opportunities
10:02 Perf Savvy and performance testing
15:10 Commit Hunter fault detection
17:17 Glitch Witcher defect prediction
21:42 RAG-based assistant (VITAI)
27:20 test generation and combinatorial testing
31:23 test results analytics and TRSS
33:52 challenges and future direction

Видео Commit to quality: AI-enhanced testing in open source канала Open Community Experience