I was delighted to contribute to a Thoughtworks’ insights article on AI testing in response to Forrester’s recent report It’s Time To Get Really Serious About Testing Your AI. The report rightly raises the importance of testing in AI systems and highlighted Thoughtworks’ Continuous Delivery for Machine Learning (CD4ML) approach. The response also discusses other important elements of testing, for instance controlling sources of variation.
A system designed to be testable enables you to identify all the sources of input variation — such as random seeds, prompt construction, LLM temperature and sampling parameters — and the more of these you can hold constant in a test environment, the more you should expect deterministic behavior.
Read the full article Time to get serious about AI testing