thoughts from the test eye

Last week I gave a talk at Øredev conference about building my own custom test tools with the help of LLM programming.
I have been doing this for a year now, and it really extends my capabilities as a tester. I can’t say it makes me faster, but I can do better testing, and it is a lot more fun!
I often test web services, and typically I notice something that is difficult or takes time, and then I look in the web traffic to find stuff that a browser plugin can help me with. It can also be a bug that I find, and a new tool can be created so I will find more of the same type with very little effort.
I have also done some plug-ins for JMeter, Zed Attack Proxy and Notepad++ and the nice feeling to have your own tools shouldn’t be underestimated.
The tools are small and specific, not generic (that is difficult!); they are built to work for my unique context.

Test tools have always been important, but with the speed of LLMs you can make them very quickly, and you don’t need to bother the people that really know how to program.
In the slides you can see a bunch of examples and different categories of test tools that I hope can inspire (Passive scanning, Test Improver, Test Data Generators, Validators, Information Collectors, Visualizer, Other).

If you want to try this, you shouldn’t use a chat LLM, becuse the result is so much worse than an IDE that can interact with many files. I use Cursor with Claude Sonnet (GPT-5 and Composer with Cursor are not far behind), but I have heard that Claude Code is at least as good. There are other tools that might be equally useful, and some of these provide free try-out periods.

I do vibe coding with a quite technical first prompt, perfection and maintainability is not important, and I just need something that works on my machine.
I don’t trust the output, but when using it I will quickly discover if it does the job or not.

As a final note I must say that I don’t use LLMs in any other ways at work, the result is just too bad and can’t be trusted.
And it is a long way to letting LLMs create important software, and even longer before it can test that software.
Right now, custom test tools might be the very best use case for LLMs, and I am amazed that a multi-billion industry was created for this!