AI is all over the place – new solutions powered by LLMs are shown every day. The test automation industry is in a similar spot and multiple projects attempt to simplify testing with the power of AI. In this talk, I will review approaches to AI-powered test automation, from code generation to direct browser manipulation. I will summarize their pros and cons for real testing. We’ll see how the solutions pass the “2+2=4” test for a simple online calculator. I will introduce Alumnium, a new open source test automation framework. We’ll deep dive into its design, the implementation details and write real tests. I will go over the challenges of building an AI-powered test automation tool (support for multiple LLMs, closed and open models, agentic workflows, reasoning limitations, stability and flakiness, performance optimizations) and show how Alumnium overcomes them. Finally, we will also dream a little bit about what the future of test automation is going to be with the power of AI.