ALL SPEAKERS

Community

András Bocsák

András Bocsák

Product Manager

BSH Hausgeräte GmbH

AT A GLANCE

Role

Product Manager

Organisation

BSH Hausgeräte GmbH

Category

Community

ABOUT THE SPEAKER

András Bocsák is a Product Manager at BSH Hausgeräte GmbH with a focus on conversational AI and production-grade AI systems. Over the past nine years, he has led digital product initiatives across the Home Connect ecosystem. Today, he works on AI Assist, the customer-facing AI assistant in the Home Connect app, where he helps define testing and evaluation strategies for large language model applications. His areas of interest include AI quality assurance, automated evaluations, observability, LLM judges, and continuous improvement of customer-facing AI systems. He is passionate about turning emerging technologies into products that deliver practical value at scale.

TALK

Testing the Untestable? Quality Control for Multi-Agent LLM Systems in Production

“It works on my prompt" is not a QA strategy, but what else can you do when the core of your system is non-deterministic, its tools live on remote servers, and the output is a stream of natural language? Our team runs a multi-agent customer support system in production, and this talk is about how we keep it from breaking. Most of the heavy lifting is still done by the boring classics: unit and integration tests for all the code around the model, because that code is where most bugs actually live. The interesting part sits on top. We built an eval layer with a small golden dataset curated from real conversations, deterministic checks that verify what the agent decided to do (which sub-agent it routed to, which tool it called, whether the answer contains what it must), and LLM judges for the softer qualities that have no objective test. I'll show you what worked, what we had to throw away, and which kind of test catches which kind of failure, with regression stories from our CI pipeline.

MORE VOICES

Other speakers at code.talks

Eric Evans photo
Hero
Eric Evans
Founder & Author ·
#Architecture
BIO
Eric Evans
Founder & Author ·

Eric has long worked in the development of software for complex domain problems. He wrote the book Domain-Driven Design and has been learning new and better ways to do it ever since.

More info →
Gergely Hodicska photo
Hero
Gergely Hodicska
Founder · Water Bear Consulting
#Culture
BIO
Gergely Hodicska
Founder · Water Bear Consulting

Gergely is a product engineering leader and coach with deep experience in building and scaling engineering organizations. He is a former VP of Engineering at Bitrise and Ustream, and has worked across engineering leadership, product development, organizational design, culture, change management, and coaching. He focuses on helping teams build better systems, improve collaboration, and create environments where people can grow and do their best work. He combines leadership experience with strong technical depth, including hands-on software engineering, architecture, developer tooling, and building systems used at significant scale.

More info →
Gitte Klitgaard photo
Hero
Gitte Klitgaard
Founder and Coach at Native Wired · Native Wired
#Culture
BIO
Gitte Klitgaard
Founder and Coach at Native Wired · Native Wired

Gitte Klitgaard is an agile coach, trainer, advisor and mentor solving problems in organisations. She uses strategy, accountability and responsibilty, collaboration, psychological safety, communication, setting good engineering practice, and what ever else is needed. She works with teams, leaders, and individuals on all organisational level. Gitte is authentic, she will cut to the chase, and help people become themselves thereby reaching success.

Gitte knows how combining human skills with good leadership skills is a necessity for an organisation to get stuff done. And getting the right stuff done. Her ability to align the individual well-being and growth together with company objectives is remarkable. One of Gitte’s key strengths is holding people accountable while fostering psychological safety.

Gitte has given talks and keynots at three continents, where she highlights and makes topics like mental health and psychological safety accessible. Other topics include courage, retrospectives, various agile topics, leadership, management, and her personal goal: bringing humanity back in the workplace.

Her community contributions include organizing coach camps for over a decade and actively engaging on social media. She creates safe and respectful environments at work and outside. She listens to and engages the more silent voices and minority groups. Gitte also contributes to the community by writing blogpost and on Bluesky as she shifted away from Twitter. She wrote a section about team safety in "The Art of Agile Development, 2. edition" by James Shore which was published in 2021.

In her spare time, Gitte collects LEGO and Yodas and keeps in touch with friends from all over the globe including some, she considers her second family.

Gitte is owner of Native Wired and has lead change at companies like IBM, LEGO, Spotify, and SVT.

More info →
Luca Rossi photo
Hero
Luca Rossi
Founder · Refactoring
#Culture
BIO
Luca Rossi
Founder · Refactoring

Luca is the founder of Refactoring.fm — a newsletter and community about running better engineering teams, that goes out weekly to more than 170,000 engineers and managers. He has written 300+ articles, and the Refactoring community includes some of the brightest minds in the industry. Before Refactoring, Luca has been Head of Engineering at Translated.com, and co-founder and CTO of Wanderio, where he grew the product from 0 to 25M customers.

More info →

See them live — and 100+ more.

One ticket, every track. Nov 4–5, 2026 at Kinopolis Hamburg.

Get your ticket

Let’s connect on LinkedIn.

Speakers, program drops and much more of the community.

A 2-day sweet spot conference for the entire dev team to stay future-proof in the age of AI.

© 2026 code.talks · alphalist GmbH · Hamburg

Let’s connect on LinkedIn.

Speakers, program drops and much more of the community.

A 2-day sweet spot conference for the entire dev team to stay future-proof in the age of AI.

© 2026 code.talks · alphalist GmbH · Hamburg