Workshop: Building Trustworthy LLM Pipelines

radionice

ponedeljak, 09. jun 2025.

18:00h

ICT Hub - Kralja Milana 10, Beograd

Potrebna prijava

As part of AI Week, Machinery is organizing a workshop focused on two essential components of modern LLM-based systems: structured outputs and RAG evaluation.

Workshop: Building Trustworthy LLM Pipelines

The first session will cover how structured outputs can reduce model errors, enforce schema consistency, and support tasks like document data extraction and Retrieval-Augmented Generation (RAG). Tools such as Pydantic and Visual Language Models will be introduced, along with techniques for contextual metadata generation and structured reasoning.

The second session will focus on evaluating RAG pipelines using standard metrics (e.g., recall@k), LLM-based tools like DeepEval and RAGAS, and custom evaluation criteria for citation accuracy, content quality, and formatting. Participants will also define their own metrics suited to real-world applications.

The workshop is intended for professionals developing or evaluating LLM-based systems.