Real-Time Monitoring and Experimentation with AI in Software Development

Add to Calendar Apr 17, 2025 Apr 17, 2025 America/Los_Angeles Real-Time Monitoring and Experimentation with AI in Software Development This session will explore how AI-powered tools like BraintrustData's real-time monitoring and iterative experimentation can transform software development workflows. Discover how developers can effectively track AI model performance and swiftly iterate with various prompts and models using the playground feature. Learn to leverage these capabilities for rapid improvements and seamless integration of AI into your projects. Main Stage | Gateway Pavilion - Pier 2 | 2 Marina Blvd, San Francisco, CA 94123

About this session

This session will explore how AI-powered tools like BraintrustData's real-time monitoring and iterative experimentation can transform software development workflows. Discover how developers can effectively track AI model performance and swiftly iterate with various prompts and models using the playground feature. Learn to leverage these capabilities for rapid improvements and seamless integration of AI into your projects.

Session Speaker

Ankur Goyal

Founder & CEO

BrainTrust

Access Slides

More sessions

Developer Day

Apr 17, 2025

9:40 am

The Full Stack of Open Generative Ai

Join an AI expert from Meta for an in depth look at the latest advancements in the open generative AI stack. This session will cover from the metal to the agent how large scale AI systems are built, the tools used and how you can build your own with open source AI from Meta including PyTorch, Triton, Llama and more.

Joe Spisak

Ensuring Responsible AI with Advanced Model Evaluation

This session will cover Sama's comprehensive Model Evaluation services, emphasizing the importance of aligning AI systems with ethical guidelines and improving their performance. Attendees will learn how to systematically assess AI outputs, identify inaccuracies and vulnerabilities, and employ strategies for accelerated time-to-market with robust model validation. Leveraging Sama's expert-driven platform, participants will discover how to build reliable, high-performing generative AI models while adhering to industry standards.

Duncan Curtis

Sama



More Information

Developer Day

Apr 17, 2025

10:20 am

Building Agents to Automate Knowledge Work

A huge promise of LLM-powered agents is to not only provide simple Q&A search interfaces, but to automate 50%+ of operational work that a knowledge worker typically does. A lot of the day-to-day of a knowledge worker - whether an investment analyst, customer agent, or engineer - is ingesting a large volume of unstructured data and using that to synthesize new insights or take actions. This talk walks through the e2e architecture for building production knowledge agents that can actually do useful work over your data. This requires both a production-quality knowledge management layer as well as agent architectures that can reliably solve certain use cases.

Jerry Liu

LlamaIndex

