- Traces: Individual LLM calls, function executions, and application flows that capture inputs, outputs, latency, and token usage for debugging and analysis.
- Evaluations: Test results that compare model performance against datasets using scoring functions to measure quality, accuracy, and other metrics.
- Monitors: Background processes that automatically score a subset of function calls using LLM-as-a-judge scorers to track quality metrics over time in production.
- Leaderboards: Ranked comparisons of model performance across different evaluation runs, helping you identify the best-performing configurations.
- Threads: Multi-turn conversations and session-based workflows that group related calls under a shared thread ID for analyzing complete user interactions.
- Assets: The Assets tab in the Weave UI stores and organizes the following resources in your project:
- Models: Versioned model definitions that capture prompts, parameters, and configuration, automatically tracking changes for reproducibility and comparison.
- Datasets: Collections of test examples stored as Weave objects that can be downloaded, browsed, and used for running evaluations.
- Prompts: Versioned system prompts that you can test and reuse in your model configurations.
- Scorers: Functions that evaluate model inputs and outputs to measure quality, safety, or custom metrics.
- Ops: Automatically versioned and tracked functions decorated with
@weave.op()that produce calls and maintain a record of function modifications. - Other objects and metadata: Additional Weave objects, custom types, and associated metadata stored within the project.
Creating and using projects
A Weave project is owned and managed by a team (also called an entity) in your W&B organization. The full project path follows this format:my-team with a project called llm-chatbot has the project path my-team/llm-chatbot.
To associate traces, evaluations, and other resources with a project, pass the project’s path as a string when you initialize Weave in your code:
- Python
- TypeScript
Viewing projects in the UI
Access a project in the Weave UI:- Navigate to wandb.ai.
- Select your project from the Projects sidebar. This opens the project’s overview page.
- From the overview page, you can:
- Review your projects’s statistics, such as the total number of traces logged to the project and how much storage the project is using.
- Navigate to traces, evaluations, and other resources using the sidebar.
- Navigate to recently logged traces, evaluations, models, and datasets.
