🖥️Run Analytics
This monitoring feature of our platform is designed to offer users detailed insights into their inference-related statistics, with a focus on flexibility, real-time data presentation, and ease of analysis through tagging. By leveraging the features and best practices outlined below, users can effectively monitor and analyze their models' performance and operational metrics. The Run Analytics page is designed to dynamically adjust displayed data in real-time based on selected filters, including tags and time duration. This allows users to customize their monitoring experience and focus on the metrics that matter most to their needs.
Features
Aggregated Data Viewing: Users can view aggregated data for each experiment, model, and other categories based on the tags created. This enables a consolidated view of related metrics for easier analysis.
Real-time Charts: The platform offers real-time charts for key metrics such as latency, cost, prompt tokens, and output tokens used. These charts provide immediate insights into prompts passed to models.
Data Sharing and Export: For further analysis, data can be exported in CSV and other formats. Users also have the ability to share a view with given set of filters for easy collaboration.
Page Components
Filtering
Users are able to view information on runs based on inputting a date range, model(s), and/or tags at the top of the page. The below information will populate (if there is data that matches the filter criteria) after clicking on "Filter":
Latency: The average latency for all the runs in the time period
Tokens: The total prompt and completion tokens used
Cost: The cost incurred for all inferences for the given time period
Queries: The total number of queries in the given time period
Charts
Below this, there are four collapsible tables (if there is data for the given filter) for Latency, Cost, Queries, and Tokens. Below is an example screenshot:
Runs Data
Similar to the Prompt Runs table, this table displays high-level run data based on the given filter.
Tag Leverage
Tags are a powerful tool for filtering and aggregating data on the Run Analytics page. Here’s how to use them effectively:
Filtering with Tags: Use the top filter bar to input all relevant tags for the data you wish to analyze. Tags allow for a focused view of aggregated stats across various metrics.
Tagging Strategy: We recommend tagging runs based on an experiment-level tag. Then, use this single tag in the monitoring view to obtain both aggregated and granular views of all runs related to that experiment. This approach simplifies the process of monitoring and analyzing specific experiments or models.
Last updated
