diff --git a/docs/error-reporting/project-releases/configure-release-views.md b/docs/error-reporting/project-releases/configure-release-views.md new file mode 100644 index 0000000000..3fd16ad14f --- /dev/null +++ b/docs/error-reporting/project-releases/configure-release-views.md @@ -0,0 +1,76 @@ +--- +id: configure-release-views +title: Configuring Release Views +sidebar_label: Configuring Release Views +description: Choose the error type, set the version attribute, select the releases to compare, and filter the Releases view. +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; + +Use the controls at the top of the **Releases** view to choose which errors to analyze, how releases are identified, and which releases to compare. The charts and comparison table update automatically when you change these settings. + +## Choose the Error Type + +The **Error type** dropdown filters the metrics on the page to a specific type of error. Each option shows the number of errors reported for that type. + +**Step 1:** Click the **Error type** dropdown. + +Error type dropdown at the top of the Releases view + +**Step 2:** Select **All**, or a single error type such as **Unhandled exception**, **Exception**, **Crash**, **Hang**, or **Low memory**. + +The available error types depend on the errors reported by your project, so your options may differ. The number next to each option shows the error count for the selected time frame. + +Error type dropdown expanded with the error count for each type + +:::tip +Select **Crash** or **Hang** to compare releases by the errors that disrupt users the most. +::: + +## Set the Version Attribute + +The **Compare releases by** dropdown determines which attribute is used to identify releases. For example, selecting `application.version` groups the data by application version. + +**Step 1:** Click the **Compare releases by** dropdown. + +Compare releases by dropdown at the top of the Releases view + +**Step 2:** Select the attribute that represents a release in your project, such as `application.version`, `application.marketing_version`, or `backtrace.version`. + +Compare releases by dropdown expanded with the available version attributes + +:::note +To use a custom attribute, make sure it's properly configured and indexed. Attributes that aren't indexed don't appear in the dropdown. +::: + +## Select Releases to Compare + +The **Comparing releases** dropdown lists the releases available for the selected attribute. Each release includes its total error count. + +:::note +You can compare up to four releases at a time. +::: + +**Step 1:** Click the **Comparing releases** dropdown. + +Comparing releases dropdown at the top of the Releases view + +**Step 2:** Select the checkbox next to each release you want to compare. Clear the checkbox to remove a release from the comparison. + +The selected releases appear in the dropdown. The number next to each release shows its error count. + +Comparing releases dropdown expanded with a checkbox and error count for each release + +:::tip +Include at least one earlier release you know was stable, so you have a baseline to compare against. +::: + +## Filter the View + +The Releases view also responds to the project's selected **Time frame** and the global filter bar. You can combine these filters with the release controls to focus on specific data. + +| Goal | How to do it | +| ----- | ----- | +| Monitor a release during its launch window | Set **Time frame** to the most recent couple of days. | +| Compare crashes only | Select **Crash** in the **Error type** dropdown. | +| Exclude development builds | If your release attribute follows a pattern that separates development builds from production builds, filter out non-production builds in the global filter bar. | diff --git a/docs/error-reporting/project-releases/release-health-metrics.md b/docs/error-reporting/project-releases/release-health-metrics.md new file mode 100644 index 0000000000..c2683d01ea --- /dev/null +++ b/docs/error-reporting/project-releases/release-health-metrics.md @@ -0,0 +1,56 @@ +--- +id: release-health-metrics +title: Check Release Health Metrics +sidebar_label: Check Release Health Metrics +description: Track user adoption, error-free sessions, errors over time, and new errors over time for each release in the Releases view. +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; + +The **Releases** view includes four charts that help you monitor the health of each release over time. Each chart displays a separate line for each selected release, making it easier to compare release stability during and after a launch. + +The charts use the **Error type**, time frame, and filters selected in the Releases view. The chart titles also reflect the attribute selected in **Compare releases by**, such as `application.version`. + +Error type dropdown at the top of the Releases view, above the health charts + +## User Adoption + +The **User adoption** chart shows the percentage of total user events associated with each release over time. Use this chart to understand how quickly users move to a new release. + +Consider user adoption when reviewing the other health metrics. A release with fewer users may appear more stable because it has less traffic. + +User adoption chart with a line for each selected release + +After a release launches, its user adoption should generally increase as users upgrade. Similar adoption levels across releases provide a better basis for comparing their health. + +## Error-Free Sessions + +The **Error-free application launches** chart shows the percentage of application launches for each release where no errors occurred. A higher percentage indicates a more stable release. + +Error-free application launches chart with a line for each selected release + +Look for a release whose percentage remains consistently lower than the others. This can indicate that the release is experiencing more errors during application launches. + +## Errors Over Time + +The **Errors over time** chart shows the number of errors reported for each release over the selected time period. Use it to identify error spikes and monitor how a release performs after launch. + +Hover over the chart to view the error count for each release at a specific point in time. + +Errors over time chart with a line for each selected release + +Look for releases with significantly higher error counts than other releases with similar user adoption. Sharp increases or spikes may indicate a stability issue that requires investigation. + +## New Errors Over Time + +The **New Errors over time** chart shows the number of errors that have not been seen previously, grouped by release. Use this chart to identify errors that may have been introduced by a new release rather than existing errors or previously known issues. + +Hover over the chart to view the new error count for each release at a specific point in time. + +New errors over time chart with a line for each selected release + +A rise above zero shortly after a release is launched can indicate a potential regression. Investigate these new errors to determine whether they are related to the release. + +:::note +When every chart drops to zero for all releases at the same time, it usually reflects a gap in incoming data rather than a real change in stability. +::: diff --git a/docs/error-reporting/project-releases/stability-metrics-table.md b/docs/error-reporting/project-releases/stability-metrics-table.md new file mode 100644 index 0000000000..b0d68a0fc9 --- /dev/null +++ b/docs/error-reporting/project-releases/stability-metrics-table.md @@ -0,0 +1,55 @@ +--- +id: stability-metrics-table +title: Stability Metrics Table +sidebar_label: Stability Metrics Table +description: Compare aggregated stability metrics and triage status for each release, and open a release in the Triage view. +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; + +The **Stability Metrics** table at the bottom of the **Releases** view provides an overview of the health and stability of each selected release. The table title includes the attribute used for comparison, such as **Compare groups by application.version**. + +Compare groups table with one row per selected release + +## Metrics in the Table + +| Ref. | Column | What it shows | +| ----- | ----- | ----- | +| 1 | Release | The release name and color, plus a link icon that opens the release in the Triage view. | +| 2 | User adoption | The release's share of user activity, as a percentage. | +| 3 | Error-free Application Launches | The percentage of application launches (sessions) in which no errors occurred. | +| 4 | Error-free users | The percentage of users who haven't experienced an error. | +| 5 | New errors | The number of errors never seen before this release. | +| 6 | Unique errors | The number of distinct errors reported for the release. | +| 7 | Total errors | The total number of errors reported for the release. | +| 8 | Triage status | A bar showing how the release's errors split across triage states, labeled with the largest state, its error count, and its share. | + +Compare groups table with each column numbered + +### Understand Triage Status + +The **Triage status** column shows the distribution of errors across different triage states. + +For example, **in-progress (316) 94.33%** means that 316 of the release's 335 total errors, or 94.33%, are in progress. + +### Understand Percentage Metrics + +Percentage-based metrics include a visual bar below the value. The dark portion represents the reported percentage, while the gray portion represents the remaining percentage up to 100%. This makes it easier to compare values across releases. + +### View Details for a Percentage Metric + +Hover over a percentage metric to view the numbers used to calculate it. + +For example, hovering over **Error-free Application Launches** displays the number of error-free launches and the total number of launches. + +## Open a Release in Triage + +You can open a release directly in the **Triage** view to investigate its errors. + +**Step 1:** Find the release you want to investigate in the table and click the **link icon** next to the release name. + +Link icon next to each release name in the compare groups table + +The **Triage** view opens with the selected release already applied as a filter. + +In the Triage view, errors are grouped by fingerprints. Start by reviewing fingerprints that affect the most users or have the highest number of reported errors. For more information, see [Triage](/docs/error-reporting/web-console/triage.md). diff --git a/docs/error-reporting/web-console/releases.md b/docs/error-reporting/web-console/releases.md index a27dca4297..a8cfb1316f 100644 --- a/docs/error-reporting/web-console/releases.md +++ b/docs/error-reporting/web-console/releases.md @@ -1,95 +1,48 @@ --- id: releases -title: Releases -sidebar_label: Releases -description: Monitor stability across released versions of your app with the Releases page. +title: Project Releases +sidebar_label: Overview +description: Monitor and compare the stability, user adoption, and overall health of your application's releases with the Releases view. --- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; import useBaseUrl from '@docusaurus/useBaseUrl'; -The Backtrace Web Console's Release view gives team leads, project managers, and developers an easy way to monitor and compare release stability, user adoption, and other health metrics. +The **Releases** view in the Backtrace Web Console lets you monitor and compare the stability, user adoption, and overall health of your application's releases. It's built for team leads, project managers, and developers who need a quick read on how each version is performing. -- Monitor an important release as it is launched to quickly identify and prioritize impactful issues -- Retroactively compare and explore release stability to learn and improve your processes going forward -- Quickly understand the state of your application across versions +Use the Releases view to: -The Releases view allows you to monitor release stability. -Shows the stability metrics for your project's releases. +* **Monitor a launch.** Watch an important release as it rolls out, and quickly identify and prioritize the issues with the most impact. +* **Compare past releases.** Look back at release stability over time to learn from earlier launches and improve your release process. +* **Understand your application across versions.** See the state of every release you care about on a single page. -You can see the user adoption of the versions, the error-free application launches, errors over time, and new errors over time. Additionally, there are quick toggles for error types (crashes, hangs, exceptions) and platforms (iOS, Android, ps4, Nintendo Switch, Xbox, Windows, Linux). The quick toggles are also available in the Overview section. +## Open the Releases View -## Configuration +**Step 1:** In the Backtrace Web Console, select your project and go to the **Overview** tab. -### Set the Version Attribute +Overview tab in the Backtrace Web Console navigation bar -You will want to ensure the Release Comparison View's concept of a "Release" is represented by the attribute most relevant to you. Use the pencil icon next to the "Comparing by \{attribute}" text at the top of the view to set this as desired. +**Step 2:** Click **View project releases** in the upper-right corner of the page. The **Releases** view opens. -Shows how to set the version attribute for the Releases view. +View project releases button in the upper-right corner of the Overview page -This will bring up a dialogue box that will let you change the attribute which underlies the view. If you are using a custom attribute, make sure that it is properly configured and indexed or it won't show up as an option here. +The **Releases** view includes three sections: -Shows the dialog to set the version attribute for the Releases view. +| Ref. | Section | Description | +| ----- | ----- | ----- | +| 1 | Controls | The **Error type**, **Compare releases by**, and **Comparing releases** dropdowns, which decide what the page compares. | +| 2 | Health charts | Four charts that track each release through time: user adoption, error-free application launches, errors over time, and new errors over time. | +| 3 | Compare groups table | One row per release, with aggregated stability metrics and triage status. | -### Select Versions to Compare +Releases view with the controls, health charts, and compare groups table numbered -To edit which release versions are being compared on the page, use the pencil icon next to the version numbers towards the top of the page. Note that you can also press the "x" icon next to each version number to quickly exclude it from the list. +Each chart and the table use the same color for a given release, so you can follow one version across the whole page. -Shows the dialog to set the version attribute for the Releases view. +:::tip +Use the Overview page for a quick, project-wide health check. Open the Releases view when you need to know which release is responsible for a change. +::: -Pressing the pencil icon will bring up the following dialogue box which can be used to include / exclude versions. Note that you can compare up to 4 versions at a time. +## In This Section -Shows the dialog to set the version attribute for the Releases view. - -## Release Health Metrics - -The Release Comparison View offers a number of metrics + visualizations. Note that every metric on the page is responsive to updates in the global filter bar at the top of the page (as with other Backtrace views). - -### Through Time Monitoring - -The top section of the page provides a view into the error rates of your project's releases through time, a great way to track the evolution of a release's stability through a launch window and beyond. - -- Errors over time: Shows your project's error count through time, broken down by release. - -- New Errors over time: Shows the count of errors never before seen through time, broken down by release. Great for distinguishing errors introduced by a new release from existing errors / backlog items. - -To see a quick breakdown of errors by release at a given point in time, hover your mouse over the charts. - -Shows errors over time by version. - -Note that as of October 2021, the Backtrace team is planning to add metrics to the page over the coming month including a visualization of user adoption through time and session stability through time. - -### Stability Metrics Table - -The bottom of the page lays out each release with aggregated health / stability metrics. This includes: - -- User adoption % -- Error-free sessions (% of sessions in which there were no errors) -- Error-free users (% of users who have not experienced an error) -- New errors -- Unique errors -- Total errors -- Triage status (distribution %) - -Shows the stability metrics for your project's releases. - -Note that you can see more information for each of the % metrics by hovering over it with your mouse. For example, by hovering over one of the error-free sessions metrics, you can see the number of error-free sessions and total number of sessions. - -Shows hover over details for error-free sessions in the stability metrics table. - -#### View Release Details - -If you want to drill into a release's details, you can use the button next to the release's name in the health metrics table to quickly open the Triage view, filtered to the selected release. - -Drills into the release's details in the Triage view. - -## Global Filter - -Every widget / health metric on the Release Comparison View is responsive to filters via the global filter bar. Use this in combination with the other features on the page to get the most value out of the page - -Example use cases: - -- Time frame: Filter to only the most recent couple days / errors to carefully monitor a release during the launch window. -- Logs vs. Errors vs. Crashes: Add the filter "error.type = Crash" to compare crashes only. This can be especially helpful in identifying and prioritizing the most problematic errors -- Filter Out Irrelevant Builds: If your release attribute has a pattern that makes it easy to identify development builds vs. production builds, you can filter out non-production builds to reduce noise on the page to focus on what you really care about. +* **[Configuring Release Views](/docs/error-reporting/project-releases/configure-release-views.md):** Choose the error type, set the version attribute, select the releases to compare, and filter the view. +* **[Check Release Health Metrics](/docs/error-reporting/project-releases/release-health-metrics.md):** Track user adoption, error-free sessions, errors over time, and new errors over time for each release. +* **[Stability Metrics Table](/docs/error-reporting/project-releases/stability-metrics-table.md):** Compare aggregated stability metrics and triage status per release, and drill into a release in the Triage view. diff --git a/sidebars.js b/sidebars.js index a55ac0c5be..070e9e6f0d 100644 --- a/sidebars.js +++ b/sidebars.js @@ -227,6 +227,17 @@ module.exports = { 'error-reporting/mcp/error-reporting-mcp-tools', ], }, + { + type: 'category', + label: 'Project Releases', + collapsed: true, + items: [ + 'error-reporting/web-console/releases', + 'error-reporting/project-releases/configure-release-views', + 'error-reporting/project-releases/release-health-metrics', + 'error-reporting/project-releases/stability-metrics-table', + ], + }, { type: 'category', label: 'Web Console Views', @@ -234,7 +245,6 @@ module.exports = { items: [ 'error-reporting/web-console/getting-started', 'error-reporting/web-console/overview', - 'error-reporting/web-console/releases', 'error-reporting/web-console/flame-graphs', 'error-reporting/web-console/debug', ], diff --git a/static/img/error-reporting/project-releases/configure-views/configure-views-1.png b/static/img/error-reporting/project-releases/configure-views/configure-views-1.png new file mode 100644 index 0000000000..d0c1d297e5 Binary files /dev/null and b/static/img/error-reporting/project-releases/configure-views/configure-views-1.png differ diff --git a/static/img/error-reporting/project-releases/configure-views/configure-views-2.png b/static/img/error-reporting/project-releases/configure-views/configure-views-2.png new file mode 100644 index 0000000000..d675c3ea7b Binary files /dev/null and b/static/img/error-reporting/project-releases/configure-views/configure-views-2.png differ diff --git a/static/img/error-reporting/project-releases/configure-views/configure-views-3.png b/static/img/error-reporting/project-releases/configure-views/configure-views-3.png new file mode 100644 index 0000000000..64c9989d49 Binary files /dev/null and b/static/img/error-reporting/project-releases/configure-views/configure-views-3.png differ diff --git a/static/img/error-reporting/project-releases/configure-views/configure-views-4.png b/static/img/error-reporting/project-releases/configure-views/configure-views-4.png new file mode 100644 index 0000000000..205a976ed1 Binary files /dev/null and b/static/img/error-reporting/project-releases/configure-views/configure-views-4.png differ diff --git a/static/img/error-reporting/project-releases/configure-views/configure-views-5.png b/static/img/error-reporting/project-releases/configure-views/configure-views-5.png new file mode 100644 index 0000000000..9343b41093 Binary files /dev/null and b/static/img/error-reporting/project-releases/configure-views/configure-views-5.png differ diff --git a/static/img/error-reporting/project-releases/configure-views/configure-views-6.png b/static/img/error-reporting/project-releases/configure-views/configure-views-6.png new file mode 100644 index 0000000000..d0905337c6 Binary files /dev/null and b/static/img/error-reporting/project-releases/configure-views/configure-views-6.png differ diff --git a/static/img/error-reporting/project-releases/health-metrics/health-metrics-1.png b/static/img/error-reporting/project-releases/health-metrics/health-metrics-1.png new file mode 100644 index 0000000000..b2dd16ef58 Binary files /dev/null and b/static/img/error-reporting/project-releases/health-metrics/health-metrics-1.png differ diff --git a/static/img/error-reporting/project-releases/health-metrics/health-metrics-2.png b/static/img/error-reporting/project-releases/health-metrics/health-metrics-2.png new file mode 100644 index 0000000000..243180c4ed Binary files /dev/null and b/static/img/error-reporting/project-releases/health-metrics/health-metrics-2.png differ diff --git a/static/img/error-reporting/project-releases/health-metrics/health-metrics-3.png b/static/img/error-reporting/project-releases/health-metrics/health-metrics-3.png new file mode 100644 index 0000000000..4755fb8f55 Binary files /dev/null and b/static/img/error-reporting/project-releases/health-metrics/health-metrics-3.png differ diff --git a/static/img/error-reporting/project-releases/health-metrics/health-metrics-4.png b/static/img/error-reporting/project-releases/health-metrics/health-metrics-4.png new file mode 100644 index 0000000000..93e914188c Binary files /dev/null and b/static/img/error-reporting/project-releases/health-metrics/health-metrics-4.png differ diff --git a/static/img/error-reporting/project-releases/health-metrics/health-metrics-5.png b/static/img/error-reporting/project-releases/health-metrics/health-metrics-5.png new file mode 100644 index 0000000000..2ddef5dd7f Binary files /dev/null and b/static/img/error-reporting/project-releases/health-metrics/health-metrics-5.png differ diff --git a/static/img/error-reporting/project-releases/overview/releases-overview-1.png b/static/img/error-reporting/project-releases/overview/releases-overview-1.png new file mode 100644 index 0000000000..55abcd3386 Binary files /dev/null and b/static/img/error-reporting/project-releases/overview/releases-overview-1.png differ diff --git a/static/img/error-reporting/project-releases/overview/releases-overview-2.png b/static/img/error-reporting/project-releases/overview/releases-overview-2.png new file mode 100644 index 0000000000..8197f30a6d Binary files /dev/null and b/static/img/error-reporting/project-releases/overview/releases-overview-2.png differ diff --git a/static/img/error-reporting/project-releases/overview/releases-overview-3.png b/static/img/error-reporting/project-releases/overview/releases-overview-3.png new file mode 100644 index 0000000000..44870a3dac Binary files /dev/null and b/static/img/error-reporting/project-releases/overview/releases-overview-3.png differ diff --git a/static/img/error-reporting/project-releases/stability-table/stability-table-1.png b/static/img/error-reporting/project-releases/stability-table/stability-table-1.png new file mode 100644 index 0000000000..407452e083 Binary files /dev/null and b/static/img/error-reporting/project-releases/stability-table/stability-table-1.png differ diff --git a/static/img/error-reporting/project-releases/stability-table/stability-table-2.png b/static/img/error-reporting/project-releases/stability-table/stability-table-2.png new file mode 100644 index 0000000000..2a8db96d59 Binary files /dev/null and b/static/img/error-reporting/project-releases/stability-table/stability-table-2.png differ diff --git a/static/img/error-reporting/project-releases/stability-table/stability-table-3.png b/static/img/error-reporting/project-releases/stability-table/stability-table-3.png new file mode 100644 index 0000000000..a81b9e0fff Binary files /dev/null and b/static/img/error-reporting/project-releases/stability-table/stability-table-3.png differ