Skip to content

Latest commit

 

History

History
140 lines (91 loc) · 3.42 KB

telemetry.md

File metadata and controls

140 lines (91 loc) · 3.42 KB
Raw

Telemetry

This doc describes what telemetry is currently recorded. This doc should evolve over time to match the implementation.

If we should change something file an issue. Thanks!

Our general goals are:

  1. Record general usage information.
  2. Don't record any text that is inspected.
  3. Don't record the URLs of any sites visited: except for Azure DevOps or Github organizations.

Individual Users

Application Insights automatically records a random and unique user_Id value for page views and custom events such as IsVTD5OcT+9c4g7k8pN+Qp.

This user_Id should remain stable as long as the browser extension is installed. If you uninstall/reinstall the extension, the user_Id will change.

Page Views

URLs are only recorded for:

  1. https://github.com/[organization]/[repository]
  2. https://dev.azure.com/*
  3. https://devdiv.visualstudio.com/*
  4. https://msazure.visualstudio.com/*

If we don't hit one of these cases, not_specified will be recorded.

Custom Events

All events are sent to the App Insights SDK at the time they occurred. It may take a few minutes for values to show up on the backend.

All values have a timestamp value recorded by the App Insights SDK.

appliedSuggestion

The user clicked a replacement, such as allowlist:

appliedSuggestion event

We also report a running total since the extension was installed via customMeasurements:

{"total":1}

manualFix

This is somewhat more complicated scenario:

  1. The user typed some text.
  2. The extension recommended a change (either via negativeWord or negativeSentence).
  3. The user changed the text.
  4. Now the extension finds no issues.

appliedSuggestion should not trigger manualFix, file a bug if it does!

ignoreSuggestion

The user clicked Ignore in this text:

ignoreSuggestion event

NOTE: Turn off rule everywhere is removed in latest version.

negativeWord

  1. The user pauses typing, and a negative "word" is underlined.

See the full word list in suggestions.js.

negativeSentence

  1. The user pauses typing, and Azure Text Analytics reported a negative sentiment with a 75% confidence or higher.

We also report customMeasurements such as:

{"positive":0,"neutral":0,"negative":1}

askAI

The user clicked Ask an AI for suggestions.

requestSuggestionsPopup

The user has clicked something that would present a popup of some sort with suggestions or a way to get suggestions.

  1. The user clicked the underlined text
  2. The user has clicked the bottom-left corner indicator

We also report customMeasurements such as:

{ "type": "inlineErrorCard" }

Or

{ "type": "indicatorDialog" }

Queries

These are just a list of Kusto queries for looking at the data:

//Users
pageViews
| where timestamp >= datetime(01/01/2022)
| summarize count() by user_Id, bin(timestamp, 7d)
| summarize Users=count() by timestamp
| order by timestamp

// Sessions
pageViews
| where timestamp >= datetime(01/01/2022)
| summarize count() by session_Id, bin(timestamp, 7d)
| summarize Sessions=count() by timestamp
| order by timestamp

// Comments improved
customEvents
| where name == "appliedSuggestion" or name == "manualFix"
| where timestamp >= datetime(01/01/2022)
| summarize Comments=count() by bin(timestamp, 7d)
| order by timestamp