Skip to content

Commit

Permalink
Add command, command options and test bot data
Browse files Browse the repository at this point in the history
These new categories are available from
Homebrew/brew#16847

There's a few TODO here that I'd love help with but don't need to
block getting this data flowing.
  • Loading branch information
MikeMcQuaid committed Apr 26, 2024
1 parent 2c4abe3 commit 7938e01
Showing 1 changed file with 58 additions and 6 deletions.
64 changes: 58 additions & 6 deletions cmd/formula-analytics.rb
Expand Up @@ -33,14 +33,22 @@ def formula_analytics_args
description: "Output Homebrew prefixes."
switch "--homebrew-versions",
description: "Output Homebrew versions."
switch "--brew-command-run",
description: "Output `brew` commands run."
switch "--brew-command-run-options",
description: "Output `brew` commands run with options."
switch "--brew-test-bot-test",
description: "Output `brew test-bot` steps run."
switch "--json",
description: "Output JSON. This is required: plain text support has been removed."
switch "--all-core-formulae-json",
description: "Output a different JSON format containing the JSON data for all " \
"Homebrew/homebrew-core formulae."
switch "--setup",
description: "Install the necessary gems, require them and exit without running a query."
conflicts "--install", "--cask-install", "--install-on-request", "--build-error", "--os-version"
conflicts "--install", "--cask-install", "--install-on-request", "--build-error", "--os-version",
"--homebrew-devcmdrun-developer", "--homebrew-os-arch-ci", "--homebrew-prefixes",
"--homebrew-versions", "--brew-command-run", "--brew-test-bot-test"
conflicts "--json", "--all-core-formulae-json", "--setup"
named_args :none
end
Expand Down Expand Up @@ -113,12 +121,21 @@ def influx_analytics(args)
categories << :homebrew_prefixes if args.homebrew_prefixes?
categories << :homebrew_versions if args.homebrew_versions?
categories << :os_versions if args.os_version?
categories << :command_run if args.brew_command_run?
categories << :command_run_options if args.brew_command_run_options?
categories << :test_bot_test if args.brew_test_bot_test?

category_matching_buckets = [:build_error, :cask_install]
category_matching_buckets = [:build_error, :cask_install, :command_run, :test_bot_test]

categories.each do |category|
additional_where = all_core_formulae_json ? " AND tap_name =~ /homebrew\\/(core|cask)/" : ""
bucket = category_matching_buckets.include?(category) ? category : :formula_install
bucket = if category_matching_buckets.include?(category)
category
elsif category == :command_run_options
:command_run
else
:formula_install
end

case category
when :homebrew_devcmdrun_developer
Expand All @@ -136,6 +153,16 @@ def influx_analytics(args)
when :os_versions
dimension_key = :os_version
groups = [:os_name_and_version]
when :command_run
dimension_key = "command_run"
groups = [:command]
when :command_run_options
dimension_key = "command_run_options"
groups = [:command, :options, :devcmdrun, :developer]
additional_where += " AND ci = 'false'"
when :test_bot_test
dimension_key = "test_bot_test"
groups = [:command, :passed, :arch, :os]
else
dimension_key = if category == :cask_install
:cask
Expand Down Expand Up @@ -189,6 +216,24 @@ def influx_analytics(args)
end
when :os_versions
format_os_version_dimension(tags["os_name_and_version"])
when :command_run_options
"#{tags["command"]} #{tags["options"]}"
when :test_bot_test
command_and_formula, options = tags["command"].split.partition {|arg| !arg.start_with?("-") }

Check failure on line 222 in cmd/formula-analytics.rb

View workflow job for this annotation

GitHub Actions / tests (ubuntu-latest)

Layout/SpaceInsideBlockBraces: Space between { and | missing.

Check failure on line 222 in cmd/formula-analytics.rb

View workflow job for this annotation

GitHub Actions / tests (macos-latest)

Layout/SpaceInsideBlockBraces: Space between { and | missing.

# Cleanup bad data before https://github.com/Homebrew/homebrew-test-bot/pull/1043
# TODO: actually delete this from InfluxDB.
# Can delete this code after 27th April 2025.
next if %w[audit install linkage style test].exclude?(command_and_formula.first)
next if command_and_formula.last.include?("/")
next if options.include?("--tap=")
next if options.include?("--only-dependencies")
next if options.include?("--cached")

command_and_options = (command_and_formula + options.sort).join(" ")
passed = tags["passed"] == "true" ? "PASSED" : "FAILED"

Check failure on line 234 in cmd/formula-analytics.rb

View workflow job for this annotation

GitHub Actions / tests (ubuntu-latest)

Style/TernaryParentheses: Use parentheses for ternary expressions with complex conditions.

Check failure on line 234 in cmd/formula-analytics.rb

View workflow job for this annotation

GitHub Actions / tests (macos-latest)

Style/TernaryParentheses: Use parentheses for ternary expressions with complex conditions.

"#{command_and_options} (#{tags["os"]} #{tags["arch"]}) (#{passed})"
else
tags[groups.first.to_s]
end
Expand All @@ -212,10 +257,9 @@ def influx_analytics(args)
dimension = dimension.strip
next if dimension.match?(/[<>]/)

# we want any valid count that isn't the options out of:
# "time", "count_options", "count_os_name_and_version", "count_package", "count_tap_name", "count_version"
# we want any valid count that isn't the time field
count = nil
result["values"].first.drop(2).find do |possible_count|
result["values"].first.compact.drop(1).find do |possible_count|
break if count.present?

count ||= begin
Expand All @@ -229,7 +273,15 @@ def influx_analytics(args)
rescue ArgumentError, TypeError
nil
end

next if count <= 0

count
end

# TODO: we don't seem to get a valid count for these categories, unclear why.
count ||= 1 if category == :command_run_options || category == :test_bot_test

Check failure on line 283 in cmd/formula-analytics.rb

View workflow job for this annotation

GitHub Actions / tests (ubuntu-latest)

Style/MultipleComparison: Avoid comparing a variable with multiple items in a conditional, use `Array#include?` instead.

Check failure on line 283 in cmd/formula-analytics.rb

View workflow job for this annotation

GitHub Actions / tests (macos-latest)

Style/MultipleComparison: Avoid comparing a variable with multiple items in a conditional, use `Array#include?` instead.

odie "Invalid amount of items" if count.blank?

json[:total_items] += 1
Expand Down

0 comments on commit 7938e01

Please sign in to comment.