Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

analytics: support command and test-bot analytics. #16847

Merged
merged 1 commit into from Apr 25, 2024
Merged

analytics: support command and test-bot analytics. #16847

merged 1 commit into from Apr 25, 2024
+103 −4

Conversation

MikeMcQuaid
Copy link
Member

These are used to analyse which commands are used and the success/failure rate of official taps using brew test-bot.

@MikeMcQuaid MikeMcQuaid requested review from SMillerDev and a team March 7, 2024 15:24
@MikeMcQuaid MikeMcQuaid marked this pull request as draft March 7, 2024 15:25
@MikeMcQuaid MikeMcQuaid mentioned this pull request Mar 7, 2024
Merged
@MikeMcQuaid MikeMcQuaid force-pushed the analytics_command_run_test_bot branch from ecf962a to fd9ceb0 Compare March 7, 2024 15:29
@MikeMcQuaid MikeMcQuaid marked this pull request as ready for review March 7, 2024 15:30
Bo98
Bo98 reviewed Mar 7, 2024
View reviewed changes
Library/Homebrew/brew.rb Outdated Show resolved Hide resolved
Base automatically changed from analytics_cleanup to master March 7, 2024 15:53
@SMillerDev
Copy link
Member

Just curious, would this log bump and bump-formula-pr if I run brew bump --open-pr php?

@MikeMcQuaid
Copy link
Member Author

Just curious, would this log bump and bump-formula-pr if I run brew bump --open-pr php?

@SMillerDev Yes, I think so.

@SMillerDev
Copy link
Member

Something to keep in mind then

@github-actions GitHub Actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale No recent activity label Mar 29, 2024
@MikeMcQuaid
Copy link
Member Author

I will get back to this 🔜

@github-actions github-actions bot removed the stale No recent activity label Mar 29, 2024
@github-actions GitHub Actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale No recent activity label Apr 20, 2024
@MikeMcQuaid
Copy link
Member Author

Will pick this back up 🔜

@MikeMcQuaid MikeMcQuaid force-pushed the analytics_command_run_test_bot branch from fd9ceb0 to f9f47e3 Compare April 25, 2024 13:54
MikeMcQuaid added a commit to Homebrew/homebrew-core that referenced this pull request Apr 25, 2024
@MikeMcQuaid
workflows/tests: enable analytics for test bot.
7b0d87e 
Will start logging test bot steps and whether they pass or fail.

Requires Homebrew/brew#16847
Requires Homebrew/homebrew-test-bot#1025
@MikeMcQuaid MikeMcQuaid mentioned this pull request Apr 25, 2024
Open
@SMillerDev
Copy link
Member

Do we record build time anywhere? That would be very valuable for tweaking CI parameters

@MikeMcQuaid
Copy link
Member Author

Do we record build time anywhere? That would be very valuable for tweaking CI parameters

@SMillerDev Job time or step time? I think that's outside the scope here but could be a future extension. Would need to figure out how it would be queried.

@carlocab carlocab removed the stale No recent activity label Apr 25, 2024
@SMillerDev
Copy link
Member

Job time or step time? I think that's outside the scope here but could be a future extension. Would need to figure out how it would be queried.

Yeah, I think it's outside the scope too. That's why I did the approval first. I think just the build step is the most valuable metric, to see how much faster ARM is and how we can adjust our build timeout to that.

Bo98
Bo98 approved these changes Apr 25, 2024
View reviewed changes
Copy link
Member

@Bo98 Bo98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks significantly better - thanks!

Would it be useful to add any of the information contained in default_package_tags?

@MikeMcQuaid
Copy link
Member Author

Yeah, I think it's outside the scope too. That's why I did the approval first. I think just the build step is the most valuable metric, to see how much faster ARM is and how we can adjust our build timeout to that.

I think using the e.g. Actions API would be a better fit here.

Would it be useful to add any of the information contained in default_package_tags?

@Bo98 Do you have any particular in mind? I guess I'd only see them being useful if we want to filter the commands run there. The only one that might be relevant is CI but not sure we'd want to have separate pages/entries based on CI?

@Bo98
Copy link
Member

Bo98 commented Apr 25, 2024 •
edited

Nothing specific at the moment, just wanted to raise the question. There might be use cases like seeing if a non-dev cmd is used primarily by devcmdrun users, or potentially explaining a low usage to a flag to being used primarily by a specific system setup, but I haven't thought much about the use scenarios here.

@MikeMcQuaid MikeMcQuaid force-pushed the analytics_command_run_test_bot branch from f9f47e3 to f4dabd9 Compare April 25, 2024 16:29
@MikeMcQuaid
analytics: support command and test-bot analytics.
28c3215 
These are used to analyse which commands are used and the
success/failure rate of official taps using `brew test-bot`.
@MikeMcQuaid MikeMcQuaid force-pushed the analytics_command_run_test_bot branch from f4dabd9 to 28c3215 Compare April 25, 2024 16:38
@MikeMcQuaid
Copy link
Member Author

@Bo98 Good idea, added a few more items.

@MikeMcQuaid MikeMcQuaid merged commit f381b2c into master Apr 25, 2024
26 checks passed
@MikeMcQuaid MikeMcQuaid deleted the analytics_command_run_test_bot branch April 25, 2024 16:52
MikeMcQuaid added a commit to Homebrew/homebrew-formula-analytics that referenced this pull request Apr 26, 2024
@MikeMcQuaid
Add command, command options and test bot data
7938e01 
These new categories are available from
Homebrew/brew#16847

There's a few TODO here that I'd love help with but don't need to
block getting this data flowing.
@MikeMcQuaid MikeMcQuaid mentioned this pull request Apr 26, 2024
Merged
MikeMcQuaid added a commit to Homebrew/homebrew-formula-analytics that referenced this pull request Apr 26, 2024
@MikeMcQuaid
Add command, command options and test bot data
21a0b65 
These new categories are available from
Homebrew/brew#16847

There's a few TODO here that I'd love help with but don't need to
block getting this data flowing.
MikeMcQuaid added a commit to Homebrew/homebrew-formula-analytics that referenced this pull request Apr 26, 2024
@MikeMcQuaid
Add command, command options and test bot data
0503d65 
These new categories are available from
Homebrew/brew#16847

There's a few TODO here that I'd love help with but don't need to
block getting this data flowing.
MikeMcQuaid added a commit to Homebrew/homebrew-formula-analytics that referenced this pull request Apr 26, 2024
@MikeMcQuaid
Add command, command options and test bot data
16c8fcb 
These new categories are available from
Homebrew/brew#16847

There's a few TODO here that I'd love help with but don't need to
block getting this data flowing.
MikeMcQuaid added a commit to Homebrew/homebrew-formula-analytics that referenced this pull request Apr 26, 2024
@MikeMcQuaid
Add command, command options and test bot data
dba4309 
These new categories are available from
Homebrew/brew#16847

There's a few TODO here that I'd love help with but don't need to
block getting this data flowing.
MikeMcQuaid added a commit to Homebrew/homebrew-formula-analytics that referenced this pull request Apr 26, 2024
@MikeMcQuaid
Add command, command options and test bot data
788e815 
These new categories are available from
Homebrew/brew#16847

There's a few TODO here that I'd love help with but don't need to
block getting this data flowing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Bo98 Bo98 Bo98 approved these changes

@SMillerDev SMillerDev SMillerDev approved these changes

@carlocab carlocab carlocab approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@MikeMcQuaid @SMillerDev @Bo98 @carlocab