Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the e2e test for metrics #24351

Open
1 of 9 tasks
hjetpoluru opened this issue May 2, 2024 · 0 comments
Open
1 of 9 tasks

Fix the e2e test for metrics #24351

hjetpoluru opened this issue May 2, 2024 · 0 comments

Comments

@hjetpoluru
Copy link
Contributor

hjetpoluru commented May 2, 2024

What is this about?

Issue Overview:
Harika was conducting tests on MixPanel v2, specifically focusing on triggering the 'Transaction Finalized' event. This process involved following the steps outlined in the automation test. To verify the event's occurrence, examined each batch call within the network tab of the background console. Despite these efforts, there was no indication of the 'Transaction Finalized' event being captured.

Further Investigation:
Upon deeper analysis, Dan identified that the 'Transaction Finalized' event has been working in v11.13.1, there were an average 6823 "Transaction Finalized" events, with status "confirmed". per hour. And v11.14.3, there an average of 0.3 "Transaction Finalized" events, with status "confirmed". per hour
https://mixpanel.com/project/2212883/view/2760759/app/insights#Nw55uTKEGnsk

Image

This discovery led to the hypothesis that a specific commit might be the root cause of the issue, potentially affecting the event's proper tracking or triggering within MixPanel v2.

Commit -  https://github.com/MetaMask/core/pull/3827/files#diff-96bc10c64242a8e23f5835532988319189239bdf40f556b86199dcbcc41c7bfcL1353-R1646

PR fix it - #24338

However, the e2e test for the 'Transaction Finalized' event should have failed, but it did not and hence we need to fix the test.

Feedback from Dan Miller

the e2e test for this metric is broken in multiple ways, such that it passes when it shouldn't. I think we will have to fix the e2e test separately from this PR to fix the transaction finalized event.

Scenario

No response

Design

No response

Technical Details

No response

Threat Modeling Framework

No response

Acceptance Criteria

No response

Stakeholder review needed before the work gets merged

  • Engineering (needed in most cases)
  • Design
  • Product
  • QA (automation tests are required to pass before merging PRs but not all changes are covered by automation tests - please review if QA is needed beyond automation tests)
  • Security
  • Legal
  • Marketing
  • Management (please specify)
  • Other (please specify)

References

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant