Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Inconsistent and flaky unit-tests #2892

Merged
merged 50 commits into from
Mar 8, 2024
Merged

fix: Inconsistent and flaky unit-tests #2892

merged 50 commits into from
Mar 8, 2024

Conversation

gaby
Copy link
Member

@gaby gaby commented Mar 3, 2024

Description

  • Fixed conflicting Key being used in storage/memory tests. There was multiple unit-tests using the same key "John".
  • Remove usage of testify.require inside goroutines created during tests
  • Update internal/storage/memory to match https://github.com/gofiber/storage/tree/main/memory
  • Added several new Benchmarks for internal/storage/memory
  • Added several new benchmarks for middleware/session

Fixes #2891 #2872

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • Enhancement (improvement to existing features and functionality)
  • Documentation update (changes to documentation)
  • Performance improvement (non-breaking change which improves efficiency)
  • Code consistency (non-breaking change which improves code reliability and robustness)

Summary by CodeRabbit

Summary by CodeRabbit

  • Refactor
    • Improved favicon data handling and HTTP request method logic.
    • Centralized server startup logic in tests for better organization.
  • Tests
    • Updated client and context tests to include server readiness checks and added timeout values.
    • Removed parallel execution in limiter middleware tests to enhance stability.

@gaby gaby requested a review from a team as a code owner March 3, 2024 06:06
@gaby gaby requested review from sixcolors, ReneWerner87 and efectn and removed request for a team March 3, 2024 06:06
Copy link
Contributor

coderabbitai bot commented Mar 3, 2024

Note

Reviews Paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Walkthrough

The recent updates encompass a variety of enhancements across the codebase, focusing on refining data handling, improving code organization, and enhancing test reliability. Key areas of improvement include the management of favicon data, the centralization of server startup logic in tests, adjustments in HTTP request handling, and the removal of concurrent execution in limiter tests. These changes collectively aim to streamline operations, boost readability, and ensure more robust testing practices.

Changes

Files Summary
middleware/favicon/favicon.go, middleware/proxy/proxy_test.go Renamed variables, updated icon length calculation, centralized server startup logic in tests.
client_test.go, ctx_test.go Introduced startServer function, added timeout value in app.Test calls for better test control.
middleware/limiter/limiter_test.go, .github/workflows/test.yml Removed t.Parallel() calls, updated job name in workflow configuration.

🐇💻✨
In the realm of code, where logic intertwines,
A rabbit hopped through, refining the lines.
With a flick and a hop, enhancements were made,
In the glow of the moon, under the code's shade.
"To improve and to fix," the rabbit declared,
Leaving behind code, far better prepared.
🌟🐾

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit-tests for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit tests for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit tests.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 3

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 67d35dc and e3db3b5.
Files selected for processing (5)
  • app_test.go (2 hunks)
  • client_test.go (69 hunks)
  • listen_test.go (5 hunks)
  • middleware/logger/logger_test.go (1 hunks)
  • middleware/proxy/proxy_test.go (34 hunks)
Additional comments: 11
listen_test.go (4)
  • 49-49: Introduced a channel errs to capture errors in the goroutine. This is a good practice for handling errors in asynchronous operations, ensuring that errors are not lost and can be asserted in the main test goroutine.
  • 52-55: Increased the timeout duration from 250 milliseconds to 5 seconds in the context creation for graceful shutdown. This change likely aims to reduce flakiness in tests by allowing more time for operations to complete, especially in environments with variable performance. However, ensure that this does not significantly increase the overall test suite execution time.
  • 72-73: Modified the test cases to reflect the updated timeout values. This is a necessary change following the increase in the context timeout to ensure consistency in test expectations. It's good to see that the test cases are updated to align with the new configuration.
  • 89-91: Removed the assertion for error in the goroutine and added it after receiving the error from the channel. This change improves the reliability of error handling in tests by ensuring that errors are properly captured and asserted in the main test goroutine. It's a good practice to centralize error assertions after asynchronous operations.
middleware/proxy/proxy_test.go (3)
  • 43-43: The integration of the startServer function within various test functions is consistent and improves code organization and readability by centralizing server startup logic.

Also applies to: 131-131, 160-160, 214-214, 536-536

  • 19-30: The changes in this PR, including the introduction of the startServer function and the removal of t.Parallel() and require.NoError() usage (as inferred from the PR objectives and the absence of these in the provided code), align with the goals of improving test reliability and code consistency. These adjustments contribute to making the tests more deterministic and easier to debug.
  • 24-26: The use of panic in the startServer function to handle errors is a clear way to indicate server startup failures in tests. However, ensure that this approach is consistent with the overall testing strategy of the project. Consider whether alternative error handling mechanisms might be more appropriate in some contexts to facilitate debugging.
client_test.go (2)
  • 46-53: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [40-57]

The modification in Test_Client_Invalid_URL to include the startServer function before making requests is a good practice to ensure the server is ready. This change aligns with the PR objectives to improve test reliability.

  • 46-53: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [40-57]

While the provided code does not explicitly show the removal of t.Parallel() and require.NoError(), it's important to ensure that their removal across the test suite aligns with the PR objectives to prevent race conditions and improve test reliability. Removing t.Parallel() helps avoid tests affecting each other by running concurrently, and replacing require.NoError() with a more controlled error handling mechanism can prevent abrupt test terminations that might mask underlying issues.

Ensure that the removal of t.Parallel() and require.NoError() is consistently applied across the test suite and that it does not introduce any unintended side effects.

app_test.go (2)
  • 824-830: The changes in Test_App_ShutdownWithTimeout introduce error handling for app.Listener(ln) and use panic in case of an error, which aligns with the PR objectives to improve error handling in tests.
  • 874-877: The changes in Test_App_ShutdownWithContext introduce error handling for app.Listener(ln) and use panic in case of an error, which aligns with the PR objectives to improve error handling in tests.

client_test.go Outdated Show resolved Hide resolved
client_test.go Outdated Show resolved Hide resolved
middleware/proxy/proxy_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between e3db3b5 and e84e5ef.
Files selected for processing (3)
  • client_test.go (69 hunks)
  • listen_test.go (4 hunks)
  • middleware/proxy/proxy_test.go (34 hunks)
Files skipped from review as they are similar to previous changes (3)
  • client_test.go
  • listen_test.go
  • middleware/proxy/proxy_test.go

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between e84e5ef and 6783b99.
Files selected for processing (1)
  • listen_test.go (4 hunks)
Additional comments: 5
listen_test.go (5)
  • 50-50: Introduced a channel errs to capture errors from the goroutine. This is a good practice for error handling in asynchronous operations, ensuring that errors are not lost and can be asserted in the main test goroutine.
  • 53-53: Increased the timeout duration from 250 milliseconds to 2 seconds in the context creation. This change likely aims to reduce flakiness in tests by allowing more time for operations to complete, especially in environments with variable performance. However, it's important to balance this with the overall test suite execution time.
  • 67-79: Implemented a server readiness check before proceeding with the test cases. This is a significant improvement, ensuring that the server is ready to handle requests before any tests are executed. It reduces the likelihood of flaky tests due to timing issues with server startup.
  • 88-88: Updated the test cases to reflect the new timeout values. This change is consistent with the increased timeout duration set earlier in the context. It's crucial that test cases are aligned with the expected behavior and configurations of the system under test.
  • 128-130: Removed the assertion for error in the goroutine and added it after receiving the error from the channel. This change correctly moves the error assertion to the main test goroutine, where it can be properly handled and reported by the testing framework. It's a good practice to centralize error handling in asynchronous test setups.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 6783b99 and 5596887.
Files selected for processing (1)
  • internal/storage/memory/memory_test.go (1 hunks)
Additional comments: 1
internal/storage/memory/memory_test.go (1)
  • 86-86: The change from "john" to "john-delete" in the Test_Storage_Memory_Delete function is a minor but positive change for clarity and specificity in test cases. This helps in distinguishing the delete operation's test case from others that use a similar key. However, it's important to ensure that this key change does not affect other tests or the overall test suite's logic.

internal/storage/memory/memory_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 5596887 and 6f05d4b.
Files selected for processing (6)
  • client.go (2 hunks)
  • listen.go (2 hunks)
  • middleware/logger/default_logger.go (1 hunks)
  • middleware/logger/template_chain.go (1 hunks)
  • middleware/proxy/proxy.go (1 hunks)
  • middleware/proxy/proxy_test.go (9 hunks)
Files skipped from review as they are similar to previous changes (1)
  • middleware/proxy/proxy_test.go
Additional comments: 10
middleware/logger/template_chain.go (4)
  • 20-22: Adding a nil check for templateB before proceeding with the function logic is a good practice to ensure that the function does not operate on invalid data. This enhances the robustness of the code by preventing potential nil pointer dereferences.
  • 25-27: The nil check for startTagB is correctly implemented. It ensures that the function does not proceed with a nil startTagB, which could lead to runtime errors. This is a good error handling practice.
  • 30-32: Similar to the previous checks, adding a nil check for endTagB is a prudent measure. It prevents the function from executing further and potentially encountering runtime errors due to a nil endTagB.
  • 35-37: The nil check for paramSeparatorB follows the same pattern as the previous checks and is a good practice. It ensures the function's robustness by preventing operations on a nil paramSeparatorB.
middleware/logger/default_logger.go (2)
  • 159-162: Adding a check for a nil io.Writer at the beginning of the writeLog function is a crucial safety measure. It prevents the function from attempting to write to a nil writer, which would result in a runtime panic. This change enhances the reliability of the logging mechanism.
  • 164-167: The updated error handling logic in the writeLog function, which attempts to write the error message to the original writer (w) and falls back to os.Stderr if that fails, is a thoughtful improvement. It ensures that error messages are not lost silently and are directed to a fallback output if necessary. This change improves the observability and debuggability of the application.
middleware/proxy/proxy.go (1)
  • 203-205: Introducing a nil check for the uri parameter in the getScheme function is a good defensive programming practice. It prevents the function from proceeding with a nil uri, which could lead to runtime errors. This change enhances the function's robustness and reliability.
listen.go (2)
  • 271-274: Adding error checking before using the listener in the createListener function is a critical improvement. It ensures that the function does not proceed with an invalid listener, which could lead to runtime errors. This change enhances the reliability of the listener creation process.
  • 382-382: Modifying the initialization of pidSlice to use make with an initial capacity of 0 is a subtle but good practice. It explicitly initializes pidSlice as an empty slice, which is more idiomatic and clear than relying on a nil slice. This change improves code readability and maintainability.
client.go (1)
  • 607-624: The addition of nil checks for a.formFiles, ff, and ff.Content before proceeding with operations on these objects is a good practice to avoid nil dereference issues. This change enhances the robustness of the code by ensuring that operations are only performed on non-nil objects, thus preventing potential runtime panics. This aligns well with the PR's objective of enhancing code reliability.

client.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 6f05d4b and b337ee1.
Files selected for processing (1)
  • client_test.go (22 hunks)
Files skipped from review as they are similar to previous changes (1)
  • client_test.go

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 1

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between b337ee1 and 53b0530.
Files selected for processing (3)
  • internal/memory/memory_test.go (1 hunks)
  • middleware/logger/logger_test.go (2 hunks)
  • middleware/proxy/proxy_test.go (10 hunks)
Files skipped from review as they are similar to previous changes (1)
  • middleware/proxy/proxy_test.go
Additional comments: 5
internal/memory/memory_test.go (5)
  • 20-20: Adding a comment here improves the readability and understanding of the test's purpose, which is a good practice.
  • 27-27: Switching from require.Equal to require.Nil for checking nil values is appropriate when validating that a key does not exist in the store. This change makes the intention of the test clearer and is a good improvement.
  • 46-47: Adding new test cases for setting keys with values and resetting all keys is a positive change, as it increases the coverage and robustness of the tests. It's good to see that different scenarios are being considered.
  • 51-54: The adjustment of sleep time for the expiration test is not directly visible in the provided code snippet. However, ensuring that the sleep time accurately reflects the expiration time of keys is crucial for the reliability of this test. If the sleep time has been adjusted elsewhere in the code, it's important to verify that it's sufficient for the expiration to take effect.
Verification successful

The sleep time of 1100 milliseconds in the expiration test is appropriate, as it exceeds the expiration time of 1 second (1000 milliseconds) set for the keys. This should allow sufficient time for the keys to expire before they are accessed again, indicating that the sleep time adjustment is correctly implemented for the purpose of the test.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Verify that the adjusted sleep time is appropriate for the expiration test.
grep -C3 "time.Sleep" internal/memory/memory_test.go

Length of output: 222



Script:

#!/bin/bash
# Find the expiration time set for keys in the test to compare with the sleep time.
grep -B2 "store.Set" internal/memory/memory_test.go

Length of output: 317



Script:

#!/bin/bash
# Find the value assigned to `exp` that represents the expiration time for the keys in the test.
grep -B5 "exp" internal/memory/memory_test.go

Length of output: 466

* 46-47: Modifying key names for clarity is a good practice, especially in a test environment where readability and understandability are crucial. Using descriptive key names like "john-reset" and "doe-reset" helps to clearly distinguish between different test scenarios.

internal/memory/memory_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 53b0530 and f1969cc.
Files selected for processing (1)
  • internal/memory/memory_test.go (1 hunks)
Files skipped from review as they are similar to previous changes (1)
  • internal/memory/memory_test.go

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between f1969cc and d7fce2d.
Files selected for processing (3)
  • client_test.go (22 hunks)
  • middleware/favicon/favicon.go (2 hunks)
  • middleware/proxy/proxy_test.go (10 hunks)
Files skipped from review as they are similar to previous changes (2)
  • client_test.go
  • middleware/proxy/proxy_test.go
Additional comments: 1
middleware/favicon/favicon.go (1)
  • 139-143: Serving the cached favicon with appropriate headers is correctly implemented. Setting the Content-Length, Content-Type, and Cache-Control headers is essential for efficient and correct delivery of static assets like favicons. This implementation ensures that browsers can cache the favicon, reducing unnecessary requests. No issues or improvements are identified in this segment.

middleware/favicon/favicon.go Show resolved Hide resolved
middleware/favicon/favicon.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between d7fce2d and c9a033c.
Files selected for processing (2)
  • middleware/favicon/favicon.go (2 hunks)
  • middleware/proxy/proxy_test.go (10 hunks)
Files skipped from review as they are similar to previous changes (2)
  • middleware/favicon/favicon.go
  • middleware/proxy/proxy_test.go

@gaby gaby changed the title 🧹 v3 (Maintenance): Updates to Client and Proxy Unit-Tests 🧹 v3 (Maintenance): Fix flaky unit-tests Mar 4, 2024
@ReneWerner87
Copy link
Member

image
image

@ReneWerner87
Copy link
Member

lets solve
image
in #2897

@ReneWerner87
Copy link
Member

image

@ReneWerner87
Copy link
Member

image

@ReneWerner87
Copy link
Member

image

@ReneWerner87
Copy link
Member

image

@ReneWerner87
Copy link
Member

image

@ReneWerner87
Copy link
Member

image

@ReneWerner87
Copy link
Member

image

@ReneWerner87
Copy link
Member

Screenshot_2024-03-08-17-59-19-48_40deb401b9ffe8e1df2f1cc5ba480b12

@ReneWerner87 ReneWerner87 merged commit 0379cc5 into main Mar 8, 2024
18 of 19 checks passed
@ReneWerner87 ReneWerner87 added this to the v3 milestone Mar 8, 2024
@gaby gaby deleted the flaky-tests branch March 11, 2024 02:37
grivera64 pushed a commit to grivera64/fiber that referenced this pull request Mar 16, 2024
* Fixes for some of the failing tests

* Add readiness check to serverStart()

* Use net/http client for tests listen test

* Use different key for this test

* Run Proxy Middleware tests in parallel. Add nil checks for potential issues pointed by nilaway

* Enable parallel client tests

* Do not run timing sensitive tests in parallel

* Remove TODO

* Revert Test_Proxy_DoTimeout_Timeout, and remove t.Parallel() for it

* Do not calculate favicon len on each handler call

* Revert logic change

* Increase timeout of SaveFile tests

* Do not run time sensitive tests in parallel

* The Agent can't be run in parallel

* Do not run time sensitive tests in parallel

* Fixes based on uber/nilaway

* Revert change to Client test

* Run parallel

* Update client_test.go

* Update client_test.go

* Update cache_test.go

* Update cookiejar_test.go

* Remove parallel for test using timeouts

* Remove t.Parallel() from logger middleware tests

* Do not use testify.require in a goroutine

* Fix import, and update golangci-lint

* Remove changes to template_chain.go

* Run more tests in parallel

* Add more parallel tests

* Add more parallel tests

* SetLogger can't run in parallel

* Run more tests in parallel, fix issue with goroutine in limiter middleware

* Update internal/storage/memory, add more benchmarks

* Increase sleep for csrf test by 100 milliseconds. Implement asserted and parallel benchmarks for Session middleware

* Add 100 milliseconds to sleep during test

* Revert name change

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

* fix: Inconsistent and flaky unit-tests

---------

Co-authored-by: M. Efe Çetin <[email protected]>
Co-authored-by: René <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

🐛 [Bug]: Enable go-require for testify-lint
4 participants