-
Notifications
You must be signed in to change notification settings - Fork 625
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nested "before" lifecycle hook #2792
Comments
So if you could put beforeContainer inside a container and it only execute
for THAT container (and not child containers), that's what you'd expect ?
…On Fri, 21 Jan 2022 at 18:35, Bryan Donovan ***@***.***> wrote:
I'm trying to use Kotest to write nested tests that have hooks that only
run once per nest level. This is how mocha works in JavaScript and how
RSpec works in ruby. In mocha the function is called before. I tried
using beforeContainer but it runs once for *each* nested container
instead of just once for the entire container including any nested
containers within that parent container. This is hard to explain, so below
is a set of tests that I would expect to pass, if a before method existed
in kotest that behaved like mocha's or RSpec's. In short, each before
method should only be called once, and only when tests in its container run
(e.g., the before method inside "nested level 2" shouldn't run until
after the tests in "nested level 1" have run).
package foo.bar
import io.kotest.core.spec.style.DescribeSpecimport io.kotest.matchers.shouldBe
class WidgetTest : DescribeSpec({
var foo = "initial"
var counterOne = 0
var counterTwo = 0
var counterThree = 0
before {
foo = "bar"
counterOne++
}
it("foo should be bar") {
foo shouldBe "bar"
counterOne shouldBe 1
}
it("beforeSpec should have been called only once") {
foo shouldBe "bar"
counterOne shouldBe 1
}
it("counterTwo should be 0") {
counterTwo shouldBe 0
}
it("counterThree should be 0") {
counterThree shouldBe 0
}
describe("nested level 1") {
before {
foo = "buzz"
counterTwo++
}
it("foo should be buzz") {
foo shouldBe "buzz"
}
it("and counterOne should be 1") {
counterOne shouldBe 1
}
it("and counterTwo should be 1") {
counterTwo shouldBe 1
}
describe("nested level 2") {
before {
foo = "jazz"
counterThree++
}
it("foo should be jazz") {
foo shouldBe "jazz"
}
it("and counterOne should be 1") {
counterOne shouldBe 1
}
it("and counterTwo should be 1") {
counterTwo shouldBe 1
}
it("and counterThree should be 1") {
counterThree shouldBe 1
}
}
}
})
I can get close to what I want if I make the first before a beforeSpec
and the other two beforeContainer (and move them above each nested
container), but then counterTwo gets incremented twice: once for "nested
level 1" and once for "nested level 2". I'm not sure if I'm just doing
something wrong or if this capability simply doesn't exist in kotest.
A somewhat more realistic example:
import io.kotest.core.spec.style.DescribeSpecimport io.kotest.matchers.shouldBeimport io.kotest.matchers.shouldNotBeimport kotlinx.coroutines.delay
class WidgetApiTest : DescribeSpec({
describe("CRUD operations") {
var widget: Widget? = Widget(0, "")
describe("when we create a widget") {
before { // This should run exactly once
createWidget(1, "widget-a")
}
describe("when we fetch the widget") {
before { // This should run exactly once
widget = getWidget(1) // assume this is a slow HTTP or database request
}
it("the widget should not be null") {
widget shouldNotBe null
}
it("and the widget should have a name") {
widget!!.name shouldBe "widget-a"
}
describe("when we update the widget's name") {
before { // the above "before" functions should not have run again in this test container
widget?.let { w ->
w.name = "new-name"
updateWidget(w)
}
}
it("then the re-fetched widget should be updated") {
widget = getWidget(1) // assume this is a slow HTTP or database request
widget!!.name shouldBe "new-name"
}
}
}
}
}
})
data class Widget(val id: Int, var name: String?)
val widgetContainer: HashMap<Int, Widget> = hashMapOf()
suspend fun createWidget(id: Int, name: String?): Widget {
delay(200) // slow HTTP request or Database call
val widget = Widget(id, name)
widgetContainer[id] = widget
return widget
}
suspend fun getWidget(id: Int): Widget? {
delay(200) // slow HTTP request or Database call
return widgetContainer[id]
}
suspend fun updateWidget(widget: Widget): Widget {
delay(200) // slow HTTP request or Database call
val id = widget.id
widgetContainer[id] = widget
return widget
}
—
Reply to this email directly, view it on GitHub
<#2792>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFVSGWNNGTGITE4LAYALX3UXH3UPANCNFSM5MQ5TBRA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Yes, exactly! I guess that was an easier way of saying it :) |
We could add this and call it beforeScope to make it clear it only runs
before that particular scope.
…On Fri, 21 Jan 2022 at 18:41, Bryan Donovan ***@***.***> wrote:
Yes, exactly! I guess that was an easier way of saying it :)
—
Reply to this email directly, view it on GitHub
<#2792 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFVSGWV6WCSGGTN5L7VLOLUXH4J7ANCNFSM5MQ5TBRA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you commented.Message ID:
***@***.***>
|
e.g,. this javascript suite passes using const assert = require('assert');
describe("Nesting", () => {
var foo = "initial";
var counterOne = 0;
var counterTwo = 0;
var counterThree = 0;
before(() => {
foo = "bar";
counterOne++;
});
it("foo should be bar", () => {
assert.equal(foo, "bar");
assert.equal(counterOne, 1);
});
it("before() should have been called only once", () => {
assert.equal(foo, "bar");
assert.equal(counterOne, 1);
});
it("counterTwo should be 0", () => {
assert.equal(counterTwo, 0);
});
it("counterThree should be 0", () => {
assert.equal(counterThree, 0);
});
describe("nested level 1", () => {
before(() => {
foo = "buzz";
counterTwo++;
});
it("foo should be buzz", () => {
assert.equal(foo, "buzz");
});
it("and counterOne should be 1", () => {
assert.equal(counterOne, 1);
});
it("and counterTwo should be 1", () => {
assert.equal(counterTwo, 1);
});
describe("nested level 2", () => {
before(() => {
foo = "jazz";
counterThree++;
});
it("foo should be jazz", () => {
assert.equal(foo, "jazz");
});
it("and counterOne should be 1", () => {
assert.equal(counterOne, 1);
});
it("and counterTwo should be 1", () => {
assert.equal(counterTwo, 1);
});
it("and counterThree should be 1", () => {
assert.equal(counterThree, 1);
});
});
});
}); |
That would be awesome. Thanks! |
We can put this in 5.2, which is due in a month or so. |
I'd suggest to think carefully about choosing a name for this particular callback. We already have some ambiguity with callback names for historic reasons. We currently have: Historically:
Newish:
Instead of introducing yet another concept with |
I was thinking the same thing, @Reevn. Maybe even |
I'm new to Kotest and Kotlin, so please forgive my naivety, but I honestly don't get why there aren't just two "before" callbacks: a |
It's a combination of trying to make a really flexible test framework and historical reasons. I've had some of the same questions in the past and have been made aware that people are writing tests in pretty different styles. Some people like to put test code inside a "container", but without wrapping it in a "leaf test". For those people for example it is important that something like I personally did not expect that and needed the classical Your issue adds another use-case to the mix, so probably another callback will be born 🙂. Offering this kind of flexibility is the reason why there are so many different callbacks and styles of writing tests. |
Thanks for detailed response, @Reevn! |
beforeTest is a confusing name but as @Reevn says, history. I'd love it if beforeTest was only leaf, but alas. beforeScope is a good name I think because the lambdas are called scopes in kotest - container scope, test scope. But I agree that we need to tread carefully. Another option is simply overloading beforeContainer to have a paramaeter
|
I'd personally like to see this new callback be called from within the container. Main reason is I like to set up empty variables inside the container and then assign values to them inside the |
Yeah it's just a matter of choosing a good name. beforeScope / setup / teardown / etc, Something like beforeThisContainerOnly is descriptive but ugly. |
I think that |
What about just |
I think |
With the current names I think just |
Good point @Reevn |
Good point. I go back to my previous suggestion of beforeScope then
…On Tue, Mar 29, 2022, 9:36 AM Bryan Donovan ***@***.***> wrote:
Good point @Reevn <https://github.com/Reevn>
—
Reply to this email directly, view it on GitHub
<#2792 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFVSGU7YUGZ4WAKWPLWIILVCMWP5ANCNFSM5MQ5TBRA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
What if we used |
@Kantis I don't think that restricting the function to the I'll +1 @sksamuel's suggestion earlier in the thread: beforeContainer(inherit = false) { } It already uses the function that is responsible for setting up a container scope, so no new concept or naming introduced, while making it configurable via a parameter to not inherit this callback to children containers. Looks like a pretty good addition to existing API without adding a lot more complexity to the mix (from the public API perspective). |
Following the thread and seeing all the suggestions I also think beforeContainer(inherit = false) { } is pretty much on point. It is clear from reading it what will happen. Also as @Reevn mentioned the concept of |
My issue with using However, I just realized that the following works. Basically removing the EDIT: The below works as a "before" hook, but there's no clean equivalent "after" hook. package foo.bar
import io.kotest.core.spec.style.DescribeSpec
import io.kotest.matchers.shouldBe
class WidgetTest : DescribeSpec({
var foo = "initial"
var counterOne = 0
var counterTwo = 0
var counterThree = 0
foo = "bar"
counterOne++
it("foo should be bar") {
foo shouldBe "bar"
counterOne shouldBe 1
}
it("beforeSpec should have been called only once") {
foo shouldBe "bar"
counterOne shouldBe 1
}
it("counterTwo should be 0") {
counterTwo shouldBe 0
}
it("counterThree should be 0") {
counterThree shouldBe 0
}
describe("nested level 1") {
foo = "buzz"
counterTwo++
it("foo should be buzz") {
foo shouldBe "buzz"
}
it("and counterOne should be 1") {
counterOne shouldBe 1
}
it("and counterTwo should be 1") {
counterTwo shouldBe 1
}
describe("nested level 2") {
foo = "jazz"
counterThree++
it("foo should be jazz") {
foo shouldBe "jazz"
}
it("and counterOne should be 1") {
counterOne shouldBe 1
}
it("and counterTwo should be 1") {
counterTwo shouldBe 1
}
it("and counterThree should be 1") {
counterThree shouldBe 1
}
}
}
}) |
I see. That would mean if you have following code example: class TestSpec : UnitSpec({
beforeContainer {
println("beforeContainer: ${it.displayName}")
}
describe("first container") {
beforeContainer {
println("Another beforeContainer: ${it.displayName}")
}
describe("with nested") {
it("does something") {
}
}
it("does something else") {
}
}
}) it will print:
So by introducing a parameter |
True, but it's also unconventional :) |
Yep, that I agree with. |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
Bump for staleness |
Just to give more ❤️ to the feature, here's my use case: class TestingTest : BehaviorSpec({
beforeSpec {
dbHelper.createTableIfNotExists()
}
beforeContainer {
dbHelper.deleteAll()
}
Given("Feature X with dataset Y in DB") {
// prepare fixtures
// add some data to DB
// do other stuff
When("subject does something") {
subject.doSomething()
Then("foo should be foo") {
"foo" shouldBe "foo"
}
Then("bar should be bar") {
"bar" shouldBe "bar"
}
}
}
}) Now, as discussed, |
Any chance this will be added soon? I'm fine with any name really. |
This feature might go a long ways toward convincing co-workers to use this framework. |
I agree, this would help convince coworkers to adopt kotest. I'm rather new to kotlin, but would be willing to try doing a pull request, but I need a bit of guidance on how to do that. I'm having trouble understanding how all the hooks work. I think kotest would be perfect if it had this feature. |
I think the name before and after without qualifier might be the right terminology. @Kantis thoughts? |
how about class TestingTest : BehaviorSpec({
beforeSpec {
dbHelper.createTableIfNotExists()
}
beforeContainer {
dbHelper.deleteAll()
}
Given("Feature X with dataset Y in DB") {
beforeCurrent { prepareFixtures() }
afterCurrent { teardownDb() }
afterEach { truncateTables() }
When("subject does something") {
subject.doSomething()
Then("foo should be foo") {
"foo" shouldBe "foo"
}
Then("bar should be bar") {
"bar" shouldBe "bar"
}
}
}
}) |
Hmmm I'm not keen on that personally :) |
Alright, go with |
I communicate in grunts. |
Was about to start using |
My workaround for abstract class BeforeEachCapableBehaviorSpec(body: BehaviorSpec.() -> Unit = {}) : BehaviorSpec(body) {
final override suspend fun beforeContainer(testCase: TestCase) {
if (testCase.parent == null) {
beforeEach()
}
}
fun beforeEach(block: () -> Unit) {
beforeContainer {
if (it.parent == null) {
block()
}
}
}
open fun beforeEach() {}
final override suspend fun afterContainer(testCase: TestCase, result: TestResult) {
if (testCase.parent == null) {
afterEach()
}
}
open fun afterEach() {}
fun afterEach(block: () -> Unit) {
afterContainer { (testCase, _) ->
if (testCase.parent == null) {
block()
}
}
}
} Method names are like that to better match JUnit's annotation names. |
I'm trying to use Kotest to write nested tests that have hooks that only run once per nest level. This is how mocha works in JavaScript and how RSpec works in ruby. In mocha the function is called
before
. I tried usingbeforeContainer
but it runs once for each nested container instead of just once for the entire container including any nested containers within that parent container. This is hard to explain, so below is a set of tests that I would expect to pass, if abefore
method existed in kotest that behaved like mocha's or RSpec's. In short, eachbefore
method should only be called once, and only when tests in its container run (e.g., thebefore
method inside"nested level 2"
shouldn't run until after the tests in"nested level 1"
have run).I can get close to what I want if I make the first
before
abeforeSpec
and the other twobeforeContainer
(and move them above each nested container), but thencounterTwo
gets incremented twice: once for"nested level 1"
and once for"nested level 2"
. I'm not sure if I'm just doing something wrong or if this capability simply doesn't exist in kotest.A somewhat more realistic example:
The text was updated successfully, but these errors were encountered: