[WIP] Step description + Screenshots #142

stefanrenne · 2020-11-09T10:58:58Z

What's in this PR?

This pr's adds a description for every step & adds the ability to create screenshots
As example, this is what the current unit tests generate:

Scenario: Seeing the Face ID permissions prompt
Given I see the biometric login screen
And device biometrics are enabled
When I tap on the log in button
Then I see the do you want to allow “TABTestKit_Example” to use face ID alert

Scenario: Allowing permission but failing authentication
Given I see the do you want to allow “TABTestKit_Example” to use face ID alert
And I tap on the OK button in the alert
When I fail to authenticate biometrics
Then I see the alert

Scenario: Cancelling Face ID
Given I see the alert
When I dismiss the alert
And I do not see the alert
Then I see the login failed alert

Scenario: Dismissing failure alert
Given I see the login failed alert
When I dismiss the login failed alert
Then I see the biometric login screen
And I do not see the table screen

Scenario: Logging in with biometrics
Given I see the biometric login screen
When I complete the biometric login screen
Then I do not see the biometric login screen

Scenario: Relaunching the app
Given I do not see the biometric login screen
When I relaunch the app
Then I go back to the TABTestKit Example app
And I see the biometric login screen

Scenario: Seeing the Face ID permissions prompt
Given I see the biometric login screen
And device biometrics are enabled
When I tap on the log in button
Then I see the do you want to allow “TABTestKit_Example” to use face ID alert

Scenario: Denying permission and seeing an error alert
Given I see the do you want to allow “TABTestKit_Example” to use face ID alert
When I dismiss the do you want to allow “TABTestKit_Example” to use face ID alert
Then I see the login failed alert

Scenario: Confirming the error alert
Given I see the login failed alert
When I dismiss the login failed alert
Then I do not see the login failed alert
And I see the biometric login screen
And I do not see the table screen

Scenario: Completing biometric login and tapping the collection tab bar button
Given I complete the biometric login screen
When I tap on the collection button
Then I see the collection view screen
And the label of the collection header is 'Collection'

Scenario: Refreshing the collection view screen
Given I see the collection view screen
When I refresh the collection view screen
Then the state of the collection cell 0 cell is visible

Scenario: Scrolling until the first cell is hidden
Given the state of the collection cell 0 cell is visible
When I scroll the collection view screen from middle to top until the collection cell 49 cell exists and is visible in the collection view
Then the state of the collection cell 0 cell does not exist

Scenario: Scrolling until the last cell is hidden
Given the state of the collection cell 49 cell is visible
When I scroll the collection view screen from middle to bottom until the collection cell 49 cell does not exist
Then the state of the collection cell 0 cell does not exist
And the state of the collection cell 49 cell does not exist

Scenario: Deep linking from current state
Given I complete the biometric login screen
When I tap on the other button
And I open 'myamazingapp://deeplink'
Then I see the other elements screen

Scenario: Deep linking from terminated state
Given I see the other elements screen
When I terminate the app
And I open 'myamazingapp://deeplink'
And I wait for the app to be running foreground
Then I do not see the other elements screen
And I see the biometric login screen

Scenario: Navigating to the screen with other elements
Given I complete the biometric login screen
And I see the tab bar screen
When I tap on the other button
Then I see the other elements screen
And the label of the other elements header is 'Other elements'

Scenario: Seeing the label and button
Given I see the other elements screen
And I see the example label label
And I see the example button button
When I tap on the example button button
Then I see the activity list view activity sheet

Scenario: Dismissing the share sheet
Given I see the activity list view activity sheet
When I dismiss the activity list view activity sheet
Then I do not see the activity list view activity sheet
And I see the other elements screen

Scenario: Seeing and interacting with the segmented control
Given I see the segmented control
Then I tap on the second button
And the state of the first button is not selected
And the state of the second button is selected
And I tap on the third button
And the state of the second button is not selected
And the state of the third button is selected

Scenario: Seeing and interacting with the text field
Given I see the example text field text field
And the state of the example text field text field has no keyboard focus
Then I tap on the example text field text field
And the state of the example text field text field has keyboard focus
And I type 'Hello world' into the example text field text field
And the value of the example text field text field is 'Hello world'
And the keyboard is regular

Scenario: Number pad keyboard type
Given I scroll the other elements screen downwards until the number pad text field text field is hittable
When I tap on the number pad text field text field
Then the state of the number pad text field text field has keyboard focus
And the keyboard is number keypad

Scenario: Decimal pad keyboard type
Given I scroll the other elements screen downwards until the decimal pad text field text field is hittable
When I tap on the decimal pad text field text field
Then the state of the decimal pad text field text field has keyboard focus
And the keyboard is decimal keypad

Scenario: Email address keyboard type
Given I scroll the other elements screen downwards until the email address text field text field is hittable
When I tap on the email address text field text field
Then the state of the email address text field text field has keyboard focus
And the keyboard is email address

Scenario: Numbers and punctuation keyboard type
Given I scroll the other elements screen downwards until the numbers and punctuation text field text field is hittable
When I tap on the numbers and punctuation text field text field
Then the state of the numbers and punctuation text field text field has keyboard focus
And the keyboard is numbers and punctuation

Scenario: Phone pad keyboard type
Given I scroll the other elements screen downwards until the phone pad text field text field is hittable
When I tap on the phone pad text field text field
Then the state of the phone pad text field text field has keyboard focus
And the keyboard is telephone keypad

Scenario: Twitter keyboard type
Given I scroll the other elements screen downwards until the twitter text field text field is hittable
When I tap on the twitter text field text field
Then the state of the twitter text field text field has keyboard focus
And the keyboard is twitter

Scenario: URL keyboard type
Given I scroll the other elements screen downwards until the uRLText field text field is hittable
When I tap on the uRLText field text field
Then the state of the uRLText field text field has keyboard focus
And the keyboard is url

Scenario: Web search keyboard type
Given I scroll the other elements screen downwards until the web search text field text field is hittable
When I tap on the web search text field text field
Then the state of the web search text field text field has keyboard focus
And the keyboard is web search

Scenario: Seeing and interacting with the secure text field
Given I see the example secure text field secure text field
And the state of the example secure text field secure text field has no keyboard focus
Then I tap on the example secure text field secure text field
And the state of the example secure text field secure text field has keyboard focus
And I type 'Password!' into the example secure text field secure text field
And the value of the example secure text field secure text field is '•••••••••'

Scenario: Seeing and interacting with the slider
Given I tap on the return key
And I see the example slider slider
When I adjust the example slider slider to '1.0'
Then I adjust the example slider slider to '0.0'

Scenario: Seeing and interacting with the switch
Given I see the example switch switch
And the value of the example switch switch is 'on'
Then I adjust the example switch switch to 'off'
And the value of the example switch switch is 'off'

Scenario: Seeing and interacting with the stepper
Given I see the example stepper stepper
And the state of the decrement button is not enabled
And the state of the increment button is enabled
Then I tap on the increment button
And the state of the decrement button is enabled
And the state of the increment button is enabled
And I tap on the increment button
And the state of the decrement button is enabled
And the state of the increment button is not enabled

Scenario: Seeing and interacting with the page indicator
Given I see the example page control page indicator
Then the value of the example page control page indicator is 'page 1 of 3'

Scenario: Seeing the image
Given I scroll the other elements screen downwards until the example picker picker is visible
When I see the example image image
Then the value of the example image image is 'Optional("Ottoman")'

Scenario: Seeing and interacting with the picker
Given I see the example picker picker
And the value of the wheel is 'Hello'
When I adjust the wheel to 'World'
Then the value of the wheel is 'World'

TODO

Request & Process feedback of the used language for the step description
Extend Tests
Update readme
Update docs

Pre-merge checklist

Before merging any PR, please check the following common things that should be done beforehand. These aren't all always required, so just check the box if it doesn't apply.

When adding files, make sure they're added to the right target. If you're adding new files that should be bundled up with Cocoapods etc, they need to be added to the TABTestKit target, not Pods-TABTestKit_Example etc.
Run pod install to ensure that the latest changes are in the Example project. Without this, Carthage might not see the latest changes.
Added and updated tests where possible. This isn't always possible but try wherever you can. The example app contains UI tests to test many of the TABTestKit features.
Updated the CHANGELOG. For any changes pending a release, add to the Pending section. For releases, move everything pending to the release section.
Updated the README. Add info for any new features, update existing info for anything that's changed or needs extra info.

…application state & ElementAttributes

KaneCheshire · 2020-11-09T11:27:48Z

Hello! Thanks for taking the time to do this :) it would be super if you could create issues to discuss changes like this before working on changes because if we decide not to incorporate this then you will have wasted your time

KaneCheshire · 2020-11-09T11:39:23Z

Example/TABTestKit_ExampleUITests/BiometricLogin_AuthenticationFailureTests.swift

@@ -11,7 +11,7 @@ import TABTestKit
 final class BiometricLogin_AuthenticationFailureTests: TABTestCase, SystemPreferencesContext {

 override func preLaunchSetup(_ launch: @escaping () -> Void) {
- resetAllPrivacyPrompts()
+ resetAllPrivacyPrompts().execute()


I'm not really a fan of having to call execute all over the place :/

I get that, the problem is that I need to return a description. Suggestions are welcome.

TABTestKit/Classes/BDD/And.swift

TABTestKit/Classes/Contexts/NavigationContext.swift

TABTestKit/Classes/Extensions/XCTestCase+Screenshot.swift

KaneCheshire · 2020-11-09T11:49:19Z

TABTestKit/Classes/TABTestCase.swift

@@ -81,6 +81,11 @@ open class TABTestCase: XCTestCase, DefaultContexts {
 // You can find this attachment in the .xcresult bundle (usually Derived Data).
 let attachment = createFailureAttachment(description: description, filePath: filePath, lineNumber: lineNumber)
 add(attachment)
+
+ if App.shared.screenshotOption.contains(.onFailure) {


I'm pretty sure XCUI already adds screenshots when a failure happens anyway?

Have to check that, will come back to that

TABTestKit/Classes/Apps/App.swift

TABTestKit/Classes/BDD/Step.swift

KaneCheshire · 2020-11-09T12:00:25Z

TABTestKit/Classes/BDD/Step.swift

@@ -21,83 +16,40 @@ public typealias And = Step
 ///
 /// Do not use this Step type directly, instead use one of the typealiases above, like Given, When, Then or And.


If we do go ahead with these changes, the docs will need updating

I know, that's in the TODO in the description of this pr. Together with extend tests. Because I first wanted to pitch this idea to you guys before finishing this task

KaneCheshire · 2020-11-09T12:55:18Z

TABTestKit/Classes/BDD/Step.swift

-public typealias When = Step
-public typealias Then = Step
-public typealias And = Step
+open class Given: Step {}


Wondering if instead of classes we could make Step a protocol instead, and then have Given/When/Then/And conform to Step? That way you'd still have lightweight structs but you'd be able to tell the difference between Given/When etc

Great addition, the only thing left was storing the current step, so I decided to move that to TABTestCase, so now you can access it like this: TABTestCase.current?.step

Ahh yes of course. What about just TABTestCase.currentStep? It can be a static property on TABTestCase rather than an instance property on TABTestCase, if that makes sense?

fine with me :) Processed feedback

stefanrenne added 6 commits November 5, 2020 20:57

Added TestAction

e49925a

Implemented CustomStringConvertible for screens, elements, keyboard, …

d4b7c2e

…application state & ElementAttributes

Started on step descriptions

f2540c4

Added humanReadableString

5a97357

Refactored steps + added the ability to setup a screenshot behavior

b0a8a59

Fixed support for older swift versions

402ead5

Fixed support for older swift versions

a694668

KaneCheshire reviewed Nov 9, 2020

View reviewed changes

TABTestKit/Classes/BDD/Step.swift Outdated Show resolved Hide resolved

KaneCheshire reviewed Nov 9, 2020

View reviewed changes

stefanrenne added 2 commits November 9, 2020 13:10

Made TABTestCase a singleton

4632722

Moved screenshot option to TABTestCase

79fe63c

KaneCheshire reviewed Nov 9, 2020

View reviewed changes

stefanrenne added 16 commits November 9, 2020 14:09

Fixed support for older swift versions

4e3fcf9

Processed feedback

af67aae

Processed feedback

8c1ad6d

Re-added step keyword

d51ff5e

Exposed element name

3a723be

Extracted number formatter

dda181a

Update order for index/name

2e8d1d8

Fixed indexString

611bbff

Cleanup

86b15ce

Exposed current XCTActivity

19fc7ed

Clear interaction context

5b9f858

Fixed clear

b01e033

Fixed clean

db4c9d2

Updated framework reference

3c741cc

renamed await to support swift 5.5

929ba76

setTextAccessibilitySizeToXXXL

3e85629

Cleanup

8fdfba3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Step description + Screenshots #142

[WIP] Step description + Screenshots #142

stefanrenne commented Nov 9, 2020 •

edited

KaneCheshire commented Nov 9, 2020

KaneCheshire Nov 9, 2020

stefanrenne Nov 9, 2020

KaneCheshire Nov 9, 2020

stefanrenne Nov 9, 2020

KaneCheshire Nov 9, 2020

stefanrenne Nov 9, 2020

KaneCheshire Nov 9, 2020

stefanrenne Nov 9, 2020

KaneCheshire Nov 9, 2020

stefanrenne Nov 9, 2020

		@@ -21,83 +16,40 @@ public typealias And = Step
		///
		/// Do not use this Step type directly, instead use one of the typealiases above, like Given, When, Then or And.

[WIP] Step description + Screenshots #142

Are you sure you want to change the base?

[WIP] Step description + Screenshots #142

Conversation

stefanrenne commented Nov 9, 2020 • edited

What's in this PR?

TODO

Pre-merge checklist

KaneCheshire commented Nov 9, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stefanrenne commented Nov 9, 2020 •

edited