Question: Is support for Android's instrumented unit tests (connectedAndroidTest task) possible?

[erawhctim]

I'm currently using this plugin in an Android project, and it works great when running non-instrumented (JVM-only) unit tests.

I'm interested in getting the same pretty-printing functionality for instrumented unit tests on Android and I'm curious about a few things and wanted to get your opinion before diving in deeper:
(I haven't looked at the code yet, so apologies if any of this is obvious)

1. Is this possible, given the currently architecture of this plugin?
2. Would it make sense to add support for that type of platform-specific gradle task (as an enhancement to this plugin), or do you think it makes more sense as a separate gradle plugin altogether?
3. Do you see any issues around supporting those types of tests? (if you have no Android knowledge or if this is outside your domain, no worries!)

thanks in advance :)

[radarsh]

Hi, thanks for using this plugin. I haven't done any Android development so have a few questions for you.

What is different about the instrumented unit tests that doesn't allow you to use the plugin? If it is due to an error, could you share it?

[erawhctim]

Great question! I believe its due to the fact that these tests are required to run on an actual android device/emulator (since they interact with the Android system classes and require android.jar), so these tests use a custom test runner (AndroidJUnit4Runner). Admittedly my knowledge of how the instrumentation engine that interacts with the device is limited so there may be more to it than that... I think the test runner also outputs test results in a different way than standard JUnit?

As far as I can tell, there are no logs reported when running these tests with your plugin enabled (I would assume the plugin is just not getting run or silently failing?) Are there logs I can check to validate this? Or can I just run the gradle task with --stacktrace or --info to get more info for you?

[erawhctim]

Or, if you want to layout out a high-level overview of the different components of your plugin and how they work, I could try and apply that to how Android instrumentation tests work and try to dig deeper there.

[radarsh]

As far as I can tell, there are no logs reported when running these tests with your plugin enabled (I would assume the plugin is just not getting run or silently failing?) Are there logs I can check to validate this? Or can I just run the gradle task with --stacktrace or --info to get more info for you?

The plugin is quite simple in concept. It attaches a custom TestListener to each test task in your project and logs pretty output using the lifecycle level. I would suggest checking if any logging works at all in your environment and if it does, try and run the task with a --info flag.

[TurpIF]

After some investigations, I saw that the class responsible of logging Android tests status is CustomTestRunListener
It's pretty deep inside Google codebase and the only current way to get access to those logs are by using the fact that the android test gradle task calling this use getLogger() from the DefaultTask of Gradle.

So a possible workaround would be to inject a logger in such task, parse the logs (this seems pretty stable as nothing has change since few years) to get partial information, and delegate to your plugin TestListener.

Another solution could be to ask Google to add a way to inject a TestListener in their android test task. They have internal data class that are similar to the TestDescriptor and TestResult of Gradle: TestIdentifier and TestResult.

[cdavie-artium]

I'd also like to be able to use this plugin with instrumented Android tests, but I'm not sure what work needs to be done to support that.