-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide automatic validation of services (as they are added to the census?) #44
Comments
Yes I think it would be nice to have a tool to automatically test compliance with the spec. I think it is difficult to do so only given the endpoint of the service, but I suspect we could already go a long way with:
With those example names provided, we would already be able to generate quite a few requests to the service:
Just by checking the compliance of the responses with the JSON schemas we have, we should already be able to pick up quite a few issues. Checking the access to the service (HTTPS, GET/POST, CORS headers…) would also be useful indeed. |
Given that the testbench is a web app without a backend, we will be a bit restricted in how we check the service access (CORS, HTTP(S)) since we are bound by the browser restrictions, but I do not think it should be a huge problem (the current test bench works well despite that). |
In the current list of available services, there are entries that do not comply to the spec (e.g. offering GET support for reconciliation only, not POST support). Maybe there could be way to automatically test submissions. Possibly even with results being mirrored in differentiated status/support values in the testbench list.
Maybe an automatic and a manual (with form values) test method?
The text was updated successfully, but these errors were encountered: