Certification/Conformance testing for OID4VC (starting with testing wallets in 4VP)

[Joseph Heenan]

Gail has asked if I could estimate the work involved in creating conformance/certification tests for SIOP & verifiable presentations, in similar ways to the tests created for OpenID Connect / FAPI / FAPI-CIBA / etc.

Before I do that, it would be great to get some input from the working group.

1. Are there test systems we can test certification tests against? Ideally these would be sandbox-type systems that contain no real user data, and where any required user interactions can be automated (this is so we can run automated testing of the tests. We have an existing system for automating simple web interactions along the ‘enter text into this field’ and ‘press this button’.) An example client that shows exactly what is any requests/responses/redirects would also be helpful.
2. Are there any particular happy-flow or negative scenarios the WG feel are particularly important to test? (For example, for OpenID Connect certification there is a happy flow that requires response_type=code&scope=openid&… to work and return a fully valid id_token, and a negative flow that requires unregistered redirect uris are rejected.)
3. Any guidance on expected certification profiles and optional/mandatory features would be helpful. (For example, OpenID Connect has a ‘Dynamic’ certification profile that requires the OP to publish authorization server metadata and to support dynamic client registration, and a ‘Basic’ certification profile that requires neither.)
4. Once we have the above, it’d be good to get confirmation there’s at least 3 OP implementations that at least come close to meeting the requirements and are willing to test the tests once a beta version is available

I’ve assumed for now that tests for identity providers are more important than tests for relying parties, as that’s generally how other working groups have viewed it, but please say if this assumption is wrong.

For example, this is some rough guidance that the ekyc-ida working group provided (with some input from the certification team): https://docs.google.com/document/d/1SX2_SjcMUQJ6SQEuNrhNjqAqpQjTUqkHl_qCReSv9-Y/edit#heading=h.cuz9mnx958lj

[Torsten Lodderstedt]

re 1) I will reach out to one of our partners that has a web wallet with OIDC4SSI support whether that could be used for testing the test

[Michael Jones]

As I mentioned on last week’s call, in the OpenID Certification program, it's up to the working group to define what to test. Joseph was right to be asking us to do so.

I believe that even a minimal amount of conformance testing could substantially improve the prospects for interoperability. We can start small and add to the set of tests as they make sense.

One of the first decisions to make is whether to start with testing OPs or RPs.

[Torsten Lodderstedt]

I think we should start with OPs (SIOPs) to set a baseline.

[Kristina Yasuda]

few relevant reference links:

• netis / oidc4ssi / openapi / openapi-oidc4ssi — Bitbucket
• Conformance API v1 | EBSI APIs & Libraries (intebsi.xyz)
• Swagger UI (aceblock.com)

‌

[Michael Jones]

Will you be at IIW, Joseph? If so, let’s dedicate some time together to this topic!

[Joseph Heenan]

Unfortunately not - I’ll be at OSW, EIC and identiverse.

[Joseph Heenan]

Here’s the initial document Kristina wrote up on SIOP certification at OSW (May 2022), with some help/hinderance from Torsten & I:

https://docs.google.com/document/d/10ApJGcnygJRqWR-iGpplmqMWdjjylBQanx07Iz18eZo/edit?usp=sharing

[Joseph Heenan]

Latest doc produced when Kristina / Torsten / I met at IETF Yokohama:

‌

https://docs.google.com/document/d/1CVuFaPX_W7tsjfhhHGuOzoD4ettd1IWWhw1przqv-TU/edit

[Mark Haine]

Migrated to GitHub

https://github.com/openid/OpenID4VP