Request syntax complexity

[Daniel Fett]

When working on my OIDC4IA prototype, I noticed that there is an enormous amount of complexity hidden in the current request syntax. (I would like to hear if other implementers have the same perception!)

Currently, we have different request semantics on almost every level of the request JSON. I feel like some simplification and unification is needed here. On top of that, a lot of the options that the current syntax gives seem useless, for example, filtering by the country of the provider of a utility bill.

See also Issue #1152: We are creating a lot of complexity on the OPs, but at the same time, RPs should not rely on proper filtering at the OP. We might be better off with some simple filters (like trust_framework) and letting the client handle the rest.

To illustrate the complexity, here are my notes for the request semantics on different levels of the JSON:

* (claims on root level)
                            Possible values except for verified_claims:    ("Standard Rules")
                             - null: send claim contents (string/JSON/etc.)
                             - JSON object: send claim contents
                                - essential: essential claim
                                - values: check if one of the values match (each value can be string/JSON/etc.)
                                - value: check if value matches
                                - max_age: if date or timestamp, check max-age of date or timestamp
verified_claims:
                            Unclear:
                             - null: ?
verified_claims/verification:
                            Always required; if omitted: null
                            Possible values:
                             - null: equals { trust_framework: null }
                             - JSON object with values: send only those fields
verified_claims/verification/*:
                            Apply Standard Rules, except for "evidence"
verified_claims/verification/evidence:
                            Possible values:
                             - null: send all available evidence (?)
                             - JSON array (!) with exactly one object.
verified_claims/verification/evidence/[0]/*:
                            Apply Standard Rules, except for "document"
verified_claims/verification/evidence/[0]/document:
                            Possible values:
                             - null: send whole available document
                             - JSON object with values: send only those fields
verified_claims/verification/evidence/[0]/document/*:
                            Apply Standard Rules
verified_claims/claims
                            Always required; if omitted: null
                            Rules to apply:
                             - null: send all claims
                             - JSON with values: send only those fields
verified_claims/claims/*
                            Apply Standard Rules plus:
                             - JSON: purpose 

‌

[Roland Hedberg]

I second Daniel’s concern.

[Vladimir Dzhuvinov]

Coming up with a reliable framework to process the metadata on the OP is not trivial.

So my question is - what would RPs normally need? Is the trust framework sufficient? If so, then requests and their processing can be made much simpler.

[Daniel Fett]

I assume that trust_framework will be the most important filter.

Limitations on the documents may be hard to express ((utility bill AND drivers license) OR (passport)?).

We want to be able to limit the Claims sent for privacy reasons.

---

Strawman alert:

Re syntax, we could also think about using a flat structure that would still be compatible to existing OIDC claim requests but potentially do away with a lot of the liberties that create complexity:

{ 
   "given_name":null,
   "verified_claims.verification.trust_framework":{ 
      "values":[ 
         "some",
         "values"
      ]
   },
   "verified_claims.given_name": null,
}

The value for each key would be the same in this syntax: null or a JSON object with value/values/max_age (if we decide that we need max_age). We would exhaustively define all keys available, e.g., verified_claims.verification.trust_framework would be okay, verified_claims.verification.evidence.provider would not be supported.

‌

[Vladimir Dzhuvinov]

((utility bill AND drivers license) OR (passport)?) - what if we simplify the grammar to make processing easier:

"evidences" : [["utility_bill", "driving_license"], ["passport"]]

I.e. the arrays designating groups of acceptable combinations. Because if we give people the options of the AND,OR,) grammar - things can get crazy

I found the flat structure nice. Even when the claim names are URIs, as recommended by OIDC Core to prevent clashes, this should work.

[Daniel Fett]

From call: → OpenAPI Specs might be helpful (Stuart Low)

[Daniel Fett]

In PR #9, the request syntax was simplified significantly by eliminating defaults (“null” except for leaf values).

If we additionally make these changes:

• max_age only for verification/time
• only one object in evidence

…and build upon the JSON schema to filter requests, then we end up with these much simpler rules:

on all levels:
 Leaf keys must be explicitly listed or they are omitted from the result.

* (claims on root level)
                            Possible values except for verified_claims:    ("Standard Rules")
                             - null: send claim contents (string/JSON/etc.)
                             - JSON object: send claim contents
                                - essential: essential claim
                                - values: check if one of the values match (each value can be string/JSON/etc.)
                                - value: check if value matches
                                                                - purpose: request purpose

verified_claims/verification/*:
                            Apply Standard Rules, except for "time" (see below) and "evidence" (subtree)

verified_claims/verification/time:
                           - max_age: if date or timestamp, check max-age of date or timestamp

verified_claims/verification/evidence/[0]/*:
                            Apply Standard Rules, except for "document" (subtree)

verified_claims/verification/evidence/[0]/document/*:
                            Apply Standard Rules, except for "issuer" (subtree)

verified_claims/claims/*
                            Apply Standard Rules

Cases taken care of by the scheme:

verified_claims: 
                        scheme error if empty or omitted
verified_claims/verification/evidence: 
                        scheme error if not omitted and it does not contain array with at least one object
verified_claims/verification/evidence/[0]/document:
                        scheme error if not omitted and it does not contain an object
verified_claims/verification/evidence/[0]/document/issuer:
                        scheme error if not omitted and it does not contain an object
verified_claims/claims
                        scheme error if empty or omitted

‌

[Takahiko Kawasaki]

If the current request spec needed to be changed and query flexibility & extensibility were desired at the same time, I would choose a script language (e.g. JavaScript, SQL) as the format of the query unless there exists any standard specification that uses JSON to describe a filter. Does anyone know such standard?

[Ivan Kanakarakis]

Lots of options:

1. https://datatracker.ietf.org/doc/rfc6901/
   JavaScript Object Notation (JSON) Pointer: JSON Pointer defines a string syntax for identifying a specific value within a JavaScript Object Notation (JSON) document.
2. http://jmespath.org/
   complete specification and (ABNF) grammar to query (as XPath) and process (as XQuery) JSON data.
3. http://jsoniq.com/
   XQuery extension to work with JSON
4. https://goessner.net/articles/JsonPath/
   this inspired other similar implementations; but I do not think there is any spec for this and the derivatives
5. https://stedolan.github.io/jq/
   popular tool to inspect a json document; it defines a language similar to JsonPath
   see also, https://github.com/stedolan/jq/wiki/For-JSONPath-users
6. https://jsonata.org/ (query and transform JSON)
7. https://github.com/dfilatov/jspath (query JSON)
8. https://github.com/lloyd/JSONSelect (CSS-like selectors for JSON)
9. https://github.com/jiren/JsonQuery (treat JSON data as if they model a database)
10. https://github.com/kriszyp/rql (treat JSON data as if they model a database)

There are probably more out there. If we go towards that direction (using a query language for JSON) I would suggest we focus more on the query part (not the transformation) and use something that has an actual specification.

[Ivan Kanakarakis]

There is another option that you may want to consider; that is GraphQL – https://graphql.org/

While GraphQL does not target JSON (or any other format) per se, it may be useful as a query language (that at the same time allows for structure definition.)