Missing required properties in response are reported on INFO level instead of ERROR

[Anatoliy Balakirev]

I have some required response fields in my openapi spec (message and title in GenericError, id and email in PersonalInfoResponse):

openapi: 3.0.0
info:
  version: 1.0.0
  title: Spec for tests
paths:
  /personal_info:
    post:
      summary: Updates personal details and returns updated data
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/PersonalInfoRequest'
      responses:
        '201':
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/PersonalInfoResponse'
        '404':
          $ref: '#/components/responses/NotFound'
components:
  responses:
    NotFound:
      description: The specified resource was not found
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/GenericError'
  schemas:
    GenericError:
      type: object
      required:
      - message
      - title
      properties:
        message:
          type: string
        title:
          type: string
    PersonalInfoRequest:
      type: object
      required:
      - email
      properties:
        email:
          type: string
        name:
          type: string
    PersonalInfoResponse:
      type: object
      required:
      - email
      - id
      properties:
        email:
          type: string
        id:
          type: string
        name:
          type: string

However, when I try to validate my pact contracts, where I don't have one of those required fields - PactProviderValidator.hasErrors() returns false. Here is an example with no id in the response:

{
  "provider": {
    "name": "request-validator"
  },
  "consumer": {
    "name": "NA"
  },
  "interactions": [
    {
      "description": "",
      "request": {
        "method": "POST",
        "path": "/personal_info",
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "email": "[e-mail]",
          "name": "James Bond"
        },
        "matchingRules": {
          "header": {
            "Content-Type": {
              "matchers": [
                {
                  "match": "regex",
                  "regex": "application/json.*"
                }
              ],
              "combine": "AND"
            }
          }
        }
      },
      "response": {
        "status": 201,
        "headers": {
          "Content-Type": "application/json;charset=UTF-8"
        },
        "body": {
          "name": "James Bond",
          "email": "[e-mail]"
        }
      }
    }
  ],
  "metadata": {
    "pact-specification": {
      "version": "3.0.0"
    },
    "pact-jvm": {
      "version": "3.5.21"
    }
  }
}

After some debugging it turned out to be due to the fact, that those errors are mapped to the validation.response.body.schema.required key, which has default reporting level INFO:

So if I start using swagger-request-validator and don't override that flag (which I don't do by default for sure, as I have no idea I have to) - then by default all missing required response fields are just ignored, which seems to be quite a bad default. So I suggest to use ERROR as a default value for validation.response.body.schema.required key (if that key is actually supposed to be used for this case; not sure, maybe actual problem is that wrong key is picked). Another option is to not have default for that key at all, as it seems to be properly reported as an ERROR by validator (see on the screen above).

Sample project is here: https://github.com/anatoliy-balakirev/request-validator Just run ./gradlew check or manually run SpecificationPactValidatorTest. Following tests are failing even though expected to be ok:

• testNoIdInResponseNok
• testNoEmailInResponseNok
• testNotFoundMissingMessageNok

To make them working - uncomment the single available line in the swagger-validator.properties (which overrides validation.response.body.schema.required to be on ERROR level by default) and rerun tests.