OOME when validating the discriminator

[James Navin]

See #269 for the original report.

Seeing OOME from the DiscriminatorKeywordValidator

Reported in 2.25.1 and above.

java.lang.OutOfMemoryError: Java heap space
    at java.base/java.util.LinkedHashMap.newNode(LinkedHashMap.java:256)
    at java.base/java.util.HashMap.putVal(HashMap.java:627)
    at java.base/java.util.HashMap.put(HashMap.java:608)
    at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:57)
    at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:19)
    ....
    at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:57)
    at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:19)
    at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:57)
    at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:206)
    at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101)

openapi: 3.0.1
info:
  title: REST API
  version: v1
paths:
  "/vehicle":
    get:
      tags:
      - Vehicles
      operationId: getVehicle
      responses:
        '200':
          description: Success|OK
          content:
            "*/*":
              schema:
                oneOf:
                - "$ref": "#/components/schemas/Car"
                - "$ref": "#/components/schemas/Plane"
components:
  schemas:
    Plane:
      type: object
      allOf:
      - "$ref": "#/components/schemas/Vehicle"
      - type: object
        properties:
          manufacturer:
            type: string
    Car:
      type: object
      allOf:
      - "$ref": "#/components/schemas/Vehicle"
      - type: object
        properties:
          isLuxury:
            type: boolean
    Vehicle:
      required:
      - type
      type: object
      properties:
        id:
          type: integer
          format: int64
        name:
          type: string
        type:
          type: string
          enum:
          - car
          - plane
      discriminator:
        propertyName: type
        mapping:
          car: "#/components/schemas/Car"
          plane: "#/components/schemas/Plane"

‌

[James Navin]

FYI @{557057:aa800350-cb11-4327-add8-aeba15c4499e}

[Bryan Jacobs]

Is there a heap dump? The descriptor pasted here is pretty much identical to the test case checked into the lib repository, which doesn’t OOME when run locally. What objects are being allocated extraneously here?

[James Navin]

I don’t have one. It was reported by @Tamas Toth in #269 and I haven’t reproduced yet.

@Tamas Toth are you able to attach a heap dump?

‌

[Bryan Jacobs]

Perhaps they’re running with the unfold-refs option that would make the object tree infinitely deep?

[Bryan Jacobs]

I gave this a bunch of thought and came up with two ways there could be an OOME, hypothetically speaking.

The first is if the schema hierarchy is infinitely recursive, so deepCopy allocated infinite memory. I think that would happen if the refs were replaced (withResolveRefs set to true), which will also make validation of the discriminators fail.

The second is if there were a very large number of threads simultaneously validating inputs. Each thread allocates its own schema copy for the duration of time it’s doing discriminator validation. But I find it difficult to believe it would be gigabytes of memory for any sane number of threads.

‌

I can’t reproduce the problem, though, so this is just speculation at this point. A heap dump would reveal either of these problems if they were to blame.

[Tamas Toth]

Heap dump created with 64MB max heap size to keep it small.

[Tamas Toth]

Simple Spring Boot maven project, where running ControllerTest causes OOME. I hope this helps to reproduce the issue.

[Bryan Jacobs]

@Tamas Toth thanks for the succinct example!

Adding .withResolveRefs(false) to the OpenApiInteractionValidator in the test in that project immediately stops it from OOME-ing, confirming my theory #1 above. The discriminator validation code won’t work properly with resolveRefs set to true, although I hadn’t realised deepCopy would infinitely recurse while copying referential objects...

Note that the test still fails because the controller is returning Car that has all the Vehicle properties, and the api.yaml is set up in a way that doesn’t ACTUALLY allow those properties (I think the example API is probably wrong). Nonetheless the result is a (correct) validation failure rather than an OOME.

@James Navin how do you want to proceed? Should we fail validation if resolveRefs is true and we run into a discriminator? Should we check that the schema has no DiscriminatorValidators when we load it if resolveRefs is true?

[James Navin]

Thanks for the detailed analysis. I’ll try to find some time today to look at it more closely. From memory the reason “resolve refs” was turned on by default in the first place was to make the discriminator validation better behaved. If that’s now causing problems we might be able to turn it off by default. Alternatively (or as well) we might be able to make deepCopy better behaved (or raise a bug upstream).

[Tamas Toth]

@{557057:aa800350-cb11-4327-add8-aeba15c4499e} thanks for checking, I got the same result after adding .withResolveRefs(false) to the OpenApiInteractionValidator.

Could you please point out why api.yaml is set up in a way that doesn’t allow those properties? I went over it countless times and compared it to the official examples. I found some differences, but after fixing those I still get the same result and the test fails.

I also changed my entire test to use Pet, Cat and Dog with the exact Open API example schema what is closest to my need (https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.1.md#models-with-polymorphism-support) and it still produces the same result.

Based on the above my understanding is that adding allOf to Car with reference to Vehicle schema should enable those properties on Car (Car extends Vehicle). I tried to understand what is happening under the hood but I still have a lot to explore because this is a new area to me. At this point with the Open API example failing in my test I am really curious what could be the cause of that.

[Bryan Jacobs]

@Tamás Tóth I’m sorry I didn’t explain well enough. I’ll try to go into detail about what’s happening in your code to make it clearer.

1. Controller returns a com.example.discriminator.Car with one property explicitly set: "type": "car”
2. Buuuuuuut… no special Jackson serialization properties are set on the project - specifically no spring.jackson.default-property-inclusion: non-null… so all of the properties of the object are going to be serialized even if they were not explicitly set.
3. This means the object returned by the controller will be {"type": "car", "luxury": false, "id": 0, "name": null}, since the Car Java class inherited from Vehicle and got its properties too.
4. This object has several issues validating against the schema. The first and most obvious is that the name does not match "type": "string" from the API, as null is not a string. This can be fixed by calling setName on the car, or changing the serialization to exclude the null property, or changing the schema to allow a null.
5. The second and less obvious problem is that the instance of Car has four properties - id, type, name, and luxury. Notice that isLuxury from the code became luxury - that’s how Jackson serializes boolean attributes whose accessors begin with is. But the Car schema only allows a property isLuxury. Thus, the API doesn’t match the object for this reason too. Changing the API to name the attribute luxury fixes this.
6. The third and final problem is the least obvious. The way the schema is written, it doesn’t say “a car must have vehicle properties, and then also have car properties”… It actually says “a car must match the vehicle schema, and then also match this other schema that only allows this one property luxury". So even fixing the previous two problems, validation will fail because although the object matches against the Vehicle part of the allOf, it won’t match the Car subpart on line 34. This can be fixed by adding an additionalProperties: true to that part, or copying the vehicle properties.

I hope this makes clearer why it is that the validator shouldn’t accept that object. An allOf is not object inheritance with properties getting copied: it literally means “you must match all of these indepedendent schemas at the same time”. The other problems with the object vs the schema may make it confusing to see what’s going on, but I think I’ve pointed each of them out here. Let me know if there’s something still unclear!

‌

[Tamas Toth]

Thank you for the detailed explanation. I had a wrong concept of allOf in my head, but all the behavior I could not explain now make sense.

[Nischit Shetty]

So… is this still open?

I used .withResolveRefs(false) and it gave 500 internal server error.

The error was due to NPE in RequestValidator

@Nonnull
    private Collection<String> getConsumes(final ApiOperation apiOperation) {
        if (apiOperation.getOperation().getRequestBody() == null) {
            return emptyList();
        }
        return defaultIfNull(apiOperation.getOperation().getRequestBody().getContent().keySet(), emptySet());
    }

In line 6, apiOperation.getOperation().getRequestBody().getContent()` was returning null for my open api structure.

I cannot paste my actual open api specs, all i can tell we follow a similar structure as described in the issue.

I then removed .withResolveRefs(false), upgraded to latest `2.34.1` version.

Got a stack over flow error as below-

Caused by: com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: java.lang.StackOverflowError
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2053)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3966)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3989)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4950)
    at com.github.fge.jsonschema.core.processing.CachingProcessor.process(CachingProcessor.java:122)
    at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:109)
    at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58)
    at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225)
    at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101)
    at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58)
    at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128)
    at com.github.fge.jsonschema.processors.validation.InstanceValidator.processObject(InstanceValidator.java:217)
    at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:150)
    at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:56)
    at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:34)
    at com.github.fge.jsonschema.core.processing.ProcessingResult.of(ProcessingResult.java:79)
    at com.github.fge.jsonschema.main.JsonSchemaImpl.doValidate(JsonSchemaImpl.java:77)
    at com.github.fge.jsonschema.main.JsonSchemaImpl.validate(JsonSchemaImpl.java:100)
    at com.atlassian.oai.validator.schema.SchemaValidator.validate(SchemaValidator.java:160)
    at com.atlassian.oai.validator.interaction.request.RequestBodyValidator.validateRequestBody(RequestBodyValidator.java:94)
    at com.atlassian.oai.validator.interaction.request.RequestValidator.validateRequest(RequestValidator.java:113)

I don't know if this is because of combination of non compatible version of com.github.fge libs in our code base.

For now, we reverted from using discriminator. Any solution in this area is appreciated.

[Abhay.Kumar]

This issue is still there in swagger-request-validator-springmvc-2.26.0 onwards till latest version 2.40.0.

It seems that it is happening due to below nested methods call.

Any solution?

[main] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error
org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
Caused by: com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:109)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58)
        at com.github.fge.jsonschema.keyword.validator.draftv4.AllOfValidator.validate(AllOfValidator.java:68)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58)
...
... << few hundreds line of duplicate logs of nested calls >>
...
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58)
        at com.github.fge.jsonschema.keyword.validator.draftv4.AllOfValidator.validate(AllOfValidator.java:68)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101)
        at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58)

‌