OOME when validating the discriminator
See #269 for the original report.
Seeing OOME from the DiscriminatorKeywordValidator
Reported in 2.25.1
and above.
java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.LinkedHashMap.newNode(LinkedHashMap.java:256)
at java.base/java.util.HashMap.putVal(HashMap.java:627)
at java.base/java.util.HashMap.put(HashMap.java:608)
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:57)
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:19)
....
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:57)
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:19)
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:57)
at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:206)
at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101)
openapi: 3.0.1
info:
title: REST API
version: v1
paths:
"/vehicle":
get:
tags:
- Vehicles
operationId: getVehicle
responses:
'200':
description: Success|OK
content:
"*/*":
schema:
oneOf:
- "$ref": "#/components/schemas/Car"
- "$ref": "#/components/schemas/Plane"
components:
schemas:
Plane:
type: object
allOf:
- "$ref": "#/components/schemas/Vehicle"
- type: object
properties:
manufacturer:
type: string
Car:
type: object
allOf:
- "$ref": "#/components/schemas/Vehicle"
- type: object
properties:
isLuxury:
type: boolean
Vehicle:
required:
- type
type: object
properties:
id:
type: integer
format: int64
name:
type: string
type:
type: string
enum:
- car
- plane
discriminator:
propertyName: type
mapping:
car: "#/components/schemas/Car"
plane: "#/components/schemas/Plane"
Comments (14)
-
reporter -
Account Deactivated Is there a heap dump? The descriptor pasted here is pretty much identical to the test case checked into the lib repository, which doesn’t OOME when run locally. What objects are being allocated extraneously here?
-
reporter I don’t have one. It was reported by @Tamas Toth in
#269and I haven’t reproduced yet.@Tamas Toth are you able to attach a heap dump?
-
Account Deactivated Perhaps they’re running with the unfold-refs option that would make the object tree infinitely deep?
-
Account Deactivated I gave this a bunch of thought and came up with two ways there could be an OOME, hypothetically speaking.
The first is if the schema hierarchy is infinitely recursive, so
deepCopy
allocated infinite memory. I think that would happen if the refs were replaced (withResolveRefs
set totrue
), which will also make validation of the discriminators fail.The second is if there were a very large number of threads simultaneously validating inputs. Each thread allocates its own schema copy for the duration of time it’s doing discriminator validation. But I find it difficult to believe it would be gigabytes of memory for any sane number of threads.
I can’t reproduce the problem, though, so this is just speculation at this point. A heap dump would reveal either of these problems if they were to blame.
-
- attached oome_heap_dump.zip
Heap dump created with 64MB max heap size to keep it small.
-
- attached validator_app.zip
Simple Spring Boot maven project, where running ControllerTest causes OOME. I hope this helps to reproduce the issue.
-
Account Deactivated @Tamas Toth thanks for the succinct example!
Adding
.withResolveRefs(false)
to theOpenApiInteractionValidator
in the test in that project immediately stops it from OOME-ing, confirming my theory#1above. The discriminator validation code won’t work properly with resolveRefs set to true, although I hadn’t realiseddeepCopy
would infinitely recurse while copying referential objects...Note that the test still fails because the controller is returning
Car
that has all theVehicle
properties, and theapi.yaml
is set up in a way that doesn’t ACTUALLY allow those properties (I think the example API is probably wrong). Nonetheless the result is a (correct) validation failure rather than an OOME.@James Navin how do you want to proceed? Should we fail validation if
resolveRefs
istrue
and we run into a discriminator? Should we check that the schema has noDiscriminatorValidators
when we load it ifresolveRefs
istrue
? -
reporter Thanks for the detailed analysis. I’ll try to find some time today to look at it more closely. From memory the reason “resolve refs” was turned on by default in the first place was to make the discriminator validation better behaved. If that’s now causing problems we might be able to turn it off by default. Alternatively (or as well) we might be able to make deepCopy better behaved (or raise a bug upstream).
-
@{557057:aa800350-cb11-4327-add8-aeba15c4499e} thanks for checking, I got the same result after adding
.withResolveRefs(false)
to theOpenApiInteractionValidator
.Could you please point out why
api.yaml
is set up in a way that doesn’t allow those properties? I went over it countless times and compared it to the official examples. I found some differences, but after fixing those I still get the same result and the test fails.I also changed my entire test to use Pet, Cat and Dog with the exact Open API example schema what is closest to my need (https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.1.md#models-with-polymorphism-support) and it still produces the same result.
Based on the above my understanding is that adding
allOf
toCar
with reference toVehicle
schema should enable those properties onCar
(Car extends Vehicle). I tried to understand what is happening under the hood but I still have a lot to explore because this is a new area to me. At this point with the Open API example failing in my test I am really curious what could be the cause of that. -
Account Deactivated @Tamás Tóth I’m sorry I didn’t explain well enough. I’ll try to go into detail about what’s happening in your code to make it clearer.
Controller
returns acom.example.discriminator.Car
with one property explicitly set:"type": "car”
- Buuuuuuut… no special Jackson serialization properties are set on the project - specifically no
spring.jackson.default-property-inclusion: non-null
… so all of the properties of the object are going to be serialized even if they were not explicitly set. - This means the object returned by the controller will be
{"type": "car", "luxury": false, "id": 0, "name": null}
, since theCar
Java class inherited fromVehicle
and got its properties too. - This object has several issues validating against the schema. The first and most obvious is that the
name
does not match"type": "string"
from the API, asnull
is not a string. This can be fixed by callingsetName
on the car, or changing the serialization to exclude the null property, or changing the schema to allow a null. - The second and less obvious problem is that the instance of
Car
has four properties -id
,type
,name
, andluxury
. Notice thatisLuxury
from the code becameluxury
- that’s how Jackson serializes boolean attributes whose accessors begin withis
. But theCar
schema only allows a propertyisLuxury
. Thus, the API doesn’t match the object for this reason too. Changing the API to name the attributeluxury
fixes this. - The third and final problem is the least obvious. The way the schema is written, it doesn’t say “a car must have vehicle properties, and then also have car properties”… It actually says “a car must match the vehicle schema, and then also match this other schema that only allows this one property
luxury
". So even fixing the previous two problems, validation will fail because although the object matches against theVehicle
part of theallOf
, it won’t match theCar
subpart on line 34. This can be fixed by adding anadditionalProperties: true
to that part, or copying the vehicle properties.
I hope this makes clearer why it is that the validator shouldn’t accept that object. An
allOf
is not object inheritance with properties getting copied: it literally means “you must match all of these indepedendent schemas at the same time”. The other problems with the object vs the schema may make it confusing to see what’s going on, but I think I’ve pointed each of them out here. Let me know if there’s something still unclear!
-
Thank you for the detailed explanation. I had a wrong concept of
allOf
in my head, but all the behavior I could not explain now make sense. -
So… is this still open?
I used
.withResolveRefs(false)
and it gave 500 internal server error.The error was due to NPE in
RequestValidator
@Nonnull private Collection<String> getConsumes(final ApiOperation apiOperation) { if (apiOperation.getOperation().getRequestBody() == null) { return emptyList(); } return defaultIfNull(apiOperation.getOperation().getRequestBody().getContent().keySet(), emptySet()); }
In line 6,
apiOperation.getOperation().getRequestBody().getContent()
` was returning null for my open api structure.I cannot paste my actual open api specs, all i can tell we follow a similar structure as described in the issue.
I then removed
.withResolveRefs(false)
, upgraded to latest `2.34.1` version.Got a stack over flow error as below-
Caused by: com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: java.lang.StackOverflowError at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2053) at com.google.common.cache.LocalCache.get(LocalCache.java:3966) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3989) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4950) at com.github.fge.jsonschema.core.processing.CachingProcessor.process(CachingProcessor.java:122) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:109) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128) at com.github.fge.jsonschema.processors.validation.InstanceValidator.processObject(InstanceValidator.java:217) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:150) at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:56) at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:34) at com.github.fge.jsonschema.core.processing.ProcessingResult.of(ProcessingResult.java:79) at com.github.fge.jsonschema.main.JsonSchemaImpl.doValidate(JsonSchemaImpl.java:77) at com.github.fge.jsonschema.main.JsonSchemaImpl.validate(JsonSchemaImpl.java:100) at com.atlassian.oai.validator.schema.SchemaValidator.validate(SchemaValidator.java:160) at com.atlassian.oai.validator.interaction.request.RequestBodyValidator.validateRequestBody(RequestBodyValidator.java:94) at com.atlassian.oai.validator.interaction.request.RequestValidator.validateRequest(RequestValidator.java:113)
I don't know if this is because of combination of non compatible version of
com.github.fge
libs in our code base.For now, we reverted from using discriminator. Any solution in this area is appreciated.
-
This issue is still there in swagger-request-validator-springmvc-2.26.0 onwards till latest version 2.40.0.
It seems that it is happening due to below nested methods call.
Any solution?
[main] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space Caused by: com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:109) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58) at com.github.fge.jsonschema.keyword.validator.draftv4.AllOfValidator.validate(AllOfValidator.java:68) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58) ... ... << few hundreds line of duplicate logs of nested calls >> ... at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58) at com.github.fge.jsonschema.keyword.validator.draftv4.AllOfValidator.validate(AllOfValidator.java:68) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:128) at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:58) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validateAllOfComposition(DiscriminatorKeywordValidator.java:225) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.doValidate(DiscriminatorKeywordValidator.java:101) at com.atlassian.oai.validator.schema.keyword.DiscriminatorKeywordValidator.validate(DiscriminatorKeywordValidator.java:58)
- Log in to comment
FYI @{557057:aa800350-cb11-4327-add8-aeba15c4499e}