OOM occurred when validating the response.
/What's-different-in-2.0#error-handling | com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
at io.reactivex.plugins.RxJavaPlugins.onError(RxJavaPlugins.java:367) ~[rxjava-2.2.21.jar:?]
at io.reactivex.internal.observers.ConsumerSingleObserver.onSuccess(ConsumerSingleObserver.java:65) ~[rxjava-2.2.21.jar:?]
at io.vertx.reactivex.impl.AsyncResultSingle.lambda$subscribeActual$0(AsyncResultSingle.java:49) ~[vertx-rx-java2-gen-4.3.1.jar:4.3.1]
at io.vertx.reactivex.ext.web.client.HttpRequest$10.handle(HttpRequest.java:770) ~[vertx-rx-java2-4.3.1.jar:4.3.1]
at io.vertx.reactivex.ext.web.client.HttpRequest$10.handle(HttpRequest.java:767) ~[vertx-rx-java2-4.3.1.jar:4.3.1]
at io.vertx.ext.web.client.impl.HttpContext.handleDispatchResponse(HttpContext.java:390) ~[vertx-web-client-4.3.1.jar:4.3.1]
at io.vertx.ext.web.client.impl.HttpContext.execute(HttpContext.java:377) ~[vertx-web-client-4.3.1.jar:4.3.1]
at io.vertx.ext.web.client.impl.HttpContext.next(HttpContext.java:355) ~[vertx-web-client-4.3.1.jar:4.3.1]
at io.vertx.ext.web.client.impl.HttpContext.fire(HttpContext.java:322) ~[vertx-web-client-4.3.1.jar:4.3.1]
at io.vertx.ext.web.client.impl.HttpContext.dispatchResponse(HttpContext.java:284) ~[vertx-web-client-4.3.1.jar:4.3.1]
at io.vertx.ext.web.client.impl.HttpContext.lambda$null$7(HttpContext.java:500) ~[vertx-web-client-4.3.1.jar:4.3.1]
at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:101) ~[vertx-core-4.3.1.jar:4.3.1]
at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:64) ~[vertx-core-4.3.1.jar:4.3.1]
at io.vertx.core.impl.EventLoopContext.lambda$runOnContext$0(EventLoopContext.java:44) ~[vertx-core-4.3.1.jar:4.3.1]
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) [netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) [netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) [netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [netty-transport-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) [netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.77.Final.jar:4.1.77.Final]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2049) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator.resolveJsonSchema(SchemaValidator.java:196) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator.validate(SchemaValidator.java:158) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.interaction.response.ResponseValidator.validateResponseBody(ResponseValidator.java:158) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.interaction.response.ResponseValidator.validateResponse(ResponseValidator.java:95) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.OpenApiInteractionValidator.lambda$validateResponse$4(OpenApiInteractionValidator.java:234) ~[swagger-request-validator-core-2.35.1.jar:?]
at java.util.function.Function.lambda$andThen$1(Function.java:88) ~[?:?]
at com.atlassian.oai.validator.OpenApiInteractionValidator.validateOnApiOperation(OpenApiInteractionValidator.java:265) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.OpenApiInteractionValidator.validateResponse(OpenApiInteractionValidator.java:231) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.emc.cyclone.contexts.infrastructure.rest.swagger.SwaggerUtils.validateResponse(SwaggerUtils.java:190) ~[cyclone-controlpath-1.0-SNAPSHOT.jar:?]
at com.emc.cyclone.contexts.infrastructure.rest.requestvalidation.RestResponseValidator.validate(RestResponseValidator.java:275) ~[cyclone-controlpath-1.0-SNAPSHOT.jar:?]
at com.emc.cyclone.contexts.infrastructure.rest.requestvalidation.RestResponseValidator.validateResponse(RestResponseValidator.java:181) ~[cyclone-controlpath-1.0-SNAPSHOT.jar:?]
at com.emc.cyclone.contexts.infrastructure.rest.requestvalidation.ExtendedRequestResponseValidator.validateResponse(ExtendedRequestResponseValidator.java:46) ~[cyclone-controlpath-1.0-SNAPSHOT.jar:?]
at com.emc.bedrock.external.rest.PostgRestHttpModule.lambda$handlePostgrestResponse$35(PostgRestHttpModule.java:589) ~[core-1.0-SNAPSHOT.jar:?]
at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
at com.emc.bedrock.external.rest.PostgRestHttpModule.handlePostgrestResponse(PostgRestHttpModule.java:589) ~[core-1.0-SNAPSHOT.jar:?]
at com.emc.bedrock.external.rest.PostgRestHttpModule.lambda$getRouteHandler$32(PostgRestHttpModule.java:515) ~[core-1.0-SNAPSHOT.jar:?]
at io.reactivex.internal.observers.ConsumerSingleObserver.onSuccess(ConsumerSingleObserver.java:62) ~[rxjava-2.2.21.jar:?]
... 20 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:56) ~[jackson-databind-2.14.2.jar:2.14.2]
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:21) ~[jackson-databind-2.14.2.jar:2.14.2]
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:59) ~[jackson-databind-2.14.2.jar:2.14.2]
at com.fasterxml.jackson.databind.node.ObjectNode.deepCopy(ObjectNode.java:21) ~[jackson-databind-2.14.2.jar:2.14.2]
at com.atlassian.oai.validator.schema.transform.SchemaDefinitionsInjectionTransformer.apply(SchemaDefinitionsInjectionTransformer.java:23) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator.lambda$readAndTransformSchemaObject$6(SchemaValidator.java:219) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator$$Lambda$13223/0x0000000102de5c40.accept(Unknown Source) ~[?:?]
at java.util.Arrays$ArrayList.forEach(Arrays.java:4390) ~[?:?]
at com.atlassian.oai.validator.schema.SchemaValidator.readAndTransformSchemaObject(SchemaValidator.java:219) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator.access$300(SchemaValidator.java:52) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator$1.load(SchemaValidator.java:111) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator$1.load(SchemaValidator.java:108) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator.resolveJsonSchema(SchemaValidator.java:196) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.schema.SchemaValidator.validate(SchemaValidator.java:158) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.interaction.response.ResponseValidator.validateResponseBody(ResponseValidator.java:158) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.interaction.response.ResponseValidator.validateResponse(ResponseValidator.java:95) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.OpenApiInteractionValidator.lambda$validateResponse$4(OpenApiInteractionValidator.java:234) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.OpenApiInteractionValidator$$Lambda$13213/0x0000000102de3440.apply(Unknown Source) ~[?:?]
at java.util.function.Function.lambda$andThen$1(Function.java:88) ~[?:?]
at java.util.function.Function$$Lambda$244/0x000000010038e440.apply(Unknown Source) ~[?:?]
at com.atlassian.oai.validator.OpenApiInteractionValidator.validateOnApiOperation(OpenApiInteractionValidator.java:265) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.atlassian.oai.validator.OpenApiInteractionValidator.validateResponse(OpenApiInteractionValidator.java:231) ~[swagger-request-validator-core-2.35.1.jar:?]
at com.emc.cyclone.contexts.infrastructure.rest.swagger.SwaggerUtils.validateResponse(SwaggerUtils.java:190) ~[cyclone-controlpath-1.0-SNAPSHOT.jar:?]
at com.emc.cyclone.contexts.infrastructure.rest.requestvalidation.RestResponseValidator.validate(RestResponseValidator.java:275) ~[cyclone-controlpath-1.0-SNAPSHOT.jar:?]
at com.emc.cyclone.contexts.infrastructure.rest.requestvalidation.RestResponseValidator.validateResponse(RestResponseValidator.java:181) ~[cyclone-controlpath-1.0-SNAPSHOT.jar:?]
Comments (5)
-
reporter -
reporter - marked as blocker
- edited description
-
reporter Can anyone help check this issue?
-
reporter Our upgrading process is blocked by this issue. Please help check it.
-
- changed status to resolved
Available in 2.38.0
Fix is to support configuring the allowed schema cache size if needed.
- Log in to comment
We are trying to upgrade swagger-request-validator from 1.5.0 to latest version 2.35.1. After running some test, got OOM.
Here are the JVM configuration:
-Xms512m
-Xmx750m
The openapi definition file has more than 4.1M, it’s a large file.
=============================================================
After analyzed the heap dump, found the heap memory was occupied by the jsonSchemaCache in SchemaValidator object and the cache in CachingProcessor object.
This two caches store many copies from the openapi definition schema, each one has more than 4.7M.
The maximum size of the jsonSchemaCache in SchemaValidator object is 100 and it seems it can’t be configured.
The maximum size of the cache in CachingProcessor object is 512 and it can be configured, but we just used the default value 512.
==============================================================
From above analysis, the 750m heap size is obviously not enough.
But I have some questions, why do we need to do deepCopy when trying to get openapi schema for every validation request.
And in SchemaDefinitionsInjectionTransformer it did deepCopy again
Can we avoid deep copying for every validation request? Can we just use a single copy to validate all the validation requests?