[TIME-169] Unbounded report size means large reports can easily cause OOMEs.

Issue #169 resolved
Andriy Zhdanov repo owner created an issue

The size of reports, and therefore the amount of memory they consume as they are generated, is unbounded. This means that a JIRA instances with large datasets can be taken out trivially by requesting a very large report.

A heap dump generated on JIRA 4.3.3 with TimeSheet plugin 2.1 contained a 180+ million characters long char[] (so 360+MB in memory), representing a report spanning 700 days and around 3000 issues.

This large char[] is was local variable held on to under this stacktrace:

org.apache.velocity.app.VelocityEngine.mergeTemplate(String, String, Context, Writer) com.atlassian.velocity.DefaultVelocityManager.getEncodedBody(String, String, String, String, Context) com.atlassian.velocity.DefaultVelocityManager.getEncodedBody(String, String, String, String, Map) com.atlassian.velocity.DefaultVelocityManager.getEncodedBody(String, String, String, Map) com.atlassian.jira.plugin.JiraResourcedModuleDescriptor.getHtml(String, Map) com.fdu.jira.plugin.report.timesheet.TimeSheet.generateReport(ProjectActionSupport, Map, boolean) com.fdu.jira.plugin.report.timesheet.TimeSheet.generateReportHtml(ProjectActionSupport, Map) com.atlassian.jira.web.action.browser.ConfigureReport.doExecute() webwork.action.ActionSupport.execute() com.atlassian.jira.action.JiraActionSupport.execute() webwork.interceptor.DefaultInterceptorChain.proceed() webwork.interceptor.NestedInterceptorChain.proceed() webwork.interceptor.ChainedInterceptor.intercept(InterceptorChain) webwork.interceptor.DefaultInterceptorChain.proceed() webwork.dispatcher.GenericDispatcher.executeAction(Action) webwork.dispatcher.GenericDispatcher.executeAction() com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.service(HttpServletRequest, HttpServletResponse) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.web.filters.JiraLastFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(ServletRequest, ServletResponse) com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(ServletRequest, ServletResponse) com.atlassian.peace.AbstractPeacePageFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(ServletRequest, ServletResponse) com.atlassian.peace.AbstractPeacePageFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.web.filters.accesslog.AccessLogFilter.executeRequest(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.jira.web.filters.accesslog.AccessLogFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.security.xsrf.XsrfTokenAdditionRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(ContentProcessor, SiteMeshWebAppContext, HttpServletRequest, HttpServletResponse, FilterChain) com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.jira.web.filters.PathExclusionFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.seraph.filter.SecurityFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.seraph.filter.BaseLoginFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.jira.web.filters.JiraLoginFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(ServletRequest, ServletResponse) com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.util.profiling.filters.ProfilingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.jira.web.filters.JIRAProfilingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.web.filters.ActionCleanupDelayFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.web.filters.RequestCleanupFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.johnson.filters.AbstractJohnsonFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.gzipfilter.GzipFilter.doFilterInternal(ServletRequest, ServletResponse, FilterChain) com.atlassian.gzipfilter.GzipFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.core.filters.cache.AbstractCachingFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.jira.web.filters.PathMatchingEncodingFilter.doFilter(HttpServletRequest, HttpServletResponse, FilterChain) com.atlassian.core.filters.AbstractHttpFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.web.monitor.ActiveRequestsFilter$PassToChainFilterFunc.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.jira.web.monitor.ActiveRequestsFilter$DebugLogFilterFunc.doFilter(ServletRequest, ServletResponse, FilterChain) com.atlassian.jira.web.monitor.ActiveRequestsFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.startup.JiraStartupChecklistFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.multitenant.servlet.MultiTenantServletFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) com.atlassian.jira.web.filters.JiraFirstFilter.doFilter(ServletRequest, ServletResponse, FilterChain) java.lang.Thread.run()

Requests for such large reports could be considered unreasonable, but it would be ideal - particularly for hosted environments - if this could be pre-empted to minimise the chances of bringing the whole app down with an OOME.

By rfernandes/Robin Fernandes on Mon, 15 Aug 2011 22:27:05 -0700

Comments (15)

  1. Andriy Zhdanov reporter

    Fixed in upcoming 2.3.5, reports and gadgets restrict time interval for 2 months by default, configurable with jira.timesheet.plugin.maxDays application property.
    Committed revisions 168486 and 168514.

    By azhdanov on Thu, 14 Jun 2012 15:12:19 -0700

  2. Igor Masen

    Hello

    this does not work for me. My jira-config.properties looks like this:

    jira.timesheet.plugin.maxDays = 360
    

    Aftre a restart, the property is listed in the system info as an "Applications Properties", but I have still the message: "The "End Date" must not be later than 2 months."

    Do I something wrong?

  3. Andriy Zhdanov reporter

    Hi Igor,

    Looks so, when I add properties to jira-config.properties they do not appear in Advanced Settings (jira/secure/admin/AdvancedApplicationProperties.jspa), but make effect for plugin. Though I'm not sure what you mean by 'Application Properties'

    I've just double verified this, it's still the case with latest 5.2.10.

    Make sure you update jira-config.properties in jira home, and restart jira. Other than that I can't get any idea what may be wrong.

    Thank you.

  4. Igor Masen

    Hi Andriy,

    thank you for your answer. Our JIRA Version 5.2.10 and the PlugIn-Version is 2.3.16.

    With "Application Properites" I mean the page "secure/admin/ViewSystemInfo.jspa" and the category "Application Properties", there are some properties for the timesheet plugin: Link to screenshot

    Can I pass it as an System-Property like -Djira.timesheet.plugin.maxDay=360?

  5. Andriy Zhdanov reporter

    Ok, you do it right. Then could you please clarify what does not work for you? Exact scenario. Your expectations and actual result.

    Regarding system like property, looks like it does not work.

  6. Igor Masen

    Okay, that's what I am doing. I go to the project-summary and select at the bottom of the page the link called "Time Sheet Report", then I come to the page called "Configure - Timesheet Report". Here I select for "Start date" the first january of this year, showed as "1/Jan/13", next I select today as the "End date", showed as "19/Apr/13". I don't touch the other options and click "Next", after that the site tells me "The "End Date" must not be later than 2 months.".

    Here is the screenshot: Timesheet Error

    I would expect to see the report for more than 2 months. A project leader in our company want's to get the report for one year. If he won't get it, he needs to extract the report every two months and save them all together somewhere.

    Thank's for your help.

  7. Andriy Zhdanov reporter

    Well, everything is correct. And is there "Invalid maxDays number" warning in JIRA log? Note, it will appear (if any) only once after restart.

  8. Igor Masen

    Hello Andriy,

    I just restarted our Instance with the parameter, but there is no warning about maxDays. But I found something else:

    2013-04-22 09:07:47,289 tomcat-http--27 INFO u210691 547x481210x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport!default.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:07:47,291 tomcat-http--27 INFO u210691 547x481210x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport!default.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:07:47,395 tomcat-http--27 INFO u210691 547x481210x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport!default.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:07:47,396 tomcat-http--27 INFO u210691 547x481210x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport!default.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:07:47,457 tomcat-http--27 INFO u210691 547x481210x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport!default.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:07:47,466 tomcat-http--27 INFO u210691 547x481210x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport!default.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:08:04,232 tomcat-http--43 INFO u210691 548x481283x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:08:04,234 tomcat-http--43 INFO u210691 548x481283x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:08:04,348 tomcat-http--43 INFO u210691 548x481283x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:08:04,350 tomcat-http--43 INFO u210691 548x481283x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:08:04,410 tomcat-http--43 INFO u210691 548x481283x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    2013-04-22 09:08:04,419 tomcat-http--43 INFO u210691 548x481283x1 16reauv 10.140.192.149,127.0.0.1 /secure/ConfigureReport.jspa [jira.web.tags.TextTag] An empty i18n key was provided in /secure/views/browser/configurereport.jsp
    

    This is a INFO-Message appears always, when I open the "Time Sheet Report"-Link or press "Next".

    It seems to me like the view has a problem with my input.

  9. Andriy Zhdanov reporter

    Hi Igor,

    Those warning are harmless, they are because of empty strings in drop down boxes used in report configuration, I have just taken care of it for future version though: commit 7eb1036

  10. Igor Masen

    Hello Andriy,

    How shall we continue with this problem? Should I raise a new Bug? If you want, I could try to reproduce this error in an independent JIRA Instance and could give you the Backup-XML, so you could try to debug or something on your own environment. I could also try to setup an VirtualBox VM with it and give you the whole VM.

  11. Andriy Zhdanov reporter

    Hi Igor,

    Yes, please try on independent clean JIRA instance. If you can reproduce it there, send me jira-home zip. I think it's ok to keep using this bug.

    Backup xml will not be enough, and I might not be able to launch a virtual image.

    Thank you.

  12. Igor Masen

    Hi Andriy,

    I found the error. It's quiet embarrassing, I had a space after the number of days because of copy&paste.

    It seems, that the plugin throws a correct "number format exception" since version 2.4, if it cannot read the number of days. The versions before it just failed with the error I mentioned before. :(

    Sorry for that.

  13. Andriy Zhdanov reporter

    Hi Igor,

    Oh, great it is solves with 2.4, hope you like the ability to change it in admin interface now :)

  14. Log in to comment