Issue #116 resolved

memory error uploading file

cdoucette
created an issue

When one of my colleagues attempts to push back the contents of a relatively new repository (it was created empty on the server and populated on the client), it gets to almost the very end and then there was an error. This is the error from the server log:

11:53:32.895 [qtp27148830-235] ERROR sonia.scm.web.cgi.DefaultCGIExecutor - could not read from ServletInputStream and write to ProcessOutputStream java.io.IOException: The pipe has been ended at java.io.FileOutputStream.writeBytes(Native Method) ~[na:1.7.0_03] at java.io.FileOutputStream.write(Unknown Source) ~[na:1.7.0_03] at java.io.BufferedOutputStream.write(Unknown Source) ~[na:1.7.0_03] at sonia.scm.util.IOUtil.copy(IOUtil.java:194) ~[scm-core-1.14-SNAPSHOT.jar:na] at sonia.scm.web.cgi.DefaultCGIExecutor.processServletInput(DefaultCGIExecutor.java:557) [classes/:na] at sonia.scm.web.cgi.DefaultCGIExecutor.execute(DefaultCGIExecutor.java:392) [classes/:na] at sonia.scm.web.cgi.DefaultCGIExecutor.execute(DefaultCGIExecutor.java:179) [classes/:na] at sonia.scm.web.HgCGIServlet.process(HgCGIServlet.java:258) [scm-hg-plugin-1.14-SNAPSHOT.jar:na] at sonia.scm.web.HgCGIServlet.service(HgCGIServlet.java:178) [scm-hg-plugin-1.14-SNAPSHOT.jar:na] at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [javax.servlet-2.5.0.v201103041518.jar:na] at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at sonia.scm.web.filter.PermissionFilter.doFilter(PermissionFilter.java:149) [scm-core-1.14-SNAPSHOT.jar:na] at sonia.scm.web.filter.HttpFilter.doFilter(HttpFilter.java:102) [scm-core-1.14-SNAPSHOT.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at sonia.scm.web.filter.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:139) [scm-core-1.14-SNAPSHOT.jar:na] at sonia.scm.web.filter.HttpFilter.doFilter(HttpFilter.java:102) [scm-core-1.14-SNAPSHOT.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at sonia.scm.filter.BaseUrlFilter.doFilter(BaseUrlFilter.java:100) [classes/:na] at sonia.scm.web.filter.HttpFilter.doFilter(HttpFilter.java:102) [scm-core-1.14-SNAPSHOT.jar:na] at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118) [guice-servlet-3.0.jar:na] at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113) [guice-servlet-3.0.jar:na] at sonia.scm.boot.BootstrapFilter.doFilter(BootstrapFilter.java:104) [classes/:na] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331) [jetty-servlet-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:477) [jetty-servlet-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) [jetty-security-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) [jetty-servlet-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.Server.handle(Server.java:349) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:894) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:948) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857) [jetty-http-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) [jetty-http-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:77) [jetty-server-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609) [jetty-io-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45) [jetty-io-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) [jetty-util-7.6.2.v20120308.jar:7.6.2.v20120308] at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) [jetty-util-7.6.2.v20120308.jar:7.6.2.v20120308] at java.lang.Thread.run(Unknown Source) [na:1.7.0_03] 11:53:32.895 [Thread-827] WARN sonia.scm.web.cgi.DefaultCGIExecutor - Traceback (most recent call last): File "C:\Documents and Settings\cdoucette.scm\lib\python\hgweb.py", line 42, in <module> wsgicgi.launch(application) File "C:\Programs\Python27\lib\site-packages\mercurial\hgweb\wsgicgi.py", line 76, in launch content = application(environ, start_response) File "C:\Programs\Python27\lib\site-packages\mercurial\hgweb\hgweb_mod.py", line 91, in call return self.run_wsgi(req) File "C:\Programs\Python27\lib\site-packages\mercurial\hgweb\hgweb_mod.py", line 127, in run_wsgi return protocol.call(self.repo, req, cmd) File "C:\Programs\Python27\lib\site-packages\mercurial\hgweb\protocol.py", line 76, in call rsp = wireproto.dispatch(repo, p, cmd) File "C:\Programs\Python27\lib\site-packages\mercurial\wireproto.py", line 348, in dispatch return func(repo, proto, *args) File "C:\Programs\Python27\lib\site-packages\mercurial\wireproto.py", line 572, in unbundle proto.getfile(fp) File "C:\Programs\Python27\lib\site-packages\mercurial\hgweb\protocol.py", line 48, in getfile for s in util.filechunkiter(self.req, limit=length): File "C:\Programs\Python27\lib\site-packages\mercurial\util.py", line 898, in filechunkiter s = nbytes and f.read(nbytes) File "C:\Programs\Python27\lib\site-packages\mercurial\hgweb\request.py", line 65, in read return self.inp.read(count) MemoryError

11:53:32.895 [qtp27148830-235] WARN sonia.scm.web.cgi.DefaultCGIExecutor - process ends with exit code 1

Comments (11)

  1. cdoucette reporter

    I pasted the error I saw in the log; however, it may be related to the fact it also ran out of disk space. FYI my account is on the C drive (and thus that's where SCM was installed by default). The C drive didn't have much space. I had to install VS 2008 C++ Express in order to build/install SCM. My TEMP folder also referred to the C drive (via %USERPROFILE%\Local Settings\Temp). Anyhow, when I redefined TEMP and uninstalled VS 2008 C++ Express, our user was able to upload his repositories. We had already redefined the repository directory for Mercurial to be on another drive.

  2. cdoucette reporter

    Well - we can still reproduce this occasionally when uploading a large changeset and/or a large number of changesets. I'm not sure if/how I can help you reproduce this or get more information for if/when it happens again.

  3. Sebastian Sdorra repo owner

    How much memory has your machine and how much memory is used by the java process of SCM-Manager?

    I ask this because SCM-Manager (or java) uses the fork method to create the python process for mercurial. The problem with fork is that the kernel tries to allocate the same amount of memory as the parent process.

  4. cdoucette reporter

    (Reply via CDou...@everyscape.com):

    The server machine has 2GB of ram. It appears that your windows service executable, prunsrv.exe, is currently using 77,224KB (77.224MB). I don't see a Java process, so I'm not sure if/how it is involved (or what its JVM limits are).

    Chuck

  5. Sebastian Sdorra repo owner
    • changed status to open

    You could try to increase the memory of the jvm by modify the scm-server.bat. Search the following line:

    set EXTRA_JVM_ARGUMENTS=-Dlogback.configurationFile=logging.xml
    

    and replace it with the line below:

    set EXTRA_JVM_ARGUMENTS="-Xmx512m -Dlogback.configurationFile=logging.xml"
    

    But i think the problem is the python process and not the scm-manager process. I've seen you are using python 2.7, which mercurial version you are using? Have you tried one of the packages provided by the mercurial installation wizzard of scm-manager?

  6. Log in to comment