Set appropriate limits for different build tools based on pipelines build limits

Issue #12792 resolved
Samuel Tannous staff created an issue

Set appropriate default maven/gradle (and others?) settings for heap sizes, fork counts etc. to run builds within the constraints that pipelines applies (such as 2gb ram etc).

Comments (12)

  1. Ruben Dijkstra

    For example using MAVEN_OPTS regarding maven, GRADLE_OPTS regarding gradle. Note that the forked VM (default surefire behavior) doesn't inherit these values. At least it might go a long way with consumption of memory.

    An alternative could be JAVA_TOOL_OPTIONS, but it's /very/ chatty regarding "Picked up JAVA_TOOL_OPTIONS: <all options>"

  2. Steven Vaccarella staff

    Hi Ruben,

    Thanks for pointing me to the support ticket for more context - I hadn't seen it previously. I still feel that the correct category is enhancement rather than bug, however I've increased the priority to reflect that it is an important issue. I can't promise that the priority won't change again, but this will at least ensure that it gets more attention when we are prioritising issues.

  3. Ruben Dijkstra

    At least thanks for hearing me out. Yeah I understand it can fluctuate per milestone, but I at least wanted a fair process as far as user experience goes.

    So I am fine with this being an enhancement, but I do am curious how fast this will be fixed. Time will tell I suppose.

    Have a good day.

  4. Joshua Tjhin Account Deactivated

    Sorry for the delayed reply.

    Is setting the appropriate variables at the beginning of your script an option? I think it will be difficult for Pipelines to tell whether your build is a Gradle or Maven build except by inferring from the Docker image. This would be mean special variables for special images which is a little hard to maintain.

  5. Ruben Dijkstra

    And sorry for my delayed response as well.

    The point is, when setting up a pretty standard Maven and junit project Java sees the memory it can use and sizes the heap accordingly. Only it turns out not all the memory can be used, because of restrictions that weren't known before.

    I'll try again soon, maybe things have changed in the meantime.

  6. Matt Ryall

    Since this ticket was raised, we've bumped the ceiling on Pipelines builds up to 4 GB by default, with a large builds configuration to run with 8 GB. We also have pop-up suggestions if your build fails due to a memory error.

    Based on these improvements, I don't think this is as much of an issue any more, so we don't plan on adding any platform-specific tweaks.

    However, we do have some default Pipelines YAML templates for popular languages that we can adjust if people have some specific suggestions. Let us know if there's something you think we should add there, with MAVEN_OPTS or similar.

  7. Ruben Dijkstra

    As long as the issue is resolved I'm fine with it. The problem wasn't the ceiling, it was that the ceiling was not perceivable by the java tool being run (java, mvn, etc). So it would guess an 'okay' heap size/heap size limit, that actually wasn't okay and would get killed.

    But a pop-up suggestion sounds good, could you for posterity also include a sample of such a message? I remember being flabbergasted why the build failed(read: got Killed) on Pipelines that I had on Travis already.

    Thanks for resolving this though!

  8. Log in to comment