Copying files in multistage docker pipeline fails

Issue #17241 resolved
Tom De Coninck created an issue

I get the following error when trying to copy files from a previous build stage

Step 7/18 : COPY --from=uikit /app/binders-ui-kit.tgz  /app

failed to copy files: failed to copy file: Container ID 166536 cannot be mapped to a host ID

The file in question and its parent directoyr are owned by a user node (id:1000) (inside the uikit container). This is also the same user of the build container by using the USER directive.

Comments (9)

  1. Kerry Martin

    Same issue with our builds that just recently started in last ~3 days.

    Step 10/14 : COPY --from=builder /home/gradle/gdpr/gdpr-request-tracker-service.jar .
    failed to copy files: failed to copy file: Container ID 166536 cannot be mapped to a host ID
    

    Directories owned by non-root user gdpr id:1002. The same configuration ran successfully as of 3 days ago.

  2. Yanne Grmek

    Our builds are also suffering from this issue.

    It seems to be from a combination of two things:

    1. Atlassian enabling user namespace remapping in the docker in docker daemon. This was done Sep 27: https://confluence.atlassian.com/bitbucket/infrastructure-changes-in-bitbucket-pipelines-872135046.html

    2. A bug with Docker with user namespace remapping enabled: https://github.com/moby/moby/issues/34645

    For now, I'm using the workaround suggested in the moby issue - chowning the files in the --from build step. It works, but it's hacky. Does anyone have any other alternative?

  3. Tom De Coninck reporter

    The suggested workaround is not working in our case, so pipelines are unusable for us at this point.

  4. Sukrit Khera

    @tadeutsch for the workaround are you using chown when your build is running as a root or a non-root user. I did notice that for chown to work, you need to be a root user. After running my build as root, this workaround seems to be working.

    USER root

    run chown or simply run build as root user

  5. Tom De Coninck reporter

    @sukrit008 Thats exactly what I tried. I modified the build so in the --from container I did a USER root before I chowned the file. Also in the stage where I do the copy I did a USER root just before the copy to be sure. But no luck...

  6. Sukrit Khera

    @tadeutsch can you add the snippet. Here is the snippet that is working for us now (After commenting out # User gradle in our build step

    # Stage 1 : Build
    FROM gradle:4.5.1-alpine as builder
    
    WORKDIR /home/gradle/scsdemo
    
    COPY . .
    
    USER root
    RUN chown -R gradle .
    # USER gradle -  Commented this for docker multistage build issue on bitbucket
    
    RUN gradle bootJar \
        && ls -la build/libs/ \
        && mv build/libs/scsdemo-*.jar scsdemo.jar \
        && rm -rf build .gradle /home/.gradle
    
    # Stage 2:  Production Image
    FROM openjdk:8u151-jre-alpine as prod
    
    WORKDIR /opt/scsdemo
    COPY --from=builder /home/gradle/scsdemo/scsdemo.jar .
    
    
    COPY docker/run.sh .
    
    RUN apk add --no-cache --update curl \
        && echo "Adding gdpr user and group" \
        && addgroup -S -g 1002 gdpr \
        && adduser -D -S -G gdpr -u 1002 gdpr \
        && chown -R gdpr:gdpr . \
        && chmod 755 run.sh \
        # DUMB Init
        && wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_amd64 \
        && chmod +x /usr/local/bin/dumb-init
    
    USER gdpr
    
    ENTRYPOINT ["/usr/local/bin/dumb-init", "--"]
    CMD ["./run.sh"]
    
  7. devste NA

    Hi,

    we had the same issue with our builds. I found out that in the message Container ID 166536 cannot be mapped to a host ID the Container ID can refer to UserIDs as well as GroupIDs.

    The files in question were owned by user root, but group staff. So adding this to the source stage solved our problem for us (in our case, we're copying from /usr/local)

    RUN chgrp -R root /usr/local

  8. Tom De Coninck reporter

    @sukrit008 @devste I was indeed missing the group in my chown. If I modify the groups as well, the pipeline succeeds! Thanks for the hint!

  9. aneita staff

    Hi everyone,

    @yanneg hit the nail on the head - we've recently made some changes to the way pipelines run. As a result of these changes, some users are running into a bug in Docker which are affecting the result of their pipelines. The public Docker issue linked mentions a workaround which may work for you.

    Given the issue is being tracked on the Docker issue tracker, I'm going to close this issue off. I encourage you to watch their issue for updates on the status of the bug.

    If you're experiencing any additional problems, please raise a support ticket so that we can look at your repository in more detail.

    I apologise for any inconvenience.

    Thanks,
    Aneita

  10. Log in to comment