Trigger pipeline only if certain files are changed

Issue #16560 open
ZenobiusJ
created an issue

So I posted this on the Community Discussion site, but it's worth having here.

We need to restrict our pipeline steps to only run when commits contain changes in our client/**/* filepath glob.

Concourse.ci, Drone.ci, and I assume other CI servers provide the option to restrict steps to commits only containing certain file paths.

At the moment it seems we are limited to manually grepping "git log" and emitting an exit 1, which would cause the build to fail.

Ideally I'd want a branch build to not even start, so being able to specify branches and steps to only run when there are (or are not) matching files in the commit would look like:

- step:
  image: node:alpine-8
  script:
    - npm install
    - npm run prod
  filter:
    includes:
      - client/src/**/*
      - package.json

...

So that commits containing file changes matching:

  • server/**/* and client/**/* would run the above step
  • server/**/* would not run the above step
  • client/**/* would run the above step
  • package.json would run the above step

Other examples:

- step:
  filter:
    excludes:
      - server/**/*.cshtml
- step:
  filter:
    includes:
      - client/build/gulp/**/*
    excludes:
      - server/**/*.cshtml

excludes would prevent the step from running if any files in the commit match the patterns.

includes would ensure that the step only ran if at least one of the changed files matches the patterns.

Comments (38)

  1. ZenobiusJ reporter

    It occured to me that perhaps the best place to include this feature is in the trigger keyword.

    ... 
    
    - step:
      trigger:
        includes:
          - client/src/**/*
    
  2. Aneita Yang staff
    • changed status to open

    Hi @ZenobiusJ,

    Thanks for reaching out and for the suggestion - I can see why having this feature would be beneficial to you.

    We currently don't have any plans to support this, but in the meantime, I'll open this issue to see whether other users are interested in seeing the same thing.

    Aneita

  3. ZenobiusJ reporter

    @Aneita Yang Thanks for that. :)

    For other peoples benefit, this is how we're dealing with this at the moment:

        sprint/*:
          - step:
              name: filter
              image: <our-nodejs-image-with-git-installed>
              script:
                - |
                  cd /data
                  ln -s $BITBUCKET_CLONE_DIR/.git/ ./
                  cp -r $BITBUCKET_CLONE_DIR/package.json $BITBUCKET_CLONE_DIR/client ./
                  node client/ci/commit-filter.js --include=client/src/**/* --include=bitbucket-pipelines.yml --include=client/build/**/* --include=client/ci/**/*
    
    ...
    

    and the client/ci/commit-filter.js looks like:

    #!/bin/node
    var run = require('child_process').execSync;
    var minimatch = require('minimatch');
    var log = require('debug')('ci/commit-filter');
    
    var config = require('nconf')
        .use('memory')
        .argv({
            include: {
                alias: 'i',
                type: 'array',
                parseValues: true,
            },
            exclude: {
                alias: 'e',
                type: 'array',
                parseValues: true,
            }
        })
        .get();
    
    
        const changed = run(`git diff-tree --no-commit-id --name-only -r HEAD`)
            .toString()
            .split('\n')
            .filter(line => line.trim().length > 0);
    
        function match (patterns = []) {
            return patterns.reduce((result, pattern) => {
                return result.concat(changed.map(path => minimatch(path, pattern)));
            }, []);
        }
    
    const {include, exclude} = config;
    log({include, exclude});
    
    if (!include && !exclude) {
        log('No include or exclude');
        process.exit(1);
    }
    const isIncluded = match(include).some(yesno => yesno == true);
    const isExcluded = !match(exclude).some(yesno => yesno == true);
    
    log({isExcluded, isIncluded});
    
    if (include && !isIncluded) {
    console.log(` Bailing because
    - Commit [${process.env.BITBUCKET_COMMIT}] didn't contain any of required files
        + ${include.join('\n    +')}
    
    - Changed files
        + ${changed.join('\n    +')}
    `);
    
        process.exit(1);
    }
    
    if (exclude && isExcluded) {
        console.log(`Bailing because:
    - Commit [${process.env.BITBUCKET_COMMIT}] contained a black listed file
        + ${exclude.join('\n    +')}
    
    - Changed files
        + ${changed.join('\n    +')}
    `)
        process.exit(1);
    }
    
    process.exit(0);
    
  4. Jason Edstrom

    I have a selenium automation framework that I built/use and only need the package compiled and cached if there was a code change in those folders. This would be extremely handy to conserve build minutes.

  5. Janne Nykänen

    I'm been using a script based method similar to this for files, directories and so on: ... script: - if git diff-tree --no-commit-id --name-only -r ${BITBUCKET_COMMIT} | grep THE_FILE_NAME; then echo "THE_FILE_NAME changed, updating..." #do-stuff ; else echo "No need for building"; fi

  6. Zvika Nadav

    There are a lot of use cases where you would like to trigger a pipeline only if a specific changeset of files would occur - such as Dockerfiles/ Helm (k8s) and many more examples. This is a very important feature for us bitbucket pipeline customers. +1 for this feature, many other tools such as bitbucket pipelines already has this implemented, no reason why not to add this here as-well :)

    Thanks

  7. Log in to comment