Need to be able to includes/excludes folders/files when checkout

Issue #22 resolved
Former user created an issue

The TFS plug-in already have some expression to detect changes from only some folders/files. It's great.

But we also don't need the files to be checkout/updated from the other folders we don't care about.

The issue is that we have a huge project structure with lot of related projects. In the root we have script that are commons and can be used to build the all solutions. Then we have 25 folders all representing sub projects. It's too slow to checkout/update all those extra folders. We need the files in the root and one or two of the folders.

So I believe this translate in TFS terms that you provide includes/excludes expression, then you use that to build the correct TFS workspace using the cloak feature.

Like: Root: pom.xml or build.gradle FolderProject1 pom.xml or build.gradle FolderProject pom.xml or build.gradle SubFolderProject pom.xml or build.gradle

Cloak http://msdn.microsoft.com/en-us/library/ms181378(v=vs.100).aspx

Comments (46)

  1. Gaurav Singh

    Please let us know the syntax for regular expression or put some examples on the page TFS Repo Page.

    While .Flows. work to fetch only Flows project, none of the below works.

    (.*Flows.*) |  (.*GradleCommons.*)
    
     .*Flows.* |  .*GradleCommons.*
    

    Additionally, the regular expressions link on the TFS page doesn't work

  2. Gaurav Singh

    Some correction : While

    .*Flows.* 
    

    work to fetch only Flows project. We want to fetch more than one projects for example Flows + Gradle Commons

    TradeData + GradleCommons

    Folder structure in TFS is

    $Java
         GradleComons
         Flows
         TradeData
         AssetLoader
         -------
    
  3. Gaurav Singh

    Thanks below worked for

    .*(Flows|GradleCommons).*  // worked
    

    Below didn't work. Can you please correct me.

    // Didn't work 
    \$/Portfolio Instrument/Development/Java/(Flows|GradleCommon)/.*
    OR
    \$/Portfolio\ Instrument/Development/Java/(Flows|GradleCommon)/.*
    
  4. Gaurav Singh

    Link started to work for me as well now. I tried couple times before and just few hours back it wasn't working. Thank you.

  5. Gaurav Singh

    I think I got it. pattern is relative to the source path. For below to work, I need to set source path to blank

    // Didn't work 
    \$/Portfolio Instrument/Development/Java/(Flows|GradleCommon)/.*
    OR
    \$/Portfolio\ Instrument/Development/Java/(Flows|GradleCommon)/.*
    
  6. Sergey Podobry
    \$/Portfolio Instrument/Development/Java/(Flows|GradleCommon)/.*
    

    This should work. Is there GradleCommon or GradleCommons?

  7. Gaurav Singh

    With the new update, Incremental checkout ( no clean ) doesn't update anything anymore. How can we downgrade the plugin to previous version. We see this in the log below but it didn't really do anything.

    simple  24-Jan-2014 13:43:43    Updating source code to revision: 296135
    simple  24-Jan-2014 13:43:44    Updated source code to revision: 296135
    
  8. Sergey Podobry

    It looks like everything is filtered out. What is your filter pattern? Try to disable it for a test.

  9. sebastien_tardif

    The issue happens without any filter of any kind but only visible when we don't do TFS Clean, so incremental update.

    I don't know how you implemented the filtering. Is it possible that old Job keep using an incompatible TFS Workspace definition?

  10. Gaurav Singh

    We just downgraded to 1.1. Plans ( incremental builds, with no filter pattern ) started to work again ( fetch new change sets )

  11. Sergey Podobry

    The issue happens without any filter of any kind but only visible when we don't do TFS Clean, so incremental update.

    Could you enable debug log (Bamboo Admin->System->Log Settings->Package->com.stellarity.bamboo.repository.TfsRepository->Save) and send me it?

  12. Sergey Podobry

    Unfortunately not, it works fine in my lab. Can we arrange a skype call and you'll show me the issue?

  13. Gaurav Singh

    I will install a new bamboo locally and try reproducing the issue. Will let you know. You may reach me out gaurav.visen @gmail OR gaurav.visen on skype

  14. Jeremy Goodwin

    I can replicate the issue with not doing a clean build but not getting any code changes to happen, we're also using TFS2012. Unfortunately I can't supply a debug log as the settings don't seem to be being sent through to the remote agent that executes the build task.

    To be clear though with the new release, executing an incremental update does NOT update the checkout folder. Executing a "Clean" however, does.

  15. Sergey Podobry

    Hi Jeremy,

    Could you describe what is your repository structure and what files have been changed and not updated?

  16. Jeremy Goodwin

    Hi Sergey,

    our repo is setup to use a branch from the default collection ( url of https://{ourinstance}.visualstudio.com/DefaultCollection , sources path of $/{project}/R4

    Updates to our project code (mainly .cs files) in a check-in are not being made to the working folder folder following checkout of the relevant revision.

  17. Jeremy Goodwin

    Glad to be of help Sergey.

    Hopefully now you can answer the reason I opened this thread ;)

    I can't get the regex include/excludes to work as expected. In our folder structure, I want all the contents in ProjectA/SubFolder1/SubSubFolder1, specific files in the root of ProjectB and specific folders within ProjectB.

    Using

    (ProjectA/SubFolder1/SubSubFolder1|(ProjectB/(SubFolder1|SubFolder2)))/.*
    

    works for getting me the content of sub folders, but I can't work out how to set the regex to also get the files in ProjectB (I don't want anything from ProjectB/SubFolder3 for example).

  18. Sergey Podobry

    You can use a negative character class:

    ProjectB/[^/]*
    

    This means everything beginning with "ProjectB/" and following by any characters except "/".

    So the resulting expression would be:

    (ProjectA/SubFolder1/SubSubFolder1|(ProjectB/(SubFolder1|SubFolder2)))/.*|ProjectB/[^/]*
    
  19. Jeremy Goodwin

    Thanks Sergey, unfortunately that doesn't work - it's not bringing down the content of ProjectB using ProjectB/[^/]* - the required sub folders come down as expected, but not the actual file content of ProjectB.

  20. Gaurav Singh

    Sergey, we tried couple of options to include all the files in root folder but didn't got it working. We want Flows project and all gradle files in root folder and we tried .

    Flows.* | ^/([^/]+.gradle)$

  21. Gaurav Singh
    Folder structure in TFS is
    $Java
         Flows
         TradeData
         AssetLoader
         artifactory.gradle
         buildAll.sh
         sonar.gradle
         version.gradle
    

    We want a fetch a project say Flows and all files in root folder ( gradle files + bash script ) . We are able to fetch Flows. We want to include the files as well.

  22. Sergey Podobry

    Confirmed. I used a wrong flag for getting a non-recursive directory content. I've created an issue #26 for that.

    Somehow I missed such configuration in my test cases. Sorry for the inconvenience. The issue will be fixed ASAP.

  23. Jeremy Goodwin

    Getting there Sergey, it's now correctly bringing back the files, however it's also bringing back the folders that we don't want (but not their sub content).

  24. Sergey Podobry

    Yes, it's a trade-off for generating less TFS requests. Do that empty folders make any troubles for you?

  25. Jeremy Goodwin

    No, it works so that's good. I guess it's rather complicated to find out if the artifact is a file or folder

  26. Log in to comment