Caching Git LFS in AppVeyor to Avoid Large GitHub LFS Bills
In order to avoid having large files in our Git history we’ve been using Git Large File Storage (LFS). It commits a marker in the Git repository and uploads files to a separate place.
On GitHub the billing for Git LFS is based on storage and bandwidth. While significant bandwidth can be used by each contributor pulling down all the files locally we’ve found a significant bandwidth contributor: Continuous Integration Builds Agents. Using AppVeyor to run out builds means each build is a full clone, which includes downloading all of the Git LFS assets needed to build the application. And, without any caching on the build server this is a full download of all necessary Git LFS assets for each build.
Here’s what you need in terms of AppVeyor configuration. This script could be adapted for services other than AppVeyor.
image: Visual Studio 2017 cache: - .git\lfs\objects clone_script: echo Skip AppVeyor Clone install: # Adapted from https://help.appveyor.com/discussions/problems/6274-git-lfs-and-build-cache#comment_43676282 - git init %APPVEYOR_BUILD_FOLDER% - cd %APPVEYOR_BUILD_FOLDER% - git remote add origin email@example.com:%APPVEYOR_REPO_NAME%.git # Updated to use SSH - git fetch -q origin %APPVEYOR_REPO_COMMIT% - git checkout -qf %APPVEYOR_REPO_COMMIT% before_build: - ps: git lfs prune | Out-Null build_script: - ps: ./build.ps1 test: off
AppVeyor typically handles the cloning of the repository for you. In this case, we want to remove that functionality by overriding the
clone_script and taking care of it ourselves.
This was adapted from https://help.appveyor.com/discussions/problems/6274-git-lfs-and-build-cache#comment_43676282, so thanks to trevor.sandy for something to work from.
I hope this helps you reduce your GitHub bills!