Caching Git LFS in AppVeyor to Avoid Large GitHub LFS Bills

In order to avoid having large files in our Git history we’ve been using Git Large File Storage (LFS). It commits a marker in the Git repository and uploads files to a separate place.

On GitHub the billing for Git LFS is based on storage and bandwidth. While significant bandwidth can be used by each contributor pulling down all the files locally we’ve found a significant bandwidth contributor: Continuous Integration Builds Agents. Using AppVeyor to run out builds means each build is a full clone, which includes downloading all of the Git LFS assets needed to build the application. And, without any caching on the build server this is a full download of all necessary Git LFS assets for each build.

appveyor.yml

Here’s what you need in terms of AppVeyor configuration. This script could be adapted for services other than AppVeyor.

image: Visual Studio 2017

cache:
  - .git\lfs\objects

clone_script: echo Skip AppVeyor Clone

install:
  # Adapted from https://help.appveyor.com/discussions/problems/6274-git-lfs-and-build-cache#comment_43676282
  - git init %APPVEYOR_BUILD_FOLDER%
  - cd %APPVEYOR_BUILD_FOLDER%
  - git remote add origin git@github.com:%APPVEYOR_REPO_NAME%.git # Updated to use SSH
  - git fetch -q origin %APPVEYOR_REPO_COMMIT%
  - git checkout -qf %APPVEYOR_REPO_COMMIT%

before_build:
  - ps: git lfs prune | Out-Null

build_script:
  - ps: ./build.ps1

test: off

AppVeyor typically handles the cloning of the repository for you. In this case, we want to remove that functionality by overriding the clone_script and taking care of it ourselves.

This was adapted from https://help.appveyor.com/discussions/problems/6274-git-lfs-and-build-cache#comment_43676282, so thanks to trevor.sandy for something to work from.

I hope this helps you reduce your GitHub bills!