Unraveling Phantom Git LFS Spikes: GitHub Billing Changes & Developer Metrics
The Mystery of the Sudden Git LFS Spike
Imagine waking up to an email stating your Git LFS storage has jumped 1GB overnight, pushing you to 100% usage, all without a single file change in your repositories. This perplexing scenario was recently shared by a GitHub user, h8ku, sparking a community discussion that revealed many developers facing similar phantom increases. While traditional developer metrics focus on code output and efficiency, unexpected resource consumption like this can severely impact workflow and budget.
Initial Theories from the Community
Before a clear root cause emerged, the community offered several plausible explanations for LFS usage increases that don't involve direct file modifications:
- Background Processing: Newly pushed LFS objects might take time to be fully processed and reflected in usage reports.
- History Rewrites: Force pushes or history rewrites can leave orphaned LFS objects that still count towards storage, even if no longer referenced by active branches.
- Multiple Versions: Every new version of a file tracked by LFS counts as a separate object, even if the latest version appears identical.
- CI/CD Artifacts: Automated workflows might inadvertently upload large artifacts tracked by LFS.
- Delayed Reporting: Billing systems and email alerts can sometimes update faster than detailed usage reports.
- Cross-Repo References: Other repositories or forks within the organization might reference the same LFS objects.
A common thread through these initial theories was the frustration over the lack of a detailed, public-facing audit log for LFS object changes, making it incredibly difficult for developers to pinpoint the exact source of increased usage.
The Breakthrough: GitHub's Billing Platform Migration
The most critical insight came from a community member who identified the likely root cause: GitHub's ongoing migration to a new billing platform. This transition has led to several key issues:
- New Billing System: Accounts are moving from an older "Data Packs" model to a new platform based on actual usage and managed via budgets.
- Default $0 Budget: Many migrated accounts, especially free organizations, are finding their default LFS budget set to $0. If your usage exceeds the *new* included limit for your plan, this $0 budget effectively stops access and reports you as over-quota, even if your actual stored data hasn't changed. This directly impacts how your developer metrics around resource consumption are perceived.
- UI Discrepancy: The new platform often removes the per-repository LFS breakdown in the billing UI, adding to the confusion when trying to reconcile usage.
Your Step-by-Step Solution Guide
If you're experiencing phantom Git LFS usage increases, here's how to diagnose and resolve the issue:
- Check Your Budget Settings:
Navigate to your Organization settings > Billing and licensing > Budget and alerts. Find the budget for Git LFS. If it's set to $0, this is likely the problem. Either increase the budget to a positive amount (you may need to add a payment method) or uncheck "Stop when budget exceeded" to disable the limit.
- Understand New Included Usage:
Before purchasing, review your new included limits. GitHub's new platform often offers significantly increased included storage and bandwidth. You might not need to pay if your usage is within these new limits; you just need to adjust the budget to allow it.
- Request a Manual Cleanup (If Necessary):
If you suspect actual over-usage due to orphaned LFS objects not reflected in your repo, contact GitHub Support. Explain that you've cleaned your repo locally but your LFS quota is still full. They can manually clean up unreferenced Git LFS objects on the remote server. This is often the only way to clear space taken up by "phantom" files in GitHub's backend.
Support Link: https://support.github.com/contact/account - Ask for Account Migration (If on Old System):
If you can't find the "Budget and alerts" section, your account might not have been migrated yet. Contact GitHub Support and ask them to manually migrate your organization to the new billing platform. Once migrated, you'll have access to the budget controls.
While there's no public-facing audit log for LFS object changes, the sudden increase is almost certainly a billing policy enforcement issue, not a data storage bug. Adjusting your LFS budget settings is the primary fix, ensuring your developer metrics accurately reflect your operational capacity.