Unpacking GitHub Copilot Metrics: Why Agent-Generated Code Doesn't Boost Your Performance Metrics

As AI-assisted development becomes standard, understanding tool impact on productivity is key. GitHub Copilot's agent mode offers powerful code generation, but a recent GitHub Community discussion highlights a critical nuance in its usage metrics. This distinction can significantly affect how organizations interpret their performance metrics for developers.

The Mystery of Missing Code Lines

A developer, haviv, reported generating over 4,000 lines of code using GitHub Copilot in VS Code's agent mode. Despite committing and pushing this code, the Copilot usage metrics API showed zero lines for code completions, recording only chat activity.

Haviv expected metrics to reflect this substantial code generation in code_lines_accepted. However, the actual API response for the period showed an empty copilot_ide_code_completions section:

{
  "date": "2026-01-30",
  "total_active_users": 5,
  "total_engaged_users": 5,
  "copilot_ide_chat": {
    "editors": [
      {
        "name": "vscode",
        "models": [
          {
            "name": "default",
            "total_chats": 26,
            "total_engaged_users": 1,
            "total_chat_copy_events": 0,
            "total_chat_insertion_events": 0
          }
        ],
        "total_engaged_users": 1
      },
      {
        "name": "VisualStudio",
        "models": [
          {
            "name": "default",
            "total_chats": 24,
            "total_engaged_users": 1,
            "total_chat_copy_events": 6,
            "total_chat_insertion_events": 0
          }
        ],
        "total_engaged_users": 1
      },
      {
        "name": "JetBrains",
        "models": [
          {
            "name": "default",
            "total_chats": 1,
            "total_engaged_users": 1,
            "total_chat_copy_events": 0,
            "total_chat_insertion_events": 0
          }
        ],
        "total_engaged_users": 1
      }
    ],
    "total_engaged_users": 3
  },
  "copilot_dotcom_chat": {
    "models": [
      {
        "name": "default",
        "total_chats": 22,
        "total_engaged_users": 3
      }
    ],
    "total_engaged_users": 3
  },
  "copilot_dotcom_pull_requests": {
    "total_engaged_users": 0
  },
  "copilot_ide_code_completions": {
    "total_engaged_users": 0
  }
}

This discrepancy prompted questions about how Copilot's agent/chat mode contributions are tracked.

Two Distinct Buckets for Copilot Metrics

Madhukar2006 clarified that GitHub Copilot metrics categorize usage into two separate buckets:

1. IDE Code Completions

• These are inline suggestions (tab-complete, ghost text).
• Only these interactions populate metrics like copilot_ide_code_completions, code_lines_suggested, and code_lines_accepted.
• If inline completions aren't used, this section will report zeros.

2. Copilot Chat / Agent Mode

• This includes code generated via the chat interface, agent mode, or multi-file actions.
• These interactions are tracked only as chat activity (total_chats, total_engaged_users, copy / insertion events).
• Crucially, lines of code from chat or agents are NOT counted in "code completion" metrics.

The API measures interaction type, not the final code outcome, and does not infer Copilot usage from git commits.

Impact on `Developer Activities` and `Software Development Plan`

This distinction significantly impacts how organizations monitor developer activities and plan their software development plan. Teams heavily using agent or chat modes may show low or zero "code completion" usage. This doesn't mean low productivity; rather, it highlights a limitation in current reporting where the value of agent-assisted code generation isn't fully captured in line-based metrics. Its impact, such as scaffolding complex structures, is currently reflected only in chat engagement.

The Path Forward: Unified Metrics

This behavior is an expected product limitation. For a holistic view of Copilot's impact on performance metrics for developers, the community should advocate for a feature request for:

• Agent-mode code attribution.
• Unified "Copilot-assisted code" metrics combining contributions from both inline completions and chat/agent modes.

Accurate metrics are vital. As AI tools evolve, so must the metrics used to evaluate their diverse contributions to the software development plan.