Review GitHub AI Security Detections Without Making Them a Merge Gate

John Burns

Security, Automation

GitHub’s AI security detections can place a new class of finding beside ordinary code-scanning results when a pull request is opened or updated. That can be useful for teams maintaining AI-assisted or conventional application code, but it is not a reason to let an automatically generated finding change a production branch by itself. GitHub describes these detections as informational: they do not block a merge, and they depend on GitHub Code Security plus CodeQL default setup.

This guide sets up a safe review path for an eligible GitHub Enterprise Cloud organization. The result is a repository where CodeQL default setup supplies the required analysis, AI security detections are enabled only after an owner approves their use, and every finding is either reproduced and fixed, documented as a false positive, or escalated for a more complete review. The menu names and feature availability are documentation-verified. The examples are a review procedure, not results from a particular repository.

Decide what the feature is allowed to do

Start with a narrow purpose. AI security detections are an additional signal for reviewers, not evidence that a vulnerability is exploitable and not a substitute for threat modeling, testing, or a human code review. In the July 2026 public preview announcement, GitHub says the findings are produced when pull requests open or change, appear as the detection engine returns them, and are informational rather than merge-blocking.

That behavior makes a review-only introduction the safe default. Do not add a branch-protection rule that treats an informational alert as a required status check. Doing so can create an opaque merge dependency with no agreed owner, remediation time, or false-positive process. It can also encourage a hurried dismissal when an alert appears late in a release.

Write down the initial policy before enabling the feature. A small policy can be enough to give reviewers a consistent outcome:

Scope: repositories with CodeQL default setup and an assigned security reviewer.
Trigger: a pull request has an AI security detection.
Response: the author acknowledges it; a reviewer classifies it before merge.
High confidence: create a remediation change and add a regression test where practical.
Uncertain: open a security review issue with the alert link, data flow, and decision owner.
False positive: record why the reported source, sink, or trust boundary is not reachable.
Secrets: never paste tokens, production prompts, customer data, or full incident logs into the alert discussion.

The policy is deliberately about decisions, not about trusting a model score. A finding involving an untrusted document, web page, repository issue, or user-provided prompt may deserve prompt-injection review even if the application does not advertise itself as an AI product. Conversely, an alert is not automatically a defect merely because a string resembles an instruction. The reviewer needs to establish the actual data flow and the action the application could take.

Confirm eligibility and enable the dependency first

GitHub’s announcement says AI security detections must be allowed by enterprise policy and enabled at the organization level. It also says CodeQL default analysis must be enabled for the repository. CodeQL is not the analysis engine that generates the AI detection, but it is a dependency for the feature.

Before changing a repository, identify the enterprise owner who can approve the feature and the organization security contact who will receive questions about findings. If those roles are not assigned, stop at the policy stage. Enabling a new source of security alerts without an owner simply transfers unreviewed work to pull-request authors.

For a repository that is in scope, use the repository’s Settings, then Code security, and find the CodeQL analysis section. GitHub’s default-setup documentation describes selecting Set up and then Default. Default setup is a good starting point when the repository has no existing customized CodeQL workflow and the supported-language detection matches the codebase.

Do not switch blindly from advanced setup to default setup. A repository with a committed .github/workflows/codeql.yml, custom build steps, a private query pack, or nonstandard generated code may need its existing advanced configuration. Record its current setup first:

git ls-files '.github/workflows/*codeql*' '.github/codeql*'
git status --short
git log -1 -- .github/workflows

These commands only inventory the checked-out repository. They do not tell you whether GitHub has default setup enabled, so use the repository settings as the source of truth for that state. If a custom configuration is present, ask the team that owns it whether default setup is appropriate rather than disabling the workflow to make the new feature appear.

After CodeQL default setup is enabled and completing successfully, have the enterprise and organization administrators enable AI security detections through the controls available to their plan and preview enrollment. GitHub’s July 14 announcement specifically lists enterprise permission, organization-level enablement, GitHub Code Security, and CodeQL default setup as prerequisites. If an expected setting is absent, treat that as an availability or entitlement question, not a reason to add a copied workflow from an unrelated repository.

Make the first pull request a controlled test

Use a normal, low-risk documentation or test-only pull request to confirm that the repository receives the expected analysis. Do not introduce an intentionally vulnerable production code path just to force an alert. A harmless pull request proves integration without creating a misleading vulnerability history or a risky example that might later be copied into application code.

On the pull request, check the Security or code-scanning area for the CodeQL run and, when the feature is available, the AI security detection area. Record the pull-request URL, repository, commit SHA, time started, and reviewer in the change record. A missing detection is not proof that the feature failed: there may simply be nothing to report, and GitHub says results appear as the engine returns them. What matters for the initial check is that the prerequisite analysis completed and that the team knows where a future result will appear.

Keep the confirmation separate from a production deployment. The reviewer should verify that the normal required checks still behave as expected, that the new informational findings have not become a hidden merge gate, and that notification routing reaches the assigned owner. If the feature generates a finding during the test, use the classification process below rather than trying to clear the alert merely to finish the rollout.

Triage a finding from the code outward

Begin with the exact code location and pull-request diff, then trace outward. The first question is not whether the alert text sounds plausible. It is whether data from an untrusted source can reach an action with a meaningful security effect.

For a suspected prompt-injection path, identify the source precisely. It might be a chat message, uploaded document, issue body, web page, ticket, database record, or tool response. Then identify the boundary where the application turns that content into model context. Finally, identify the sensitive action: invoking a tool, changing a record, sending a message, disclosing data, or selecting a privileged instruction.

Use a review worksheet that captures those facts without copying sensitive content into the pull request:

Alert URL and commit:
Untrusted source and who can control it:
Where the application labels, quotes, or isolates the data:
Model, agent, tool, or workflow that receives it:
Sensitive action reachable from that component:
Authorization check before the action:
Reproduction in a non-production environment:
Decision, reviewer, and follow-up issue:

A useful reproduction is small and owned by the team. For example, use synthetic input in a test environment to show whether untrusted text can cause a tool invocation that the application should have rejected. Do not use a real customer document, a production credential, or a live third-party target. The goal is to test the application’s authorization and data-handling boundary, not to demonstrate that a model can repeat hostile text.

If the data cannot reach an action, document the concrete reason: perhaps the input is never placed into a model context, a strict allowlist rejects the requested tool, or a server-side authorization check independently denies the operation. “The model should ignore it” is not a sufficient closure reason. Model instructions can help, but a sensitive action needs an enforcement point outside untrusted model output.

Fix the boundary and verify the change

Remediation usually belongs at the boundary closest to the sensitive action. Keep tool capabilities narrow, pass structured parameters instead of untrusted free text where possible, and check authorization in the service that performs the action. Treat model output as a request that must pass the same validation an ordinary user request would need.

For an agent that reads external content and can call an internal tool, a safer shape is an allowlisted operation with server-side checks:

external content -> untrusted-data field -> model proposes action
model proposal -> schema validation -> allowlisted operation
allowlisted operation -> server-side authorization -> audited execution

The important setting is not the label on the model prompt. It is that the executor accepts only known operations and evaluates identity, target, and permission after the model has proposed an action. Log the decision with a request or trace identifier, but avoid logging credentials, raw private documents, or complete prompts when a summary is enough.

Add a regression test that uses synthetic hostile-looking input and asserts the protected action is denied or requires an explicit authorized approval. The test should verify the security property, such as “a document cannot expand the permitted recipient set,” rather than asserting a particular model sentence. Models and detection engines can change; authorization behavior should remain stable.

Before merging, link the remediation pull request to the original alert and have the assigned reviewer confirm the source, boundary, action, and test evidence. If the issue needs architectural work, track it with an owner and deadline rather than closing it as a false positive. This preserves the value of a new detection source without treating any one alert as a final security verdict.

GitHub’s AI security detections are most useful when they add a visible review signal to an already owned process. With CodeQL default setup confirmed, enterprise and organization controls approved, and a documented path from alert to evidence-based decision, the feature can improve pull-request review without becoming an unexamined automated gate.

Sources

Previous post

Prepare a Node.js Project for npm v12 Install Security