Product case study · October 30, 2026

Gemini CLI Headless JSON: Validate Before You Automate

Capture Gemini CLI output and errors separately, validate its JSON envelope, and reject a wrong answer before passing it to another script.

Reading time
7 min
Checked
Oct 30, 2026
A blue paper funnel and charcoal inspection frame sort cream evidence cards beside an ochre review card on warm paper
Check the command, unpack the response, then verify the answer.
Bottom line

Treat process status, response format and answer correctness as three separate checks before accepting an automated result.

A script can receive valid JSON from an AI agent and still accept the wrong answer. This lab gives you three checkpoints: did the command succeed, did it return the expected format, and does the answer match the supplied facts?

You will feed Gemini CLI a fictional release log, save the raw response, and validate a tiny answer before another program uses it. The useful artifact is a repeatable acceptance gate with failure evidence.

Sources checked October 30, 2026. This is a documentation-backed exercise. We tested the local validator with synthetic responses; we did not run an authenticated Gemini request. The model output described here is an acceptance target, not a reported result.

You need Bash, Python 3, and an installed, authenticated Gemini CLI for the final model step. You can complete the validator exercise without credentials. Use only the fictional data below.

Understand the two JSON layers

Google’s headless reference documents --output-format json as a single object with a response string, usage statistics in stats, and optional error details. That outer object is the CLI’s response envelope. Asking the model to answer with JSON does not turn that string into a guaranteed application schema.

Our own answer contract is an object with exactly two fields: failed_count, an integer, and failed_checks, an array of names. The validator will parse both layers and compare the answer with this lab’s known result. This contract is authored for the exercise; it is not a Gemini API feature.

Set up one disposable workspace

Keep this Bash terminal open for the exercise:

lab_root=$(mktemp -d /tmp/gemini-json-lab.XXXXXX)
mkdir -p "$lab_root/project" "$lab_root/evidence"
cd "$lab_root/project"
cat > checks.txt <<'LOG'
unit-tests: PASS
link-audit: FAIL
content-schema: PASS
image-audit: FAIL
LOG
printf 'Lab root: %s\n' "$lab_root"

The correct result is two failed checks: link-audit and image-audit. Preserve the printed path. If you change terminals, set lab_root to that path and return to its project directory.

Save the following as validate.py in that directory:

import json
import sys
from pathlib import Path


def validate(raw):
    envelope = json.loads(raw)
    if not isinstance(envelope, dict):
        raise ValueError("CLI envelope must be an object")
    if envelope.get("error") is not None:
        raise ValueError("CLI reported an error")
    response = envelope.get("response")
    if not isinstance(response, str) or not response.strip():
        raise ValueError("Missing nonempty response string")
    answer = json.loads(response)
    if not isinstance(answer, dict):
        raise ValueError("Answer must be an object")
    if set(answer) != {"failed_count", "failed_checks"}:
        raise ValueError("Unexpected answer fields")
    count = answer["failed_count"]
    checks = answer["failed_checks"]
    if type(count) is not int or count != 2:
        raise ValueError("Expected integer failed_count of 2")
    if not isinstance(checks, list) or not all(
        isinstance(item, str) for item in checks
    ):
        raise ValueError("failed_checks must be a string array")
    if len(checks) != 2 or set(checks) != {"link-audit", "image-audit"}:
        raise ValueError("Failed check names do not match the fixture")
    return answer


if __name__ == "__main__":
    try:
        result = validate(Path(sys.argv[1]).read_text(encoding="utf-8"))
    except (IndexError, OSError, ValueError) as exc:
        print(f"REJECT: {exc}", file=sys.stderr)
        sys.exit(1)
    print(json.dumps(result, sort_keys=True))

This intentionally rejects Markdown fences and extra commentary inside response. Do not quietly strip unexpected output to make the check green. Record the mismatch so you can decide whether the prompt or the consumer needs to change. The validator tolerates extra outer envelope fields because the CLI may include metadata unrelated to the answer.

Prove the rejection paths without a model call

Run this from the same directory:

python3 - <<'PY'
import json
from validate import validate

expected = {"failed_count": 2, "failed_checks": ["link-audit", "image-audit"]}
good = json.dumps({"response": json.dumps(expected)})
assert validate(good) == expected
bad_cases = {
    "broken envelope": "{",
    "missing answer": "{}",
    "CLI error": json.dumps({"error": {"message": "synthetic failure"}}),
    "plain text answer": json.dumps({"response": "Two checks failed."}),
    "wrong count": json.dumps({"response": json.dumps({**expected, "failed_count": 0})}),
    "wrong names": json.dumps({"response": json.dumps({**expected, "failed_checks": ["unit-tests", "image-audit"]})}),
}
for name, raw in bad_cases.items():
    try:
        validate(raw)
    except ValueError:
        print("PASS: rejected", name)
    else:
        raise SystemExit("FAIL: accepted " + name)
print("PASS: accepted the valid synthetic answer")
PY

Expect six rejection messages and one acceptance message. These are tests of your Python gate, not evidence that Gemini produces any of these responses. The fixed expected answer makes this a small regression exercise; a real log processor would need an independent way to derive its expected facts.

Capture one headless request

Record the installed interface before running the request:

gemini --version > "$lab_root/evidence/cli-version.txt"
gemini --help > "$lab_root/evidence/cli-help.txt"

Check that help lists --prompt and --output-format. Google’s configuration reference documents these options. If your installation lacks them, resolve that mismatch first.

The automation guide documents supplying input through stdin alongside a prompt. Here we use input redirection and separate output files:

if gemini --prompt 'Use only the supplied fictional log. Do not use tools. Return only a JSON object with failed_count (integer) and failed_checks (array of exact names whose status is FAIL). No Markdown or other fields.' \
  --output-format json \
  < checks.txt \
  > "$lab_root/evidence/raw.json" \
  2> "$lab_root/evidence/stderr.txt"; then
  cli_status=0
else
  cli_status=$?
fi
printf '%s\n' "$cli_status" > "$lab_root/evidence/exit-status.txt"

The if preserves the failed command’s exit status even in a shell using set -e. Keeping stderr separate prevents diagnostics from being mixed into the JSON file. Avoid piping directly into a parser for this first run: you want the original output and command status available when something fails.

Keep your normal permission settings. The no-tools request is an instruction, not an access restriction; headless mode does not itself isolate your filesystem. If the request needs an unexpected permission, investigate instead of enabling automatic approval to get past the failure.

Accept the answer only after all checks pass

if [ "$cli_status" -eq 0 ]; then
  if python3 validate.py "$lab_root/evidence/raw.json" \
    > "$lab_root/evidence/accepted.json" \
    2> "$lab_root/evidence/validation-error.txt"; then
    printf 'PASS: accepted.json matches this fixture.\n'
  else
    printf 'REJECT: inspect validation-error.txt and raw.json.\n'
  fi
else
  printf 'REJECT: CLI exited with status %s; inspect stderr.txt.\n' "$cli_status"
fi

Do not treat the existence of accepted.json as success: shell redirection can create an empty file before validation fails. Only the successful validation branch authorizes a downstream step. This exercise stops at printing a message.

SymptomWhat to inspect next
Nonzero CLI exitRead the saved status and stderr; resolve authentication, arguments or service errors before another attempt.
Invalid outer JSONInspect the raw bytes for empty or truncated output, diagnostics, or the wrong output format.
Outer JSON parses but answer failsCheck error, then inspect whether response is plain text, fenced JSON, or a different shape.
Valid shape, wrong count or namesRecord a factual failure. Do not weaken the expected answer to accept it.

Keep each retry in a new evidence directory so you retain the first failure. Google’s headless reference distinguishes json from stream-json, which emits separate events. This validator is only for the single-object format; switching formats requires a different consumer.

For your next task, write the expected facts and failure behavior before asking the agent to produce an answer. The acceptance-check lesson helps you choose what to test. Preserve raw evidence, and let a passing gate authorize only the specific action it actually checks.

Sources checked October 30, 2026

Put this to work

Build an acceptance gate for a small headless agent task.

Try

Ask Gemini to report the failed checks in a fictional release log.

Prove it worked

Reject malformed output and an incorrect count using synthetic fixtures.

Keep in view

  • Keep stdout, stderr and the command exit status as separate evidence.
  • Distinguish the CLI JSON envelope from the model's answer inside it.
  • Test rejection paths locally before running a model request.
Learn the workflow: Define concrete acceptance checks for AI work