Industry analysis · October 22, 2026
Gemini 4 Argan: what one heavy tester actually found
Creator-reported Gemini 4 Argan tests: intro price doubles Dec 1, weak cache hits, and mostly false code-audit claims.

Treat Gemini 4 Argan as a Google-ecosystem data model with decent frontend drafts, not a default coding agent. Creator-reported code audits show majority-false findings. Keep a stronger review model in the loop, and budget for cache misses plus the Dec 1 price step-up.
These findings come from one creator’s YouTube testing, not from Google’s documentation. The source is So I’ve been testing Gemini 4…. Names follow the auto-transcript (“Argan,” and peer nicknames such as Soul, Muse Wonderful, Fable, and Astra). Where the transcript is ambiguous, this page says so. Superbash did not re-run the private benches.
Gemini 4 Argan is interesting as a Google-ecosystem model: YouTube data access, multimodal habits, and frontends that can look acceptable when a design skill is on. It is disappointing as a default coding agent in that video’s real-repo work. List price looks friendly until token hunger, weak cache hits, and a December 1 step-up are included. Treat every codebase claim as untrusted until a stronger review model, or a human, verifies it.
Top findings at a glance
| Area | Claim in the video | Evidence type | Confidence for readers |
|---|---|---|---|
| Release gap | About 8 months since the last large Gemini Pro-class drop; Flash-only in between | Creator narrative | Medium. A calendar claim, not independently checked here |
| List price | Intro $2 in / $10 out per million, then $4 / $20 from Dec 1 | Creator reading Google’s pricing UI | Medium. Verify live pricing before you budget |
| Effective cost | Token-hungry, with about 70 to 80% cache hits versus 99%+ on peers | Creator telemetry (Antigravity / T3 Code) | Medium-low. Possible client or reporting bug |
| AA intelligence | Above Astra and GPT-6.1 Soul; below Opus 5.5 High and above; worse dollars per score than Opus High | Artificial Analysis charts shown on camera | Medium. A third-party index, not coding-specific |
| Skatebench | Perfect score; Opus 5.5 Max about 79% | Creator private bench | High for that niche task only |
| Code audits | Scores about 2.25 to 4.4 / 10; majority false claims on several runs | Creator private panel plus judges | High as that tester’s experience, not a public leaderboard |
| Unique keep | YouTube transcripts and Google data APIs, not Gmail quality | Creator production use | Medium. Workflow-specific |
| Ship rule | Will not merge Gemini-authored PRs; plans to auto-close them | Explicit closing position | Opinion. Useful as a process warning |
What was tested
The creator reports early access through Google contacts, then public availability through:
- Developer and enterprise: API, Studio, and the Gemini platform
- Google AI Ultra: Antigravity
- Ultra and Pro: the Gemini app and AI Mode in Search
Work in the video spans marketing-page frontend generation (with and without Anthropic’s design skill), a recurring Fishslop 3D game prompt used across models, a private Skatebench for naming skateboard tricks, Artificial Analysis charts, and several real codebase audits. Those audits include a Rust rewrite of a TypeScript-related compiler stack, T3 Code bugs, a cloud product (“Lakebed”), and an architecture pass, plus a live SQLite connectivity “fix” that became a closed pull request.
Google marketing claims shown on stream (VL Index number one, “100% success in evals,” Chrome refactors, Deloitte-style quotes) are claims to discount, not Superbash-verified facts.
Pricing: the sticker versus the bill
The video’s core cost argument has four parts:
- The sticker looks cheap. Roughly $2 / $10 per million input and output during an introductory window.
- The sticker rises. The same creator reads a step-up to $4 / $20 starting December 1, framed as a pattern Google has used on recent drops: reasonable now, double later.
- Efficiency is poor. On Artificial Analysis-style cost curves, Argan is not a bargain once token volume is counted. The creator places it more expensive than Opus 5.5 on High despite a lower intelligence score.
- Cache misses dominate. Reported cache-read hit rates sit around 70 to 80%, versus 99% or better on models they use daily. A large advertised discount on cache reads does not help when the hit misses and you pay full input more often.
“I’m talking like 70 to 80% cash hit versus other models where I see 99% plus.”
They flag that this might be an Antigravity reporting bug or an MCP / T3 Code integration quirk, and they explicitly refuse to debug it further (“Google should be the ones doing this”). For Superbash readers: instrument your own cache-hit percent before you believe any dollars-per-million comparison.
Practical budget checklist
- Price the post-December 1 rates if you are building a long-lived agent path.
- Log prompt tokens, completion tokens, cache reads, retries, and tool loops for one representative job.
- Compare dollars per accepted result, not the list price per million tokens.
Intelligence indexes versus “does it lie about my repo?”
Artificial Analysis
On the Artificial Analysis intelligence index as shown in the video, Gemini 4 Argan sits above Astra and GPT-6.1 Soul, and meaningfully below Opus 5.5 at High, XHigh, and Max. The creator treats that index as a coarse intelligence signal, not a coding ranking.
On the hallucination or “omniscience” chart, Argan is described as having about a 15% hallucination rate, the lowest among models scoring over 45 on the intelligence index, yet fourth overall once refusals and incomplete answers are scored. A low “wrong” rate can mean “I will not answer,” not “I know your codebase.”
Private code-audit panel
This is where the video’s tone hardens. The creator audits repos by collecting model claims, validating them, and scoring with a judge panel:
| Task (as described) | Gemini 4 Argan | Stronger peer cited | Notable failure mode |
|---|---|---|---|
| TS / Rust rewrite audit | 4.38 / 10; 24 of 36 claims false; about 43 min; about $3 | GPT-6.1 Soul 8.84 (38 true / 0 false, about 28 min, about $4); Opus 5.5 about 8.78 | Majority of claims wrong |
| Real T3 Code bug audit | Called too weak to justify; about 14 / 40 in the wrong class of errors versus Muse | Opus 5.5 best (about 8.22); Soul cheap and clean (about $1, 10 min) | Distinct behavior, not useful accuracy |
| Lakebed cloud product audit | 2.25 / 10; 56 of 78 findings wrong (about 70%) | Soul 9 / 10; Sonnet/Opus-class about 8.5 | “More likely to gaslight your agents” |
| T3 architecture pass | 3.95 / 10; 31 true / 14 partial / 47 false | Muse Wonderful Full 29 true / 8 partial / 4 false | More false findings than Muse had true ones |
“Of its 36 claims, 24 were false. … when this model is doing real work in a real codebase, it’s as likely, if not more likely, to be wrong than right.”
That is the operational takeaway for Hermes and other agent stacks: do not let a single frontier model both author and certify changes. Pair Argan drafts, if you use them at all, with Soul- or Opus-class review, or with a frozen human checklist. The comparison habit is the one in evaluating closed frontier models.
The SQLite pull request as a worked example
On a known SQLite connectivity instability in T3 Code, Argan:
- Spent a long synchronous “read every file” hunt, which the creator treats as a familiar Gemini habit.
- Roughly identified a read-pool / single-writer shape, then built a roughly 1,200-line pull request that should have been on the order of about 30 lines.
- Ignored the repo’s pull-request formatting skill and wrote a wall of summary instead of a short problem statement.
- Failed CI and lint, and did not remediate review-bot comments.
- Missed existing pull requests that already fixed the issue, and invented incorrect routing (described as a regex-based approach).
Opus 5.5, given the thread and asked for an honest take, agreed it was slop, closed the pull request, and landed an about 21-line fix. The diagnosis was partly right. The implementation was wrong and oversized.
Frontend, 3D, and the Google-only path
Frontend
With the Anthropic design skill enabled, Argan produced marketing pages the creator would not immediately clock as AI, including a functional interactive section. With skills off, or with a “taste” skill they dislike, results slid into heavy top navigation, font noise, and layouts they called cringe.
They still rank Fable 5.1 highest for distinctive marketing and homepage aesthetics, and they stress that iteration latency beats raw token speed for real product work. Anecdotally, Argan can stream tokens quickly without finishing hard problems quickly.
Fishslop, the 3D game bench
Same prompt and tools as the other models. The first pass completed a five-tank loop with acceptable fish motion, a broken water surface, and a lethal food mechanic: food never expires on the tank floor, so the fish starve while staring at it. The second pass, after feedback, looked better and feeding improved, but a powerup expired before monsters spawned. The creator’s phrase was “no big model smell”: local code that looks fine without modeling the system.
The lesson for agents: a graphics or 3D fine-tune does not prove general engineering judgment.
Skatebench and multimodal work
Argan took a perfect score on the creator’s private skate-trick naming bench. Opus 5.5 Max scored about 79%, and an earlier Gemini Flash was already very strong. Answers were fast, with little overthinking. The creator uses that result to puncture Google’s “100% of evals” quotes: success depends entirely on what you evaluate.
Vision, documents, video, and audio remain historical Gemini strengths. This video did not re-benchmark them for Argan. If your workload is motherboard labeling, PDF grounding, or something similar, run your own trial.
YouTube and Google apps
Gemini models get generous YouTube and Google-data capabilities. The creator calls the Gemini API one of the best current ways to fetch YouTube transcripts, and will keep using it for that. Separately, they prefer Grok Bot or ChatGPT over Gemini inside Gmail and other Google apps.
That split maps cleanly onto Superbash Learn: route by capability, not by brand. A Hermes skill for YouTube transcripts can stay on Gemini while coding skills pin Opus or Soul.
Antigravity, accounts, and agent plumbing
Non-capability pain still matters for agent operations:
- Account blast radius. Gemini auth is a Google account. In the creator’s risk model, a ban (even if Google promises product-only bans) can threaten YouTube, Gmail, billing, and Chrome passwords.
- ACP and Antigravity quirks. Effort levels are paired into model slugs, so “Argan high,” “medium,” and “low” are picked as if they were separate models. The gateway can disconnect mid-run after long file reads. The creator blames a large share of T3 Code bugs on the Antigravity integration.
For Hermes and other multi-agent setups: isolate Google credentials, prefer reversible deny lists, and do not let an unstable gateway be the only long-horizon runner without checkpoints.
How to try this yourself
- Pin the job. One repo, one question (“list true defects with file and line evidence”), and a fixed timeout and budget.
- Run Argan once. Save every claim as structured data: the claim, the path, and an evidence quote.
- Second-pass with Soul or Opus. Mark each claim true, partial, or false. Do not let Argan grade itself.
- Optional frontend lane. Generate a marketing page with Anthropic’s design skill on, regenerate with skills off, and compare.
- Cost telemetry. Record cache hits and total spend. Re-price the same job at December 1 rates.
- Keep YouTube on Gemini if transcripts are the win. That is the clearest capability the creator still trusts.
If you already use a transcript-to-article loop, you are exercising the same capability the creator still trusts.
Caveats before you quote this
- One YouTuber’s stack. Heavy Antigravity and T3 Code usage. Results may differ in Cursor, Claude Code, or a raw API with different tools.
- Not official Google docs. Pricing dates, VL Index leadership, and “100% eval” quotes are as presented on stream.
- Transcript noise. “Argan,” “cash” for cache, peer model nicknames, and product names may differ from marketing strings. Verify them against Google’s live model card before you hard-code IDs.
- A rumor of “Carbon.” The creator mentions hearing about a better upcoming snapshot. Unverified here.
- Opinions are not Superbash measurements. We report their numbers. We did not re-run the private benches.
FAQ
Is Gemini 4 Argan the best-value frontier model?
Not according to this tester, once token use and cache behavior are included. The intro list price looks cheap. Effective cost and code-audit quality do not.
Should I use it for agentic coding?
Only behind a stronger review model or a human merge gate. The video’s private audits show majority-false claim rates on several real repos, and a large broken pull request that a stronger model rewrote in tens of lines.
What is it still good for?
YouTube and Google-data pulls, trials of complex vision tasks (a historical strength, not re-proven in this video), and frontend drafts with a solid design skill. Iterate with a faster or more tasteful model if the draft needs it.
Why do hallucination charts look fine if code audits look bad?
The definitions differ. Refusal-heavy behavior can look like “low hallucination” on knowledge benches while the same model invents confident, wrong repository claims under tool use.
What should operators change next?
Route YouTube transcript skills to Gemini if that path is winning. Pin coding and pull-request authorship to models that survive your claim-validation loop. Log cache hits. Budget the December price step-up. Do not auto-merge Gemini-authored pull requests without a second judge.
Sources
- So I’ve been testing Gemini 4…, the creator testing video this page digests. Figures, prices, and bench scores are reported from that video and its auto-transcript.
- Not a Google model card, pricing page, or launch post. Check live Google pricing and model IDs before production use.
- Evaluating closed frontier models, the related Superbash lesson on choosing a model by the job.
Put this to work
Separate marketing evals and hallucination indexes from whether a model's claims about your codebase survive a second-model review.
Try
Run one bounded repo audit with Gemini 4 Argan, then re-judge every claim with Opus- or Soul-class review before any merge.
Prove it worked
Track false-claim rate, cache-hit percent, wall-clock to a green PR, and total dollars, not list price alone.
Keep in view
- Intro list price looks cheap. Dec 1 doubles it, and weak cache hits erase much of the discount.
- Artificial Analysis places Argan below Opus 5.5 on intelligence-for-cost once token use is counted.
- Private code-audit benches in the source video show majority-false claims on real repos.
- Keep Gemini in the stack for YouTube transcripts and vision trials. Do not merge unvetted agent PRs.