Choose where your AI model should run

Last checked and updated: September 8, 2026. Editorial and source review; exercises below are authored practice, not reported benchmark runs.

A model can perform well on your task and still be inconvenient to operate. This lesson turns your comparison sheet into a decision about access and hosting.

You will write a short decision record. You do not need to deploy anything to finish it.

Separate three decisions

Quality: Can this model produce an acceptable result on your task?

Access: Will you use a chat app, an API, or downloaded model weights? An API is a way for software to request a model response. Weights are the files that contain a trained model’s parameters.

Operation: Who runs the service, handles updates, and fixes failures?

“Frontier” describes a position near the leading edge of capability; “open-weight” describes access to model weights. They are not opposite ends of a quality scale. An open-weight model may be available through a hosted API as well as for self-managed deployment.

Compare operating arrangements

Arrangement	What you take on	Check before choosing
Hosted chat app	Supplying context and reviewing results	Available tools, data policy, limits, and whether you can export your work
Hosted model API	Integration, request handling, usage monitoring, and review	Model version, endpoint behavior, data policy, and observed task cost
Self-managed model	Hardware, runtime, updates, access controls, monitoring, and review	License, hardware fit, output quality on your setup, and ongoing operating effort

Use the arrangement that fits the work and the time you can spend maintaining it. Downloadable weights do not guarantee that a model will run usefully on your laptop, and they do not make compute or maintenance free.

Trace where your data goes

Draw the actual route: input file, app, model endpoint, tools, logs, and output. Check each service that receives the data. A locally running model can still be surrounded by tools that send requests elsewhere.

For example, Hugging Face documents a local-files option for using cached downloads. That controls the download behavior described in those docs; it does not establish that every component of your application is offline. Download and cache documentation.

For work governed by an employer or client agreement, use their allowed services and data-handling requirements. Do not infer permission from the model’s price or license label.

Write the decision record

Use a real task, or keep the supplied-note update task from the previous lesson:

Task and expected output:
Quality checks and observed results:
Chosen model, version, and access route:
Where inputs, tool requests, and logs go:
Who handles setup, updates, and failures:
Observed cost or limits; unknown costs:
Why this is a better fit than the alternative:
When to reconsider:

Keep unknowns visible. If you have not measured latency on your hardware, write “unmeasured” rather than treating a provider demonstration as your own result.

Change one thing when the workflow fails

If a result misses a source fact, inspect the input and retrieval first. If a tool fails, check access and the tool response. If the model repeatedly fails a well-specified check, compare another model on the same input.

Changing the model, prompt, tools, and input together makes it hard to learn what helped. Save the failing example and make one deliberate change at a time.

You are finished when your record explains both why the model passes your quality bar and why its operating arrangement fits your situation. Return to the model-choice path to review your two deliverables. Use the model-family references when you need candidates for the next comparison.