Where should an AI model run?

Where does your text go when you ask an AI assistant a question? A computer runs a model on it and sends an answer back. Who controls that computer affects who can see your text, what you pay, and whether last month's model is still answering today.

Consider a fictional neighborhood book swap, written for this guide. It wants help with 3 jobs: drafting its weekly newsletter, answering member questions about the rules, and sorting a sign-up sheet that includes members' home addresses.

Use the maker's app, or have your software call its API: a published way for programs to exchange requests. The model runs on the maker's computers. You get its newest versions without any setup, and you agree to its terms for your data.

A cloud platform gives you models from many makers through one account. Amazon, Google, and Microsoft each offer one. Requests go to the cloud company's computers under your existing contract with it.

You can also run a model yourself. Some makers publish its weights, the large file of learned numbers that turns input into output. Download it to a laptop, office server, or rented machine. Nothing leaves unless you send it.

The book swap's newsletter could go any of the 3 ways. The sign-up sheet with home addresses is where the choice starts to matter.

A model whose weights you can download is called open-weight. Open source asks for more. The Open Source Initiative's definition (version 1.0) asks for the freedom to use, study, modify, and share the system. For that, it requires the weights plus enough information about the training data to build something substantially equivalent, and the code used to train and run it. A model that releases its weights without the data information and code does not meet it.

Each open-weight model has its own license terms. On Hugging Face, a widely used public library of models, each model has a model card: a page describing intended uses, known limits, training data, and a license field. Read the license before you build on the model.

In a July 2024 report on widely available model weights, the US Commerce Department's NTIA listed benefits, such as letting people use a model without sharing data with a third party, and risks, such as misuse with little oversight. It recommended monitoring the evidence rather than restricting open weights at that time.

Follow one member question, "Can I bring 6 books?", along each path.

Sent to the maker's API, the question and any rules you attach go to the maker. What happens next depends on that maker's terms: how long it keeps requests, and whether it uses them to improve its models. Read your product's terms; they differ between products.

Sent through a cloud platform, the question goes to the cloud company. Amazon's Bedrock data protection page says models run in accounts the Bedrock team operates, and that model makers have no access to customer prompts or answers there. That is one platform's statement, checked on October 28, 2026. For Google and Microsoft, read their data residency and retention pages before assuming the same.

Run on the book swap's own laptop, the question never leaves the room. The swap must supply enough memory to hold the weights, install updates, and notice when something breaks. Hugging Face's guide to running models locally lists privacy, speed limited only by your hardware, control, and no API bill as the reasons people do it. A larger model needs a larger machine, and a small one may answer rule questions less reliably.

Updates differ too. A hosted model can be retired. Anthropic's deprecation page promises at least 60 days' notice before it retires a public model. After that, requests to the model fail. The same page says Amazon Bedrock and Google-managed copies set their own schedules, so one model can retire on different dates depending on where you call it. Downloaded weights don't retire. They also don't improve unless you download a newer file.

Older articles may use these platforms' earlier names.

• Amazon Bedrock offers what AWS describes as hundreds of models from many companies through one service.
• Gemini Enterprise Agent Platform is Google's platform for models and agents. Google's April 22, 2026 announcement calls it the evolution of Vertex AI, which now includes the Vertex AI services. Google says it offers more than 200 models, including its own Gemini models, open models, and Anthropic's Claude.
• Microsoft Foundry was earlier called Azure AI Studio and then Azure AI Foundry, according to the "Evolution of Foundry" table on Microsoft's overview. Microsoft says it offers more than 10,000 models from Microsoft, OpenAI, Anthropic, Meta, and others.

Those counts are each company's own description. A long catalog matters less than whether the specific model you tested is available in the region, the cloud provider's group of data centers, where your data must stay.

Write down what the model will see. Public newsletter text and home addresses need different answers. Ask 5 questions:

1. Where may this text go? If a rule or a promise to members says it stays on your machines, running the model yourself may be the only fit. If a cloud contract already covers your data, a platform under that contract may be simpler.
2. How much will you send? A few requests a week cost little anywhere. Heavy, steady use can make your own hardware cheaper, once you count the machine and the time to run it. What AI costs shows how per-token prices add up.
3. Who will look after it? A hosted model needs an account and someone watching the bill. A model you run needs someone to install, update, and restart it.
4. Must the answers stay the same? If you need repeatable results, use a fixed version name where the provider offers one, and plan for its retirement date. How to read a model claim explains why a result belongs to one version on one date.
5. How would you leave? Lock-in is the cost of switching later. Prompts and tests written for one provider can often move with you. Agents and settings built with one platform's own tools may need rebuilding. Keep your test questions so you can rerun them elsewhere.

You can also use more than one: a hosted model for public drafts and a local one for private records.

Pick a home for each fictional book swap job, then name one fact that would change your choice.

1. Drafting the weekly newsletter from a list of donated titles.
2. Answering member questions about the swap's posted rules.
3. Sorting the sign-up sheet with home addresses into pickup routes.

One reasonable answer: the newsletter can go to any hosted service, since the text is public anyway. Rule questions also suit a hosted model or platform, but test the exact version you choose and note its retirement date. The address sheet is the case for a local model, or for a platform whose data terms you have read and can show members.

The deciding facts are different for each. If the newsletter grew to thousands of requests a day, cost would decide. If no volunteer could keep a laptop updated, a cloud platform with written data terms might beat a local model nobody maintains. If the swap promised members that addresses stay offline, that promise decides the third job before any comparison of models starts. For how a connected assistant handles private information, see AI privacy and actions.

• Open Source Initiative: The Open Source AI Definition sets out what a model must share to count as open source.
• NTIA: Open model weights report weighs the benefits and risks of downloadable weights. A policy report; its summary page is a good start.
• Hugging Face: Model cards explains what a model card contains and where to find the license.
• Amazon Bedrock: Data protection is one cloud platform's account of who can see prompts and answers. Written for administrators.
• Microsoft: What is Microsoft Foundry? includes the table mapping Foundry's former names to current ones.