Which model and which vendor fit the workflow?
The decision should not begin with a favorite model. Start with the task, the data involved, the required quality, response time, budget, control requirements and available infrastructure. This guide is about choosing a model and its vendor. If your question is where the model should run, read local AI vs. cloud AI.
An open-weight model is one whose trained weights you can download and run yourself. A hosted API, such as OpenAI's, is a model you reach over the internet under the vendor's terms. The table compares the two on the points that decide most projects.
| Factor | Hosted API (for example OpenAI) | Open-weight model you run |
|---|---|---|
| Capability | The vendor's most capable models, updated by the vendor | Strong on focused tasks with your documents; may trail the largest hosted models on open-ended work |
| Data terms | Set by the vendor's contract and product settings; check the current terms | Set by you, because the model runs inside your environment |
| Cost at volume | Grows with usage | Mostly fixed after hardware and setup |
| Lock-in and change | The vendor can retire or change a model version | You choose when to change; the model file stays yours |
| Maintenance | Handled by the vendor | Handled by you or a support partner |
| Customization | Prompts, retrieval and, for some products, fine-tuning | Prompts, retrieval and fine-tuning on your own hardware |
| Works offline | No | Yes |
When might OpenAI or another hosted API fit?
Commercial cloud models can be a strong choice when you need high capability, rapid deployment, broad language support, advanced reasoning and managed infrastructure.
When might an open-weight model fit?
Local models can make sense when you require infrastructure control, offline operation, specialized deployment, or a design that keeps certain processing inside a controlled environment. Running a model yourself also means owning its hardware, updates and monitoring. For more on that trade-off, see local AI vs. cloud AI, or read how our private AI work is set up.
What does an open-weight licence allow?
Open-weight does not always mean open source. Licences differ on commercial use, on size or revenue thresholds and on what you may do with the model's outputs. Before you build on a specific model, read its licence for your use case, and keep a note of which version you deployed.
How do data and security compare?
Local does not automatically mean secure, and cloud does not automatically mean insecure. Evaluate data classification, identity, permissions, retention, encryption, contractual controls, monitoring and where information is processed.
Be careful with claims in both directions. OpenAI states that inputs and outputs from its business offerings and API are not used to train models by default, and it describes its controls on its business data page. Businesses still need to evaluate retention, permissions, data classification, contracts and the architecture that suits their use case, and to check the current terms for the specific product they plan to use. The NIST AI Risk Management Framework offers a neutral vocabulary for that review.
What should you ask a model vendor?
- Is our data used for training by default, and how do we change that?
- How long are inputs and outputs retained, and can we configure that?
- Where is the data processed, and by which sub-processors?
- What happens when the model version we use is retired?
- Can we get a data processing agreement, and what does it cover?
- What does it cost at our expected volume, and how is it billed?
What is a short checklist for the decision?
- What kind of data will the workflow send to the model?
- Is any of it confidential, regulated or covered by a contract that limits where it can go?
- How capable does the model need to be for this task?
- Does the work need to run offline?
- Who would run and maintain a local model?
- What does each option cost to operate at your expected volume?
How do you compare them on one real workflow?
Abstract comparisons only go so far. Pick one actual workflow, such as drafting replies to customer emails, and write down how each option would handle it: what data goes where, who reviews the output, what it would cost at your volume, and who would look after it. Doing this for a single task usually makes the right answer clearer than any general comparison, and it shows which requirements are real and which are assumed.
For our own inbox we chose an open-weight model on our own hardware, described in a private email assistant that only drafts replies. That was our answer for our data and volume, not a rule for yours.
How do you test a model before you commit?
Build a small test set of real examples from the workflow, with sensitive details removed. Run the same examples through each candidate, and score every output against what a correct result looks like. Compare the outputs side by side, note which mistakes are minor and which would matter, and calculate the cost at your expected volume. Repeat the test whenever the vendor changes the model version or you change the model you run.
What does not change whichever you choose?
Some good practices apply to both options. Give the system access only to the data the job needs. Keep a person approving anything that goes to a customer or changes an important record. Keep a log of what the system did. Test it on your own documents and cases before relying on it. These controls matter more to the outcome than the logo on the model.
Can you use both?
A business may use a local model for one workflow and a cloud model for another, or route requests based on sensitivity and capability requirements. Hybrid design can provide flexibility without forcing every task into the same architecture.
It also keeps the choice reversible. If a workflow is built so the model can be swapped, you can start with whichever option is quickest and revisit it when requirements change. What an AI agent is explains why the model is only one part of the workflow.
We can evaluate your workflow and recommend the architecture that best balances capability, cost, security and operational complexity. See our AI integration services, or start with AI automation services if you are still deciding what to automate.