Local or external AI: the decision starts with the process
Open models can be used in hosted services, while local tools can run models managed by the company itself. This distinction matters for internal documents, operational information and applications that require direct control over the environment.
But where the model runs doesn't, on its own, define the data's path. Integrations, databases, updates and connected services can send information out. The decision needs to follow the complete process, from first access to recovery in case of failure.
Start with the data flow
In a vendor-managed service, part of the updates, availability and maintenance stays outside the company. When it adopts self-managed execution, the company becomes responsible for hardware, access, security, compatibility, updates, failures and recovery capacity.
That doesn't make self-managed execution automatically safer. It offers direct control, but that control only has value when there is a team, a process and a budget to run it. A local outage can affect a critical routine as much as a change at the external vendor.
Compare the cost of keeping the process running
Managed services usually come with a subscription or usage-based pricing. In self-managed execution, you add hardware, power, storage, maintenance, updates and the time of the team in charge. The right comparison is the total cost of keeping the process running, including the cost of fixing failures and replacing components.
A small or rarely used application may still make more sense in a contracted service. Predictable volume, high control requirements or restrictions on data movement can justify evaluating local. In many cases, the two alternatives can coexist depending on the application.
Define the exit too
Before choosing, describe what information will be processed, where it can flow, what volume is expected, what availability the process requires, who will handle maintenance and what cost fits the result. Also record how configurations, data and flows will be migrated if the solution needs to be replaced.
This diagnosis may lead to a hybrid architecture. The choice gets sturdier when infrastructure comes after the operational need, with security, continuity and cost criteria that can be checked over time.
Compare the complete flow
To decide, record where data enters, who accesses it, which integrations are needed, which security controls remain the company's responsibility and what alternative exists in case of an outage. Where it runs is one criterion, not the whole decision.
Sources
Ollama. https://docs.ollama.com/faq
Ollama. https://docs.ollama.com/cloud
National Institute of Standards and Technology (NIST), Sep 2011. https://csrc.nist.gov/pubs/sp/800/145/final
National Institute of Standards and Technology (NIST), Jul 2024. https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence