Local AI for SMEs: your own language models

Local AI from Fox Computers starts from CHF 7,500 for model selection, setup, connecting your documents and access for your team, excluding hardware. Hardware comes on top and depends heavily on the model: a workstation with a powerful graphics card is enough for many tasks, while a server for large models can quickly cost as much as a small car. We clarify beforehand what you really need and whether Swiss hosting is the cheaper option. After setup there are no fees per request, only electricity, maintenance and, where applicable, hosting.

For many tasks, a powerful computer with a modern graphics card or a Mac with Apple Silicon and plenty of memory is enough. What matters most is the graphics card’s memory: models with 7 to 14 billion parameters run smoothly with 16 to 24 GB, while models with around 70 billion parameters need considerably more, usually server graphics cards or several cards. The number of people asking questions at the same time also plays a role. Fox Computers sizes model and hardware so that performance and cost match, and first checks whether your existing devices are sufficient.

With ChatGPT, your inputs go through the provider’s servers, often in the USA; with local AI, they stay on your server or with a Swiss host. That’s the core of it: client files, contracts or patient data don’t leave your environment. There are other differences too. Local AI costs a one-off amount for setup and hardware, after which there are no fees per request. You are independent of price and contract changes by big providers. On the other hand, the largest cloud models are often still stronger at very complex tasks. Fox Computers shows you which tasks run well locally.

Open language models such as Qwen, Mistral, Llama or Gemma run locally; Fox Computers usually deploys them on your server via Ollama. Which model fits depends on the task: summaries and e-mails in German call for different models than extracting tables or programming. We test several candidates with questions from your day-to-day work and choose the model that gives the best answers on your hardware. Because models evolve quickly, we build the solution so that switching to a newer model is possible without a rebuild.

For many everyday tasks, yes: current open models reliably handle summarising, drafting e-mails, searching documents and extracting information. For very complex analyses, long chains of reasoning or demanding programming, the largest cloud models from OpenAI or Anthropic often still have the edge. But the gap has narrowed considerably in recent years. Before you decide, Fox Computers tests with real questions from your business, so you can judge the quality yourself. Where necessary, we combine the two: sensitive data stays with the local model, non-critical tasks go to a cloud model.

Local AI makes compliance with the revFADP and the GDPR much easier, because no personal data is transferred to external AI providers or abroad. This is especially valuable for industries bound by professional secrecy, such as fiduciary services, law firms, medical practices or recruitment. You can prove where the data is processed and who has access. Fox Computers sets up access rights, logging and storage locations properly. Other obligations remain, such as informing people in your privacy policy. We show you openly what is solved technically and what you should take care of organisationally.

Yes, Fox Computers connects the local model to your documents, policies and templates, so it answers questions from your own knowledge. Technically, this works through a search (RAG) that finds the relevant passages for each question and passes them to the model. The model answers with a source reference, so you can check every statement in the original document. Things like folders on the server, PDF manuals, quotes, reports or minutes can be connected. Access rights can be set so that each person only sees what they are otherwise allowed to see. Everything stays within your environment.

Both are possible: your own server offers maximum control, while a dedicated server with a Swiss host saves on purchase costs and office space. On your own hardware, the data never leaves your building, and the model doesn’t even need an internet connection. In return, you bear the purchase cost and need a suitable location with enough cooling. With Swiss hosting, you rent the computing power, the data stays in Switzerland, and you can upgrade when needed. For large models in particular, this is often the cheaper option. Fox Computers calculates both options for your case.

Setting up local AI at Fox Computers takes three to six weeks, from choosing the model to giving your team access. First, we clarify which tasks the AI should take on and which data it needs. That determines the model and hardware. Then we install the system, connect your documents and test with real everyday questions until the answers are right. If new hardware has to be bought, the schedule also depends on its delivery time. After launch, we fine-tune based on the questions your team actually asks.

If you wish, Fox Computers takes care of operation and updates: we keep the model, software and server up to date and monitor that everything runs reliably. New model versions come out all the time, and switching often brings noticeably better answers or more speed on the same hardware. We test such updates with your typical questions first before rolling them out. On top of that come security updates for the operating system and software, and adding new documents. We bill the effort at CHF 160 per hour or agree on fixed support in the quote. That way, your local AI stays up to date.