Private AI, installed in your office, on a machine you may already own.
When data must not leave the company, I install open-source models directly on your servers. This setup is in production. A computer more than five years old, upgraded with 64 GB of RAM and a 12 GB graphics card, is already enough for real-world use.
- Status
- In production at the client's site
- Hardware
- 5+ year-old PC · 64 GB RAM · 12 GB graphics memory
- Software
- LM Studio + open-source models
- Data
- Processed on site, never sent to an outside AI service
Some data must never leave the company.
Client files, patient records, contracts, trade know-how, financials: many businesses give up on AI because they won't hand that information to an online service. In healthcare, law or finance, it is often simply not an option.
They often assume private AI requires prohibitively expensive servers. My tests show the opposite for a large share of use cases: with the right model and a governed method, an existing machine is enough.
From an existing machine to an in-house assistant.
Scroll down: the four stages of an installation, reconstructed. For US clients, everything is done remotely.
We start from what you have.
Memory, graphics card, storage: we measure what the machine can run and pick a model to match. If a purchase makes sense, we size it as tightly as possible.
An open-source model, running locally.
The software and the model are installed once. After that, everything runs on your network, even with no internet connection.
The agent knows only your sources.
Your documents are indexed on the machine. With the governed method, the agent applies your rules and always cites the document its answer comes from.
Your team asks its questions.
From their browser, on the internal network. Answers arrive in a few seconds, with their source.
◆ Your data stays with you$ inventory --hardware cpu 8 cores # 2019 memory 64 GB graphics 12 GB of video memory storage 1 TB SSD → recommended model: 7 to 14 billion parameters, compressed (quantized) to fit on the graphics card → verdict: no purchase needed to get started
$ lms get mistral-nemo ✓ model downloaded once · 7 GB $ lms load mistral-nemo --gpu max ✓ loaded into graphics memory $ lms server start ✓ local server running · internal network only # no data leaves the machine
- ✓Answer only from indexed sources
- ✓Always cite the document and page
- ✓Missing information: say so and refer to a manager
Confidential, under control, no pay-per-use bill.
AI installed on your premises doesn't charge you for every question. Its cost comes down to the machine, the installation and the maintenance.
Your data stays with you
Nothing goes to an AI provider. A simpler answer to the requirements of your clients, your insurers and your industry, and one less vendor in your compliance review.
No usage-based bill
Your team can ask as many questions as it likes. No per-seat subscription, no per-request fee.
Works without internet
The assistant stays available on your internal network, even during an outage or at a remote site.
A limitation, stated plainly: models that run on a desktop machine are smaller than the big online models. So we reserve them for well-defined tasks (searching your documents, drafting, sorting, extraction). That's exactly what the governed method is for: a narrow domain, controlled sources, written rules.
Regulated industries: keeping data on your own hardware removes a third-party AI provider from the picture, which simplifies HIPAA, attorney-client privilege or financial confidentiality questions. It doesn't replace your own security and compliance program: access control, backups and policies remain yours, and we work within them.
Install, govern, maintain
- Audit the machine and choose the model
- Install and secure the server on your network
- Index your documents and write the rules
- Update the models and monitor quality
Own, approve, use
- Keeps ownership of the machine and the data
- Chooses which documents the assistant can access
- Approves the rules and the test answers
- Decides who gets access
+Under the hood for technical teams
- Hardware
- Workstation more than five years old, 64 GB of RAM, graphics card with 12 GB of VRAM. A dedicated server is possible for more users.
- Software
- LM Studio and its local server, compatible with the OpenAI API, so the same tools as online can be plugged in.
- Models
- Compressed (quantized) open-source models, chosen based on available memory and the task.
- Document search
- Local document index (RAG); every answer links back to the document and page.
- Access
- Web interface on the internal network, user accounts, no inbound access from outside.
- Maintenance
- Model updates validated against a set of test questions, index backups.
Does your data need to stay in-house?
Let's talk about your documents, your constraints and your hardware. I'll tell you what on-site AI can do, and with which machine.