What does the assistant do?
The Credantic mailbox receives enquiries, and many of them call for a similar first reply. We built a small assistant that reads the company inbox over IMAP, sorts new messages, and writes a branded, threaded draft reply for a person to review. It also prepares a daily digest draft that summarizes what arrived.
It never sends email. A person reads each draft and decides whether to edit it, send it or throw it away.
Why did we build it ourselves?
Two reasons. The inbox holds messages from people we do not know, and we wanted that text processed on our own hardware and not on a third-party service. We also wanted first-hand experience of what a drafts-only design needs before recommending the approach to anyone else.
It is an internal tool, so this write-up is not evidence of how it would perform in your business. We describe the design, and we do not publish performance numbers because we have not measured any.
How does it work?
- Read. It connects to the mailbox over IMAP and picks up new messages.
- Classify. It decides what kind of message each one is, such as a service enquiry or something that needs no reply.
- Draft. For messages that call for a reply, a local open-weight language model writes a draft in our brand voice, threaded under the original message. The model is Llama 3.1 8B, run through Ollama on our own hardware with a single consumer GPU.
- Hand over. The draft goes to a person, who edits, sends or discards it.
- Digest. Once a day it prepares a digest draft of new mail.
Llama 3.1 is a family of open-weight models, and the 8B size is small enough to run on one consumer GPU. You can see the family on the Ollama model library. Running the model locally means the text of the messages is processed on our own machine. For the wider trade-offs, read local AI vs. cloud AI.
Why does it never send email?
There is no sending code. This is not a switch that is turned off: the code that would send a message does not exist in the system. A bug, a badly written prompt or a hostile message cannot make it send mail, because it has no way to.
The cost of this choice is that a person has to read every draft. That is the design. The assistant removes the typing and the lookup, and a person keeps the decision, in line with the approval pattern we describe in what an AI agent is.
Why a small local model?
Drafting a short, polite reply from a few facts is a narrow job, and a small model keeps the hardware requirement modest: one consumer GPU, not a rack of servers. The trade-off is capability. A small model is more likely to miss nuance than the largest hosted models, which is one more reason a person reads every draft. If a task needed more capability, the model is a replaceable part, and the rest of the design would not change.
How does it treat untrusted email?
Every incoming message was written by someone we do not control, and a message can contain text meant to steer a model. That attack is called prompt injection. The assistant is built on the assumption that any email may be hostile:
- The message text is kept inside delimiters in the prompt, as material to read and not instructions to follow.
- The model has no tools. It cannot browse, run commands or call other systems.
- Its output is parsed into a fixed shape, and anything that does not fit is discarded.
- The only thing the output can ever become is a draft addressed to the original sender.
These steps reduce the risk. They do not remove it, which is why a person still reads every draft before it goes anywhere.
What does it log?
Logs record that a message was processed and what the system did with it. They never contain message bodies. If a log file were ever exposed, it should not contain anyone's correspondence.
What are its limits?
- An 8B model is small next to the largest hosted models. Drafts need editing, and some will be discarded.
- Classification can be wrong, so a person still looks over the inbox.
- It is only as good as the brand-voice instructions and reference text it is given.
- We have not measured accuracy or time saved, so we do not quote either.
What can you take from this design?
- Remove the capability you do not want. If an assistant should not send, give it no way to send.
- Treat every input as untrusted, and keep it separate from instructions.
- Give the model no tools unless the job needs them.
- Keep message content out of logs.
- Make a person the final step.
These ideas are not specific to email. The NIST AI Risk Management Framework gives a broader vocabulary for the risks involved.
What would a client version look like?
A client project would be scoped separately, not copied from ours. It would start with one workflow, use your mailbox and your brand voice, and size the hardware in writing before you buy anything. The drafts-only principle would carry over. Whether it should send anything on its own is a decision for your risk appetite, and our default is that it should not.
Could the same approach work for your inbox?
Possibly. Our AI automation services page explains how we scope this kind of work, and AI integration services covers connecting a model to mail, a CRM and documents. For models that run on your own hardware, see private AI. The first step is a single workflow and an honest look at whether a drafts-only design is enough.