Giving a local model the context to work on my website

I wanted a local AI project that could do useful work on something I already maintain: this website. The first version can answer questions about selected portfolio sources, draft an article, and propose a content edit for review. Inference runs on my Mac.

The interesting part was connecting the model to the right evidence and giving it a way to suggest changes without treating its output as permission to act.

The runtime is one layer

The setup uses a MacBook Pro with an M3 Pro and 18 GB of unified memory, Ollama, and Qwen3.5 4B. The downloaded model is approximately 4 GB. The local service uses Ollama's MLX runner on this machine.

The runtime loads the model and generates text. It does not, by itself, know my project history or understand which website files it should change. That requires an application around it.

For this prototype, the surrounding application is a Python service with a small browser workspace and a command-line interface. Its tools are explicit: ask a question, draft an article, or propose one exact text replacement. The model does not autonomously choose commands or run a shell.

What “reading the code” actually means

The knowledge base starts with selected local files: the website README, two portfolio components, Markdown articles, public evidence JSON, and Shipper records. The initial index contained 144 chunks before adding this post and the new project record.

Indexing splits files into 45-line passages starting every 35 lines, preserving the file path and starting line. SQLite FTS5 ranks keyword matches. A question retrieves up to five passages; the application rereads those locations from disk and includes them in the model's prompt.

The flow is:

Question
  → search selected local sources
  → retrieve passages with file paths and line numbers
  → send question and evidence to Ollama on localhost
  → show the answer and retrieved sources

This is retrieval augmented generation. It supplies knowledge at request time rather than changing the model's weights. No fine-tuning was involved.

So yes, the assistant can answer questions using local code and content. Its access currently covers an allowlisted slice of this repository. It does not inspect every file, execute the code to establish behavior, or automatically discover newly added files. Reindexing is required after source updates; an approved local apply also rebuilds the index.

The first answer exposed a retrieval problem

One test question was: “What does Qalib verify, and what do its tests not prove?”

The first retrieval pulled in passages about Shipper and Loqmap because generic words such as “tests” and “prove” matched those records. The model used some of that evidence in its answer about Qalib. Providing sources had not prevented it from mixing projects.

The correction was in retrieval: when a question names one recognized project, results are restricted to file paths containing that project's name. A regression test checks that a Qalib question excludes Shipper evidence.

That restriction is deliberately conservative. It can miss relevant material in shared files, and questions comparing multiple projects still use broader retrieval. It addresses the failure we observed; it does not establish that every future answer will be correct. The revised demonstration retrieved Qalib evidence and kept its component-test results separate from unmeasured model accuracy and production reliability.

Edits are proposals first

For an edit, the application gives the model the selected file and asks for one exact replacement: the old text, the new text, and a reason.

The application then checks the path, requires the old text to occur exactly once, parses JSON files when applicable, and saves a proposal containing the before/after content and a hash of the original. It displays a unified diff. Source files remain unchanged at this stage.

To apply an edit, the operator types its proposal ID. The application checks that the source still matches the saved hash, then writes the change to the isolated local checkout. A stale proposal is rejected. Paths outside the public-content allowlist and symlink targets are rejected too.

Article generation follows the same proposal/review path. New drafts start with published: false. Applying a local edit does not commit, push, or publish it. The website can be previewed locally; publishing remains a separate, explicitly authorized step. This post and its portfolio entry were prepared and published through that separate engineering workflow, not autonomously by the local model.

A runtime detail that needed a fallback

The installed Ollama MLX build could run the model, but its structured-output grammar library was unavailable. Requests for constrained JSON returned HTTP 501.

The edit tool now retries without the runtime's JSON constraint, asks for JSON in the prompt, and validates the returned object and exact replacement itself. That fallback makes output validation more important: generated text is still a proposal, and a malformed or nonmatching edit is rejected.

What was actually verified

The prototype's seven automated tests cover retrieval source locations, separation of proposals from source writes, stale-source rejection, path boundaries, symlinks, malformed JSON, and exclusion of another project's evidence for a single-project question.

Manual runs demonstrated a grounded question, an article draft, and an exact homepage edit proposal. The browser displayed the proposed diff and local-approval controls. The website's static production build passed.

These are software checks and workflow demonstrations. They are not a benchmark of answer accuracy, a comprehensive prompt-injection evaluation, or evidence of production reliability. The application still needs human review of claims and generated changes. It does not automatically verify TSX semantics, run a deployment, or audit links.

The project entry records the architecture and verification scope. A machine-readable verification record keeps the tested behaviors and limitations together.

Trying the same pattern

With Ollama installed and its service running, the model itself can be tried with:

ollama pull qwen3.5:4b
ollama run qwen3.5:4b

Those commands run the model. To give it application-specific context, the next pieces are a deliberately scoped source index, retrieval that returns useful evidence, a call to the local chat API, and tools whose outputs can be checked before applying them. The commands above do not install this project's retrieval or editing application.

The pattern can extend beyond a personal website. A business might connect documentation, records, and approved operations in a similar way. The source permissions, retrieval quality, tool boundaries, and evaluation would need to be designed for that business. This prototype demonstrates that application layer on a small, inspectable workflow.