docs: record issue #24's smoke-test findings and local litellm-pgvector patches

memory-knowledgebase.md no longer says 'not yet verified' — it's been
smoke-tested end-to-end (direct search + the file_search tool on a chat
completion) and the bugs found are fixed in the preceding commits.
VENDORED.md documents the three local patches on top of the upstream
litellm-pgvector commit so a future re-vendor doesn't silently drop them.

docker-compose.yml

@@ -271,7 +271,11 @@ services:
       - SERVER_API_KEY=${LITELLM_PGVECTOR_API_KEY}
       # Calls back into litellm for embeddings, same pattern as any other
       # workload — see docs/proxy-key-onboarding.md for issuing this key.
-      - EMBEDDING__MODEL=local-embedding
+      # openai/ prefix required — litellm.aembedding can't infer a provider
+      # from a bare model name plus a custom api_base (raises "LLM Provider
+      # NOT provided"), same reasoning as the openai/ prefix on
+      # qwen3.8-27b-local and local-embedding in litellm-config.yaml.
+      - EMBEDDING__MODEL=openai/local-embedding
       - EMBEDDING__BASE_URL=http://litellm:4000
       - EMBEDDING__API_KEY=${LITELLM_PGVECTOR_EMBEDDING_KEY}
       - EMBEDDING__DIMENSIONS=768

docs/memory-knowledgebase.md

@@ -2,7 +2,7 @@
 
 Three gateway-level capabilities added on top of the [AI gateway/proxy](https://git.arthurerlich.de/haylan/LLM-Server/issues/9), so every client behind LiteLLM gets them — not just Open WebUI. See [issue #21](https://git.arthurerlich.de/haylan/LLM-Server/issues/21) for the rationale.
 
-**Not yet verified on real hardware** — see [issue #24](https://git.arthurerlich.de/haylan/LLM-Server/issues/24). In particular: `litellm-pgvector`'s Prisma migrations on first boot, and the exact `vector_store_registry` field names for the `pg_vector` provider.
+**Verified against a live deploy** — see [issue #24](https://git.arthurerlich.de/haylan/LLM-Server/issues/24), closed after smoke-testing found and fixed several bugs: a missing `api_key` in `vector_store_registry` (was silently falling through to the real `api.openai.com`), `litellm-pgvector`'s Prisma schema never actually being pushed to `pgvector-db` (now handled by `./scripts/update.sh`), a 1536- vs 768-dim vector column mismatch, and its create endpoint ignoring any caller-supplied store id (both fixed locally — see `vendor/litellm-pgvector/VENDORED.md`). `scripts/ingest-memory.sh` was also silently broken (posted chunks with no embedding attached) and has been fixed to embed via LiteLLM before inserting.
 
 ## Web search (SearXNG)
 

litellm-config.yaml

@@ -53,16 +53,27 @@ search_tools:
 # Qdrant — LiteLLM's native vector-store feature has no Qdrant provider, see
 # docs/research/litellm-knowledgebase.md). vector_store_id is this proxy's
 # own identifier for the store, not assigned by a backend.
-# ponytail: field names here (custom_llm_provider: pg_vector, api_base
-# pointed at litellm-pgvector) are the best fit from the litellm-pgvector
-# README, not confirmed against a running deploy yet — smoke-test before
-# relying on it. See issue #24.
+# Smoke-tested end-to-end against a running deploy (issue #24): search via
+# both /v1/vector_stores/{id}/search directly and the file_search tool on a
+# chat completion. Needed several fixes beyond this block to work — a
+# missing api_key here, litellm-pgvector's Prisma schema never having been
+# pushed, a 1536- vs 768-dim mismatch, and its create endpoint ignoring any
+# caller-supplied id — see scripts/update.sh, scripts/ingest-memory.sh, and
+# vendor/litellm-pgvector/'s local patches (models.py, main.py,
+# prisma/schema.prisma).
 vector_store_registry:
   - vector_store_name: memory-and-notes
     litellm_params:
       vector_store_id: "memory-and-notes"
       custom_llm_provider: pg_vector
       api_base: http://litellm-pgvector:8000
+      # Required by litellm's pg_vector provider (see
+      # PGVectorStoreConfig.validate_environment in litellm's source) — it's
+      # the Bearer token litellm-pgvector's own API checks against its
+      # SERVER_API_KEY. Was missing entirely, which is why every vector
+      # store call was failing with "Incorrect API key provided: None"
+      # before litellm-pgvector was ever reached. See issue #24.
+      api_key: os.environ/LITELLM_PGVECTOR_API_KEY
       embedding_model: local-embedding
 
 router_settings:

scripts/ingest-memory.sh

@@ -1,7 +1,8 @@
 #!/usr/bin/env bash
 # Loads data/memory.md and data/claude-legacy-memory.md into the LiteLLM
 # knowledgebase (the "memory-and-notes" vector store, see litellm-config.yaml)
-# via litellm-pgvector's batch-embeddings endpoint.
+# via litellm-pgvector's batch-embeddings endpoint. Both files are optional —
+# a file that doesn't exist yet is skipped, not an error.
 #
 # ponytail: one chunk per non-empty, non-heading line — both source files are
 # already one fact/paragraph per line (no hard-wrapping), so this needs no
@@ -15,34 +16,68 @@ cd "$(dirname "$0")/.."
 [ -f .env ] && set -a && . ./.env && set +a
 
 : "${LITELLM_PGVECTOR_API_KEY:?Set LITELLM_PGVECTOR_API_KEY in .env first}"
+: "${LITELLM_PGVECTOR_EMBEDDING_KEY:?Set LITELLM_PGVECTOR_EMBEDDING_KEY in .env first}"
 LITELLM_PGVECTOR_URL="${LITELLM_PGVECTOR_URL:-http://localhost:8000}"
+LITELLM_URL="${LITELLM_URL:-http://localhost:${LITELLM_PORT:-4000}}"
 VECTOR_STORE_ID="memory-and-notes"
 
 # Must match litellm-config.yaml's vector_store_registry entry — the
 # registry just points at a store the backend must already know about.
 # Ignores failure if it already exists (no documented idempotency check).
+# id is a local addition to litellm-pgvector's create endpoint (see
+# vendor/litellm-pgvector/main.py) — without it, create always minted a
+# random UUID and this script's writes could never land on VECTOR_STORE_ID.
 curl -sf -X POST "${LITELLM_PGVECTOR_URL}/v1/vector_stores" \
   -H "Authorization: Bearer ${LITELLM_PGVECTOR_API_KEY}" \
   -H "Content-Type: application/json" \
-  -d "{\"name\": \"${VECTOR_STORE_ID}\"}" > /dev/null 2>&1 || true
+  -d "{\"id\": \"${VECTOR_STORE_ID}\", \"name\": \"${VECTOR_STORE_ID}\"}" > /dev/null 2>&1 || true
 
 ingest_file() {
-  local file="$1" section=""
-  local batch="[]"
+  local file="$1"
+  if [ ! -f "$file" ]; then
+    echo "Skipping $file (not present)."
+    return
+  fi
+
+  local section="" contents="[]" metas="[]"
   while IFS= read -r line; do
     case "$line" in
       "#"*) section="${line#\# }"; section="${section#\#\# }"; continue ;;
       ""|"---") continue ;;
     esac
-    batch=$(jq --arg content "$line" --arg source "$file" --arg section "$section" \
-      '. += [{"content": $content, "metadata": {"source": $source, "section": $section}}]' <<<"$batch")
+    contents=$(jq --arg c "$line" '. += [$c]' <<<"$contents")
+    metas=$(jq --arg content "$line" --arg source "$file" --arg section "$section" \
+      '. += [{"content": $content, "metadata": {"source": $source, "section": $section}}]' <<<"$metas")
   done < "$file"
 
-  echo "Ingesting $(jq 'length' <<<"$batch") chunks from $file..."
+  local n
+  n=$(jq 'length' <<<"$contents")
+  if [ "$n" -eq 0 ]; then
+    echo "Nothing to ingest from $file (no fact/paragraph lines)."
+    return
+  fi
+
+  # litellm-pgvector's embeddings endpoints take a precomputed vector per
+  # chunk — they don't call the embedding model themselves (only query-time
+  # search does, via its own EMBEDDING__* config). So this has to embed
+  # client-side first, via the same proxy every other workload uses.
+  echo "Embedding $n chunks from $file via LiteLLM..."
+  local embeddings
+  embeddings=$(curl -sf "${LITELLM_URL}/v1/embeddings" \
+    -H "Authorization: Bearer ${LITELLM_PGVECTOR_EMBEDDING_KEY}" \
+    -H "Content-Type: application/json" \
+    -d "$(jq -n --argjson input "$contents" '{"model": "local-embedding", "input": $input}')" \
+    | jq '[.data[].embedding]')
+
+  local batch
+  batch=$(jq -n --argjson metas "$metas" --argjson embeds "$embeddings" \
+    '[range(0; ($metas | length)) as $i | $metas[$i] + {"embedding": $embeds[$i]}]')
+
+  echo "Ingesting $n chunks from $file..."
   curl -sf -X POST "${LITELLM_PGVECTOR_URL}/v1/vector_stores/${VECTOR_STORE_ID}/embeddings/batch" \
     -H "Authorization: Bearer ${LITELLM_PGVECTOR_API_KEY}" \
     -H "Content-Type: application/json" \
-    -d "$batch" > /dev/null
+    -d "$(jq -n --argjson embeddings "$batch" '{"embeddings": $embeddings}')" > /dev/null
 }
 
 ingest_file data/memory.md

scripts/update.sh

@@ -106,5 +106,15 @@ set -a && . ./.env && set +a
 echo "==> recreating changed services"
 docker compose up -d --remove-orphans
 
+# litellm-pgvector's Dockerfile only runs `prisma generate` (codegen) at
+# build time — nothing ever applied the schema to pgvector-db itself, so the
+# vector_stores/embeddings tables plain didn't exist until this was added
+# (see issue #24). --accept-data-loss is the same "no rollback/backup logic,
+# git revert is the recovery path" tradeoff as the rest of this script — a
+# schema-incompatible change here would need a manual look regardless.
+echo "==> syncing litellm-pgvector's database schema"
+docker compose up -d --wait pgvector-db litellm-pgvector
+docker compose exec -T litellm-pgvector prisma db push --accept-data-loss
+
 echo "==> status"
 docker compose ps

vendor/litellm-pgvector/VENDORED.md

@@ -1,8 +1,25 @@
 Vendored from https://github.com/BerriAI/litellm-pgvector at commit
-`b553f84a32f580b4303297df5567f25912b59d93` (main, 2026-09-02) — no changes
-made to the source. See `docker-compose.yml`'s `litellm-pgvector` service
-comment for why this is vendored instead of built from a remote git context.
+`b553f84a32f580b4303297df5567f25912b59d93` (main, 2026-09-02). See
+`docker-compose.yml`'s `litellm-pgvector` service comment for why this is
+vendored instead of built from a remote git context.
+
+**Local changes on top of that commit** (found smoke-testing issue #24 —
+without these, the store can never be searched or written to):
+
+- `prisma/schema.prisma`: `Embedding.embedding` was `vector(1536)`
+  (OpenAI ada-002's size); changed to `vector(768)` to match this stack's
+  actual embedding model (nomic-embed-text-v1.5).
+- `models.py` / `main.py`: `POST /v1/vector_stores` always minted a random
+  UUID for the new store's `id`, ignoring anything the caller asked for.
+  Added an optional `id` field to `VectorStoreCreateRequest` and made
+  `create_vector_store` use it when given — `litellm-config.yaml`'s
+  `vector_store_registry` addresses this store by a fixed id
+  (`memory-and-notes`), which never matched a real row otherwise.
+
+Re-applying these after a re-vendor: diff this directory against upstream
+before overwriting, or just redo the three edits above.
 
 To update: `git clone https://github.com/BerriAI/litellm-pgvector.git`
-somewhere, copy everything except `.git/` over this directory, update the
-commit hash above, and run `./scripts/update.sh`.
+somewhere, copy everything except `.git/` over this directory, re-apply the
+local changes above, update the commit hash above, and run
+`./scripts/update.sh`.

vendor/litellm-pgvector/main.py

@@ -86,13 +86,21 @@ async def create_vector_store(
         # Use raw SQL to insert the vector store with configurable table/field names
         vector_store_table = settings.table_names["vector_stores"]
 
+        # ponytail: honor a caller-supplied id (request.id) instead of
+        # always minting one — litellm's vector_store_registry addresses
+        # this store by a fixed id (see litellm-config.yaml), which never
+        # matched anything when this always generated a random UUID.
+        import uuid as _uuid
+        vector_store_id = request.id or str(_uuid.uuid4())
+
         result = await db.query_raw(
             f"""
             INSERT INTO {vector_store_table} (id, name, file_counts, status, usage_bytes, expires_after, metadata, created_at)
-            VALUES (gen_random_uuid(), $1, $2, $3, $4, $5, $6, NOW())
+            VALUES ($1, $2, $3, $4, $5, $6, $7, NOW())
             RETURNING id, name, file_counts, status, usage_bytes, expires_after, expires_at, last_active_at, metadata,
                      EXTRACT(EPOCH FROM created_at)::bigint as created_at_timestamp
             """,
+            vector_store_id,
             request.name,
             {"in_progress": 0, "completed": 0, "failed": 0, "cancelled": 0, "total": 0},
             "completed",

vendor/litellm-pgvector/models.py

@@ -5,6 +5,12 @@ from datetime import datetime
 
 class VectorStoreCreateRequest(BaseModel):
     name: str
+    # ponytail: not part of upstream litellm-pgvector — added locally so
+    # callers (litellm's vector_store_registry, scripts/ingest-memory.sh)
+    # can pin a human-readable id instead of getting a random UUID back.
+    # litellm-config.yaml's vector_store_registry addresses stores by a
+    # fixed vector_store_id, which only works if creation can honor it.
+    id: Optional[str] = None
     file_ids: Optional[List[str]] = None
     expires_after: Optional[Dict[str, Any]] = None
     chunking_strategy: Optional[Dict[str, Any]] = None

vendor/litellm-pgvector/prisma/schema.prisma

@@ -30,7 +30,12 @@ model Embedding {
   id              String      @id @default(cuid())
   vector_store_id String
   content         String
-  embedding       Unsupported("vector(1536)")
+  // 768, not OpenAI's ada-002-sized 1536 — this stack's embedding_model is
+  // nomic-embed-text-v1.5 (see docker-compose.yml's embedding-server and
+  // litellm-config.yaml's local-embedding entry), confirmed 768-dim live
+  // against /v1/embeddings. A push with the wrong size here makes every
+  // insert fail on a pgvector dimension mismatch.
+  embedding       Unsupported("vector(768)")
   metadata        Json?
   created_at      DateTime    @default(now())