Deployment and handover

1The map

Three products, four repositories, four machines. Everything below was read off the running systems rather than remembered, so where this document and your memory disagree, believe this document and then check the machine.

Getting in:

ssh -i ~/Downloads/sailagentecsdevkey.pem ubuntu@35.91.251.211   # Service, landing, docs
ssh -i ~/Downloads/sailagentecsdevkey.pem ubuntu@54.254.25.0    # Market
ssh -i ~/Downloads/sailagentecsdevkey.pem ubuntu@54.188.207.85  # Community

The model machine has its own key and is normally reached from the AWS console instead, through EC2 Instance Connect, so nobody has to be sent a private key.

2The four repositories

The last two are staged under a personal account because write access on the organisation copies has not been granted yet. When it is, the remote changes and nothing else does.

Each repository has the same shape: backend/ is FastAPI, frontend/ is Next.js exported to static files, and docs/ holds the written documents. The landing page has only a frontend.

Node is not on the system path on the build machine. Every frontend command below needs this first:

export PATH="$HOME/.local/node/bin:$PATH"

3Deploying each one

The pattern is the same everywhere and it is deliberate. Build locally, rsync to a staging directory in the deploy user's home, then move it into place with sudo. Never build on the server: these machines have under 2GB of memory and a Next.js build will take one down.

3.1SmartService

Backend, at /home/ubuntu/plumber/backend, unit plumber, port 8100.

rsync -az --exclude '__pycache__' --exclude '.venv' --exclude '.env' \
  plumber-assistant/backend/app/ ubuntu@35.91.251.211:/home/ubuntu/plumber/backend/app/
ssh ubuntu@35.91.251.211 'sudo systemctl restart plumber'

Frontend, served from /var/www/serviceagent.

cd plumber-assistant/frontend && npm run build:serviceagent
rsync -az --delete out/ ubuntu@35.91.251.211:/home/ubuntu/deploy-serviceagent/
ssh ubuntu@35.91.251.211 \
  'sudo rsync -a --delete /home/ubuntu/deploy-serviceagent/ /var/www/serviceagent/'

build:serviceagent rather than build: it sets an empty base path and an empty API url, because the app sits at the root of its own subdomain and the API is same origin.

3.2SmartMarket

Backend, at /var/www/ai-order/backend, unit aiorder.

rsync -az --exclude '__pycache__' --exclude '.env' \
  AI-Orders-main/backend/app/ ubuntu@54.254.25.0:/home/ubuntu/deploy-market-app/
ssh ubuntu@54.254.25.0 \
  'sudo rsync -a /home/ubuntu/deploy-market-app/ /var/www/ai-order/backend/app/ && sudo systemctl restart aiorder'

Frontend, served from /var/www/ai-order/frontend-dist.

cd AI-Orders-main/ai-order && npm run build:deploy
rsync -az --delete out/ ubuntu@54.254.25.0:/home/ubuntu/deploy-market/
ssh ubuntu@54.254.25.0 \
  'sudo rsync -a --delete /home/ubuntu/deploy-market/ /var/www/ai-order/frontend-dist/'

3.3SmartCommunity

Backend, at /home/ubuntu/community/backend, unit community, port 8200.

rsync -az --exclude '__pycache__' --exclude '.venv' --exclude '.env' \
  community-assistant/backend/app/ ubuntu@54.188.207.85:/home/ubuntu/community/backend/app/
ssh ubuntu@54.188.207.85 'sudo systemctl restart community'

Frontend, served from /var/www/community.

cd community-assistant/frontend && npm run build
rsync -az --delete out/ ubuntu@54.188.207.85:/home/ubuntu/deploy-community/
ssh ubuntu@54.188.207.85 \
  'sudo rsync -a --delete /home/ubuntu/deploy-community/ /var/www/community/ && sudo chown -R www-data:www-data /var/www/community'

Port 8200 rather than 8100 is deliberate. This machine was cloned from the Service Assistant, and a stray request meant for that product must not find something listening here and be answered by it.

3.4The landing page

Static only, served from /var/www/smartzees on the Service machine.

cd smartzees-landing/app && npm run build
rsync -az --delete out/ ubuntu@35.91.251.211:/home/ubuntu/deploy-smartzees/
ssh ubuntu@35.91.251.211 \
  'sudo rsync -a --delete /home/ubuntu/deploy-smartzees/ /var/www/smartzees/'

3.5The documents

Served from /var/www/serviceagent-docs, which is outside the site root on purpose: the frontend deploy above uses --delete and would otherwise remove them.

python3 docs/_house/hub.py            # rebuilds docs/index.html from its list
python3 docs/deployment/build.py      # this document
rsync -az docs// ubuntu@35.91.251.211:/home/ubuntu/deploy-docs//
ssh ubuntu@35.91.251.211 \
  'sudo rsync -a /home/ubuntu/deploy-docs/ /var/www/serviceagent-docs/'

Generated document sets (source.md plus the render_*.py scripts) deploy their build/web/ directory. Hand written ones (srs, lab, deployment, the two specifications) deploy the index.html beside the builder.

4What the code files do

Only the ones somebody unfamiliar would otherwise have to read to understand. The rest are named for what they do.

Shared by all three products

SmartService only

SmartCommunity only

SmartMarket only

5The model, and the switch

One switch decides which engine answers, for all three products, and it lives on servicez.smartzees.com/admin under AI runtime. The other two read it over HTTP and never write it.

Settings on the Service machine, in .env:

GPU_INSTANCE_ID=i-0c9e7a485d54e4e92
AWS_REGION=us-west-2
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
OLLAMA_MODEL=llama3.2:3b
OLLAMA_TIMEOUT_SECONDS=45

The other two only need OLLAMA_URL and OLLAMA_MODEL, because they never start or stop anything.

6Rules that are not optional

Checking the first rule takes one command and belongs in every handover:

for h in 35.91.251.211 54.254.25.0 54.188.207.85; do
  echo -n "$h "
  ssh ubuntu@$h 'systemctl is-active plumber-sync aiorder-sync | tr "\n" " "'
  echo
done

7Checking a deploy worked

Not "it returned 200", which a stale page also does. Ask it something.

curl -s -X POST https://servicez.smartzees.com/api/v1/chat \
  -H 'Content-Type: application/json' \
  -d '{{"message":"my boiler is leaking","session_id":"check"}}'

curl -s -X POST https://marketz.smartzees.com/api/v1/chat \
  -H 'Content-Type: application/json' \
  -d '{{"message":"milk","session_id":"check"}}'

curl -s -X POST https://livz.smartzees.com/api/v1/chat \
  -H 'Content-Type: application/json' \
  -d '{{"message":"quiet hours","session_id":"check","community":"serenity"}}'

A good answer from the third names a document. If it does not, the index did not load, whatever the status code said.

And the platform switch, which the other two agents depend on:

curl -s https://servicez.smartzees.com/api/v1/ai/provider

8Leftovers on the machines

Written down because they look important and are not. Every one is a copy left behind when a machine was cloned.

None of these are served by nginx. If you are editing something and nothing changes, check you are on the right machine before you check anything else.