Mars City Cloud Infrastructure

This page describes a recommended cloud architecture for production deployments of Digital Asset Manager Pro ("Mars City"). It is intended as a starting point and should be adapted to your cloud provider and compliance needs.

High-level topology

  • VPC with public and private subnets across multiple availability zones.
  • Public-facing load balancer (ALB / Cloud Load Balancer) handling TLS termination.
  • Kubernetes cluster (EKS/GKE/AKS) or managed container service for the backend, workers, and auxiliary services.
  • Managed relational database (RDS / Cloud SQL) for metadata and user accounts.
  • Object storage (S3 / GCS / Blob Storage) for binary assets and backups.
  • Redis or managed queue for background job coordination.

Security & Secrets

  • Use a secrets manager (AWS Secrets Manager, Google Secret Manager, HashiCorp Vault) for DB credentials and FLASK_SECRET.
  • Enforce HTTPS only (redirect HTTP to HTTPS) and use HSTS.
  • Use IAM roles/service accounts with least privilege for storage and DB access.

Scalability & Reliability

  • Autoscaling for worker and API pods based on CPU/queue-depth.
  • Multi-AZ DB with automated backups and point-in-time recovery enabled.
  • Use CDN for serving static UI assets and offload bandwidth from origin.

Observability

  • Centralized logging (Cloud logging, ELK) and structured logs from backend and workers.
  • Metrics + alerting (Prometheus/Grafana or cloud-managed metrics + alerting).
  • Tracing (OpenTelemetry) for request flows across frontend/backend/workers.

CI/CD and Deployment

  • Use a CI pipeline (GitHub Actions / GitLab CI / Cloud Build) to run tests and build artifacts.
  • Deploy to Kubernetes via manifests / Helm / GitOps (ArgoCD, Flux).
  • Automate MkDocs build and publish for documentation site; if the wiki must remain protected, deploy the Docs behind the same auth layer.

Backups & DR

  • Regular backups of the database and object storage retention policies.
  • Periodic verification of backup restores.

Cost & Governance

  • Tag resources for cost allocation.
  • Implement resource quotas and alerts for unexpected spend.

Next steps

  • Add provider-specific deployment instructions (AWS/GCP/Azure) and example Terraform or Helm charts.

Developer mapping & local dev checklist

This section maps the high-level architecture to local files, runtime flows, and common development tasks so you can plan and track work.

What the app does (quick)

  • Serves a static frontend (multiple pre-built pages in the repo) that calls a small Flask backend (server.py) for authentication and asset metadata.
  • Stores users, settings, assets, and provider tokens in a local SQLite DB (users.db). DB schema lives in init_db.py.
  • Assets can be created via JSON, uploaded via multipart (/api/assets/upload), and optionally stored in GCS or other external storage via connectors in storage/.
  • The documentation is built with MkDocs (mkdocs.yml) and served from the site/ folder behind the protected /wiki/ route.

Key files / components

  • server.py — main Flask app and API endpoints (login, assets CRUD, upload, auth callback, wiki serving).
  • init_db.py — DB schema and seeding (users, assets, settings, secrets).
  • add_user.py — helper to add/update hashed passwords in users.db.
  • storage/ — connector scaffolding for Google Drive, OneDrive, and Google Cloud Storage (connectors.py) and DB helpers in storage/db.py.
  • uploads/ — (local) directory for temporary uploaded files (used when GCS not configured).
  • docs/ and mkdocs.yml — documentation source; site/ is the built site.
  • tests/ — pytest tests that validate endpoints and flows (runs against local dev server at http://127.0.0.1:8000).

Runtime flow (asset upload example)

  1. User authenticates via /sign_in.html → /api/login (session cookie created).
  2. Frontend submits multipart upload to /api/assets/upload.
  3. Server validates file size / MIME type, stores locally or uploads to GCS (if GCS_BUCKET is set), records an assets row, and returns an asset object with url (signed if GCS).
  4. Background workers (future) would generate thumbnails, transcode, and update metadata; currently seeding and simple returns are included in init_db.py and server.py.

Local dev commands (quick)

cd "d:\WebDev\DigitalAssetMgrPro\stitch_digital_asset_manager_pro"
python -m pip install -r requirements.txt
python init_db.py             # one-time DB init + seed data
python server.py              # run dev server (127.0.0.1:8000)
python -m mkdocs serve -a 127.0.0.1:8001  # live docs (optional)
python -m pytest             # run tests (server should be running)

Things to map & test when extending the app

  • External storage: create adapters (S3/GCS) and tests that mock provider calls; store short-lived credentials in secrets table via storage/db.py.
  • Background processing: add worker process (RQ/Celery) and a queue (Redis) for long-running tasks (thumbnails, ingestion).
  • Auth: consider migrating to JWT/session cookies consistently and protect admin APIs.
  • Docs: keep docs/ and site/ in sync; update mkdocs.yml nav when adding pages (pages in site/ are served behind /wiki/).

Quick troubleshooting tips

  • If /wiki/ redirects to /sign_in.html, you need an authenticated session or init DB seeded user (run init_db.py and use admin@example.com / password123).
  • Check the server logs — server.py prints helpful debug lines for /wiki/ requests (use ?_debug=1 on wiki paths to get internal checks).
  • If uploads fail, ensure env vars UPLOAD_MAX_BYTES, UPLOAD_ALLOWED_MIMETYPES, and GCS_BUCKET (if using GCS) are set appropriately.

If you want, I can add a mapping checklist as a page under docs/ (e.g., docs/development_map.md) with linked tasks and initial issues you can track. Would you like that created now?