Deployment Guide

Last Updated: 2026-02-16 Status: Active Audience: Developers, DevOps

This guide documents deployment procedures for the Client Portal application at scopeforged.com.

Table of Contents

1. Overview
2. Environment Setup
3. Deployment Commands
4. Server Structure
5. Server Configuration
6. Rollback Procedures
7. Migration from Legacy Deployment
8. Monitoring

Overview

Release-Based Deployment

The application uses a Capistrano-style release-based deployment system that provides:

• Atomic deployments - New release serves traffic only after full verification
• Instant rollback - Switch to previous release in seconds
• Release history - Keep N previous releases for debugging/comparison
• Zero-downtime - Old release serves traffic until new release is ready
• Shared resources - .env and storage persist across releases

Deployment Environments

Infrastructure

• Server: AWS EC2 Ubuntu instance
• Web Server: Apache with mod_php
• Database: MySQL 8.0
• SSL: Let's Encrypt (certbot)
• Cache/Session/Queue: Database driver

Requirements

• PHP 8.2+
• MySQL 8.0+
• Node.js 18+ (for local asset compilation)
• Apache with mod_rewrite

Optional Server Tooling

• OpenAPI Generator CLI — required only if you use the admin SDK generator (admin.scopeforged.com → API → SDKs). The deploy archive does not ship node_modules, so this must be installed once on the server.

  Run on the production host:

  sudo bash scripts/setup/install-openapi-generator.sh
  

  The script installs a JRE (the CLI is a Java JAR wrapper) and installs @openapitools/openapi-generator-cli globally. Re-run with --force to reinstall. Without this, php artisan api:generate-sdk <language> will fail and the admin UI surfaces "OpenAPI Generator CLI is not installed"; the --spec-only path still works without it.

Environment Setup

Local Environment Variables

Add these to your local .env file for deployment:

# Deployment Configuration
SERVER_HOST=54.190.150.0
SERVER_USERNAME=ubuntu
SERVER_PRIVATE_KEY=C:/Users/me/.ssh/ps4_new
SERVER_BASE_PATH=/var/www/client-portal-laravel
RELEASES_TO_KEEP=5

Production .env Location

The production .env is stored on the server at /var/www/client-portal-laravel/shared/.env and is symlinked into each release.

Deployment Commands

Deploy New Release

# Standard deployment (builds and deploys)
npm run deploy

# Skip npm build (if assets haven't changed)
npm run deploy -- --skip-build

# Skip composer install (if dependencies haven't changed)
npm run deploy -- --skip-composer

# Fresh deployment (clear staging directory and rebuild)
npm run deploy -- --fresh

# Run database migrations during deployment
npm run deploy -- --migrate

# Combine flags as needed
npm run deploy -- --skip-build --migrate

Rollback

# Roll back to previous release
npm run deploy:rollback

# Roll back 2 releases
npm run deploy:rollback -- --releases=2

# Roll back to specific release
npm run deploy:rollback -- --to=20260115120000

# Preview rollback without making changes
npm run deploy:rollback -- --dry-run

List Releases

# Show all releases
npm run deploy:releases

# Show verbose info (sizes, shared directory status)
npm run deploy:releases -- --verbose

# Output as JSON (for scripting)
npm run deploy:releases -- --json

What the Deploy Script Does

The scripts/deploy.js script performs these steps:

1. Local Build Phase

   • Syncs source files to staging directory
   • Runs npm run build (Vite) - unless --skip-build
   • Runs composer install --no-dev --optimize-autoloader - unless --skip-composer

2. Package Phase

   • Creates a ZIP archive named release-{timestamp}.zip
   • Includes: app/, bootstrap/, config/, database/, public/, resources/, routes/, storage/app/, storage/framework/, vendor/, artisan, composer.json

3. Upload Phase

   • Uploads ZIP to server via SCP

4. Server Phase

   • Creates new release directory: releases/{timestamp}/
   • Extracts package to release directory
   • Links shared resources (.env, storage/)
   • Sets permissions (ubuntu:www-data, 775 for writable dirs)
   • Runs artisan optimization commands (config:cache, route:cache, view:cache)
   • Optionally runs migrations (--migrate flag)
   • Verifies release is functional
   • Switches current symlink to new release (atomic!)
   • Reloads Apache and restarts queue workers/Reverb
   • Cleans up old releases (keeps configurable number)

5. Error Handling

   • If any step fails, the incomplete release is deleted
   • The current symlink is never changed if deployment fails

Server Structure

Directory Layout

/var/www/client-portal-laravel/
├── releases/                    # Release directories
│   ├── 20260114120000/
│   ├── 20260114150000/
│   └── 20260115093000/          # Latest release
├── current -> releases/20260115093000/   # Symlink to active release
└── shared/                      # Persistent files
    ├── .env                     # Environment config
    └── storage/
        ├── app/
        │   └── public/          # User uploads
        ├── framework/
        │   ├── cache/
        │   ├── sessions/
        │   └── views/
        └── logs/                # Application logs

Inside Each Release

releases/20260115093000/
├── app/
├── bootstrap/
├── config/
├── database/
├── public/                      # Apache DocumentRoot (via current symlink)
├── resources/
├── routes/
├── storage -> ../../shared/storage   # Symlink to shared
├── vendor/
├── .env -> ../../shared/.env         # Symlink to shared
├── artisan
├── composer.json
└── composer.lock

Server Configuration

Apache Virtual Host

Located at /etc/apache2/sites-available/scopeforged.com.conf:

<VirtualHost *:80>
    ServerName scopeforged.com
    Redirect permanent / https://scopeforged.com/
</VirtualHost>

<VirtualHost *:443>
    ServerName scopeforged.com
    DocumentRoot /var/www/client-portal-laravel/current/public

    <Directory /var/www/client-portal-laravel/current/public>
        Options FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/portal-error.log
    CustomLog ${APACHE_LOG_DIR}/portal-access.log combined

    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/scopeforged.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/scopeforged.com/privkey.pem

    # WebSocket proxy for Reverb
    ProxyPreserveHost On
    ProxyPass /app ws://127.0.0.1:8080/app
    ProxyPassReverse /app ws://127.0.0.1:8080/app
    ProxyPass /apps ws://127.0.0.1:8080/apps
    ProxyPassReverse /apps ws://127.0.0.1:8080/apps
</VirtualHost>

Important: Options FollowSymLinks is required for Apache to follow the current symlink.

Queue Workers (Supervisor)

Located at /etc/supervisor/conf.d/client-portal-worker.conf:

[program:client-portal-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/client-portal-laravel/current/artisan queue:work database --sleep=3 --tries=3 --max-time=3600
directory=/var/www/client-portal-laravel/current
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=2
redirect_stderr=true
stdout_logfile=/var/www/client-portal-laravel/shared/storage/logs/worker.log
stopwaitsecs=3600

Reverb WebSocket Server (Supervisor)

Located at /etc/supervisor/conf.d/client-portal-reverb.conf:

[program:client-portal-reverb]
process_name=%(program_name)s
command=php /var/www/client-portal-laravel/current/artisan reverb:start --host=127.0.0.1 --port=8080
directory=/var/www/client-portal-laravel/current
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=1
redirect_stderr=true
stdout_logfile=/var/www/client-portal-laravel/shared/storage/logs/reverb.log
stopwaitsecs=10

Scheduler (Cron)

Located at /etc/cron.d/client-portal-scheduler:

* * * * * www-data cd /var/www/client-portal-laravel/current && php artisan schedule:run >> /dev/null 2>&1

File Permissions

# Ownership for all releases
sudo chown -R ubuntu:www-data /var/www/client-portal-laravel

# Writable directories in shared
sudo chmod -R 775 /var/www/client-portal-laravel/shared/storage

Rollback Procedures

Automatic Rollback (Recommended)

# Roll back to previous release
npm run deploy:rollback

# Roll back 2 releases
npm run deploy:rollback -- --releases=2

# Roll back to specific release
npm run deploy:rollback -- --to=20260114150000

The rollback script:

1. Switches the current symlink to the target release
2. Clears and rebuilds caches
3. Reloads Apache
4. Restarts queue workers and Reverb

Manual Rollback (On Server)

# SSH to server
ssh -i ~/.ssh/ps4_new ubuntu@54.190.150.0

# List available releases
ls -la /var/www/client-portal-laravel/releases/

# Check current release
readlink /var/www/client-portal-laravel/current

# Switch to previous release (atomic)
cd /var/www/client-portal-laravel
ln -sfn releases/20260114150000 current

# Clear caches
cd current
php artisan config:clear
php artisan config:cache
php artisan route:cache
php artisan view:cache

# Reload services
sudo systemctl reload apache2
sudo supervisorctl restart client-portal-worker:*
sudo supervisorctl restart client-portal-reverb

Database Rollback

# SSH to server
ssh -i ~/.ssh/ps4_new ubuntu@54.190.150.0

# Navigate to current release
cd /var/www/client-portal-laravel/current

# Rollback last migration
php artisan migrate:rollback --step=1

# Check migration status
php artisan migrate:status

Emergency Procedures

# SSH to server
ssh -i ~/.ssh/ps4_new ubuntu@54.190.150.0
cd /var/www/client-portal-laravel/current

# 1. Enable maintenance mode
php artisan down --secret="emergency-bypass-token"

# 2. Check error logs
tail -50 ../shared/storage/logs/laravel-$(date +%Y-%m-%d).log

# 3. Roll back if needed
cd ..
ln -sfn releases/PREVIOUS_RELEASE_TIMESTAMP current

# 4. Clear all caches
cd current
php artisan optimize:clear

# 5. Reload services
sudo systemctl reload apache2
sudo supervisorctl restart client-portal-worker:*

# 6. Disable maintenance mode
php artisan up

Migration from Legacy Deployment

If migrating from the old direct-deploy system to release-based deployment:

One-Time Server Setup

# SSH to server
ssh -i ~/.ssh/ps4_new ubuntu@54.190.150.0

# Create new directory structure
cd /var/www/client-portal-laravel
sudo mkdir -p releases
sudo mkdir -p shared/storage/app/public
sudo mkdir -p shared/storage/framework/{cache,sessions,views}
sudo mkdir -p shared/storage/logs

# Move existing .env to shared
sudo mv .env shared/.env

# Move existing uploads to shared storage
sudo mv storage/app/public/* shared/storage/app/public/ 2>/dev/null || true

# Move logs to shared storage
sudo mv storage/logs/* shared/storage/logs/ 2>/dev/null || true

# Set permissions
sudo chown -R ubuntu:www-data .
sudo chmod -R 775 shared/storage

Update Apache Configuration

# Edit Apache config
sudo nano /etc/apache2/sites-available/scopeforged.com.conf

# Change DocumentRoot to use /current/public:
#   DocumentRoot /var/www/client-portal-laravel/current/public
# Update <Directory> block similarly with Options FollowSymLinks

# Test config
sudo apache2ctl configtest

Update Supervisor Configuration

# Edit queue worker config
sudo nano /etc/supervisor/conf.d/client-portal-worker.conf
# Change paths to use /current/

# Edit reverb config
sudo nano /etc/supervisor/conf.d/client-portal-reverb.conf
# Change paths to use /current/

# Reload supervisor
sudo supervisorctl reread
sudo supervisorctl update

Update Cron

sudo nano /etc/cron.d/client-portal-scheduler
# Change path to use /current/

First Release Deployment

After setup, run npm run deploy which will:

1. Create the first release in releases/{timestamp}/
2. Create the current symlink
3. Apache will start serving from the new structure

Monitoring

Health Check

The application includes a health check endpoint at /admin/system/health (requires admin login).

Log Files

# SSH to server
ssh -i ~/.ssh/ps4_new ubuntu@54.190.150.0

# View today's Laravel log (in shared directory)
tail -f /var/www/client-portal-laravel/shared/storage/logs/laravel-$(date +%Y-%m-%d).log

# View Apache error log
tail -f /var/log/apache2/portal-error.log

# View queue worker log
tail -f /var/www/client-portal-laravel/shared/storage/logs/worker.log

Admin System Tools

The application provides admin tools at:

• /admin/system/health - System health overview
• /admin/system/logs - Log viewer with filtering
• /admin/system/cache - Cache management

Deployment Checklist

Pre-Deployment

• All tests passing (php artisan test)
• Code formatted (./vendor/bin/pint)
• Database migrations tested locally
• No uncommitted changes

Post-Deployment

• Site loads correctly at https://scopeforged.com
• Login functionality works
• Check error logs for issues
• Verify database connectivity
• Verify current symlink points to new release

Verification Commands

# Check current release
npm run deploy:releases

# Check symlink on server
ssh -i ~/.ssh/ps4_new ubuntu@54.190.150.0 "readlink /var/www/client-portal-laravel/current"

# Check .env symlink in release
ssh -i ~/.ssh/ps4_new ubuntu@54.190.150.0 "ls -la /var/www/client-portal-laravel/current/.env"

Best Practices

Do

• Run tests before deploying
• Check logs after deployment
• Use --skip-build when only PHP changes
• Keep production .env secure in shared/ directory
• Use npm run deploy:releases to verify deployment status

Don't

• Deploy on Fridays
• Skip testing
• Use migrate:refresh in production
• Commit production .env to git
• Manually edit files in release directories (changes will be lost)

Sentry Release Tracking

The deploy and rollback scripts automatically integrate with Sentry to track releases.

What Happens Automatically

During deploy:

1. SENTRY_RELEASE is written to the shared .env on the server (before config:cache)
2. After services restart, sentry-cli creates a Sentry release with associated commits
3. A deploy is registered in Sentry's timeline for the production environment

During rollback:

1. SENTRY_RELEASE is updated to the rolled-back-to release version
2. A deploy is registered in Sentry for the rollback

Configuration

Add these to .env.deployment (deploy-machine only, not .env.example):

SENTRY_AUTH_TOKEN=your-auth-token-here
SENTRY_ORG=your-org-slug
SENTRY_PROJECT=your-project-slug

Generate an auth token at Sentry Settings > Auth Tokens with project:releases and org:read scopes.

Graceful Degradation

• If SENTRY_AUTH_TOKEN is not set, deploys proceed normally — only SENTRY_RELEASE is written to .env
• Sentry CLI failures are non-fatal — deploys and rollbacks always complete
• --clear-cached mode does not create Sentry releases or modify SENTRY_RELEASE

Related Documentation

• CONFIGURATION.md - Environment configuration
• LOGGING_STANDARDS.md - Logging setup
• SECURITY.md - Security practices
• Plan 118 - Implementation plan