WSGI Server Session Storage Configuration

Problem

When running NOW LMS with production WSGI servers (Gunicorn multi-process or Waitress multi-threaded), users can experience erratic session behavior:

Sometimes appeared logged in after authentication
Sometimes appeared logged out after refresh
"Already logged in" messages appeared inconsistently
UI flickered between authenticated and anonymous states

Root Cause

The WSGI server does not make Flask's signed-cookie sessions worker-local. The failure occurs when NOW LMS selects server-side sessions but that interface is not actually installed, its backend is unavailable, or workers use different secrets/backends.

Gunicorn (Multi-Process)

Gunicorn spawns multiple worker processes (e.g., gunicorn app:app --workers 4). Every process must use the same stable SECRET_KEY and the same Redis or SQLAlchemy session backend. NOW LMS also disables Gunicorn app preloading so workers do not inherit pooled database connections opened before the fork.

Waitress (Multi-Threaded)

Waitress uses a single process with multiple threads, so it does not switch requests between process-local workers. It still uses the same server-side backend as Gunicorn, which keeps deployment behavior consistent and supports multiple NOW LMS instances behind a load balancer.

Solution

Implement shared session storage that all workers/threads can safely access:

1. Redis (Preferred)

Redis provides optimal performance and is the recommended solution for production with both Gunicorn and Waitress:

Fast in-memory storage
Shared across all workers/threads
Thread-safe operations
Persistent across server restarts
Supports session expiration

2. SQLAlchemy (Fallback)

When Redis is not configured, sessions use the same SQLAlchemy database as NOW LMS:

Shared across worker processes and application instances
Requires no additional service
Works with every database backend supported by NOW LMS

3. Testing Mode

During tests (when pytest is detected), the system uses Flask's default signed cookie sessions since tests run in a single process.

Configuration

Automatic Configuration

The system automatically detects the best available session storage:

# Priority order:
1. Redis (if REDIS_URL or SESSION_REDIS_URL is set)
2. SQLAlchemy database (if not in testing mode)
3. Default Flask sessions (for testing)

Redis Configuration

Set the Redis URL in your environment:

export REDIS_URL=redis://localhost:6379/0
# or
export SESSION_REDIS_URL=redis://localhost:6379/0

Then run your WSGI server of choice:

# Using Waitress (default)
lmsctl serve

# Using Gunicorn
lmsctl serve --wsgi-server gunicorn

# Or directly
gunicorn "now_lms:lms_app" --workers 4 --bind 0.0.0.0:8000

Without Redis

If Redis is not configured, the system automatically stores sessions in the configured SQLAlchemy database. No additional service is needed.

SECRET_KEY (Critical!)

ALWAYS set a stable SECRET_KEY in production:

export SECRET_KEY="your-long-random-secret-key-here"

⚠️ Never use the default "dev" SECRET_KEY in production! This will cause session issues even with shared storage.

Generate a secure key:

python -c "import secrets; print(secrets.token_hex(32))"

Session Settings

All session storage backends use these production-ready settings:

SESSION_PERMANENT: False (sessions expire when browser closes, but PERMANENT_SESSION_LIFETIME still applies)
SESSION_USE_SIGNER: True (sessions are cryptographically signed for security)
PERMANENT_SESSION_LIFETIME: 86400 seconds (24 hours)
SESSION_KEY_PREFIX: "session:" (for Redis, to namespace keys)
SESSION_COOKIE_HTTPONLY: True (prevents JavaScript access to session cookie)
SESSION_COOKIE_SECURE: True in production (enforces HTTPS)
SESSION_COOKIE_SAMESITE: "Lax" (protects against CSRF attacks)
SESSION_CLEANUP_N_REQUESTS: 100 for the SQLAlchemy backend

Files Modified

requirements.txt: Added flask-session dependency
now_lms/session_config.py: Session configuration with cookie security settings
now_lms/__init__.py: Integrated session initialization
now_lms/config/__init__.py: Added SECRET_KEY warning
run.py: Waitress configuration with shared session storage
now_lms/cli.py: Both Gunicorn and Waitress configuration with shared session storage

Testing

Run the session configuration tests:

pytest tests/test_session_multiworker.py -v

Tests verify: - Redis configuration when REDIS_URL is set - SQLAlchemy fallback when Redis is not configured - Proper settings for production use - Flask-Login persistence across independent worker processes

WSGI Server Configuration

NOW LMS has built-in configuration for both Waitress and Gunicorn in run.py and now_lms/cli.py with optimal settings for session handling:

Waitress Configuration

# Key configurations for session support
serve(
    lms_app,
    host="0.0.0.0",
    port=PORT,
    threads=threads,  # Automatically calculated based on system resources
    channel_timeout=120,
    cleanup_interval=30,
)

Important: - Waitress is single-process, multi-threaded - Thread count is automatically calculated based on CPU and RAM - Works well with both Redis and SQLAlchemy sessions - Cross-platform (Windows, Linux, macOS)

Gunicorn Configuration

# Key configurations for session support
options = {
    "preload_app": False,  # Keep SQLAlchemy engines worker-local
    "workers": workers,  # Intelligent calculation based on CPU and RAM
    "threads": threads,  # Default 1, can be >1 for more concurrency
    "worker_class": "gthread" if threads > 1 else "sync",
    "graceful_timeout": 30,
}

Important: - preload_app = False prevents workers from inheriting pooled database connections - Works with both Redis and SQLAlchemy sessions - Worker/thread counts are automatically calculated based on system resources - Linux/Unix only (not supported on Windows)

WSGI Server Best Practices

Using the CLI Command

The recommended way to run NOW LMS is using the built-in CLI:

# Using Waitress (default, cross-platform)
lmsctl serve

# Using Gunicorn (Linux/Unix only)
lmsctl serve --wsgi-server gunicorn

Or directly with Python:

# Uses Waitress by default
python run.py

The CLI command automatically configures your chosen WSGI server with: - Intelligent worker/thread calculation based on CPU and RAM - Environment variable support (NOW_LMS_WORKERS, NOW_LMS_THREADS) - Proper session storage configuration

With Environment Variables

export SECRET_KEY="your-secret-key"
export REDIS_URL="redis://localhost:6379/0"
export DATABASE_URL="postgresql://user:pass@localhost/dbname"
export NOW_LMS_WORKERS=4  # For Gunicorn only
export NOW_LMS_THREADS=4  # For both Waitress and Gunicorn

# Using Waitress (default)
lmsctl serve

# Using Gunicorn
lmsctl serve --wsgi-server gunicorn

Advanced: Direct Server Commands

Waitress

waitress-serve --host=0.0.0.0 --port=8000 --threads=4 now_lms:lms_app

Gunicorn

gunicorn "now_lms:lms_app" --workers 4 --threads 2 --bind 0.0.0.0:8000

Note: Using the CLI command (lmsctl serve) is recommended as it automatically configures the server with optimal settings.

Worker/Thread Count Recommendations

NOW LMS automatically calculates optimal counts, but you can override:

For Gunicorn (Multi-Process)

CPU-bound workloads: workers = (2 × CPU cores) + 1
I/O-bound workloads: Use more threads per worker instead

Example for 4 CPU cores:

export NOW_LMS_WORKERS=9
lmsctl serve --wsgi-server gunicorn

For Waitress (Single-Process Multi-Threaded)

Threads: Automatically calculated based on available RAM and CPU
I/O-bound workloads: Can benefit from higher thread counts

Example:

export NOW_LMS_THREADS=8
lmsctl serve

Additional Server Options

Gunicorn with Timeout

gunicorn "now_lms:lms_app" --workers 4 --timeout 120 --bind 0.0.0.0:8000

Waitress with Custom Settings

waitress-serve --host=0.0.0.0 --port=8000 --threads=8 --channel-timeout=120 now_lms:lms_app

Monitoring

Check logs for session configuration:

INFO: Configuring Redis-based session storage for multi-worker/multi-threaded WSGI servers
INFO: Session storage initialized: redis
INFO: Using Redis for session storage - optimal for multi-worker WSGI servers

or

INFO: Configuring SQLAlchemy-based session storage for multi-worker/multi-threaded WSGI servers
INFO: Session storage initialized: sqlalchemy
INFO: Session table: flask_sessions

Troubleshooting

Sessions still erratic with Redis

Verify Redis is running: redis-cli ping (should return "PONG")
Check Redis URL is correct: echo $REDIS_URL
Verify SECRET_KEY is set and stable: echo $SECRET_KEY
Check WSGI server logs for session initialization messages

Sessions not persisting

Check SECRET_KEY is not "dev": echo $SECRET_KEY
Verify the application database is reachable and the flask_sessions table can be created
Check session expiration (default 24 hours)

Redis connection errors

If Redis is configured but not available:

# Disable the Redis setting to use the SQLAlchemy fallback
unset REDIS_URL
unset SESSION_REDIS_URL

Choosing Between Waitress and Gunicorn

Both servers work well with NOW LMS and support the same session storage configuration:

Use Waitress When:

Running on Windows (Gunicorn not supported)
Want simple, single-process deployment
Prefer Python-only dependencies
Need cross-platform compatibility
Single server with moderate traffic

Use Gunicorn When:

Running on Linux/Unix
Need multiple worker processes for better CPU utilization
Want traditional Unix-style process management
High-traffic production environment
Need worker-local database connection pools

Switching Between Servers

The configuration is designed to work seamlessly with both:

# No configuration changes needed!

# Using Waitress
lmsctl serve

# Using Gunicorn
lmsctl serve --wsgi-server gunicorn

Session storage configuration is shared and works identically with both servers.

Performance

Redis vs SQLAlchemy

Feature	Redis	SQLAlchemy
Speed	Very Fast	Database-dependent
Scalability	Excellent	Good
Multi-server	Yes	Yes, with a shared database
Persistence	Configurable	Yes
Setup	Requires Redis	Uses the NOW LMS database
Concurrent workers	Yes	Yes

Recommendations

Single server, low traffic: SQLAlchemy is sufficient
Single server, high traffic: Use Redis
Multiple servers: Use Redis (required)
Development: Either works
Testing: Automatic (no configuration needed)

Security Notes

Always use HTTPS in production - sessions are transmitted in cookies
Set SECRET_KEY to a strong, random value - never use "dev"
Enable SESSION_USE_SIGNER (automatic) - prevents session tampering
Use Redis with authentication if exposed to network
Rotate SECRET_KEY periodically - invalidates all sessions