studspider/doc/NGINX_SETUP.md

Nginx Reverse Proxy Configuration for studspider

Overview

When using nginx as a reverse proxy:

• Nginx handles HTTPS (SSL/TLS termination)
• studspider runs on HTTP internally (no SSL needed)
• nginx forwards requests to studspider

Architecture

Internet (HTTPS) → nginx (443) → studspider (7432/HTTP)
                      ↓
            Handles SSL certificates
            Handles HTTPS traffic
            Forwards to HTTP backend

Benefits

✓ No SSL certificates needed in studspider ✓ nginx handles SSL efficiently ✓ Easy certificate renewal (Let's Encrypt) ✓ Better performance (nginx is optimized for this) ✓ Additional security (rate limiting, IP filtering, etc.)

---

Configuration Examples

Basic Configuration (HTTP → HTTPS redirect + webhook proxy)

Create /etc/nginx/sites-available/studspider:

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name studspider.sh www.studspider.sh;
    
    location / {
        return 301 https://$host$request_uri;
    }
}

# HTTPS server
server {
    listen 443 ssl http2;
    server_name studspider.sh www.studspider.sh;

    # SSL certificates (Let's Encrypt)
    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
    
    # SSL configuration (modern, secure)
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Logs
    access_log /var/log/nginx/studspider_access.log;
    error_log /var/log/nginx/studspider_error.log;

    # Webhook endpoint (Square webhooks)
    location /hook {
        proxy_pass http://localhost:7432/hook;
        
        # Forward headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Timeouts
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
        
        # Disable buffering for real-time responses
        proxy_buffering off;
    }

    # Main API endpoint (optional)
    location / {
        proxy_pass http://localhost:7432/;
        
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Separate Port Configuration (webhook on different port)

server {
    listen 443 ssl http2;
    server_name studspider.sh;

    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;

    # Main API
    location / {
        proxy_pass http://localhost:7432/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
    }
}

# Separate server for webhooks on different port
server {
    listen 8443 ssl http2;
    server_name studspider.sh;

    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;

    # Webhook only
    location /hook {
        proxy_pass http://localhost:8080/hook;  # studspider webhook port
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
    }
}

With Rate Limiting (recommended for webhooks)

# Define rate limit zone (outside server block)
limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=10r/s;

server {
    listen 443 ssl http2;
    server_name studspider.sh;

    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;

    # Webhook with rate limiting
    location /hook {
        # Rate limit: 10 requests per second per IP
        limit_req zone=webhook_limit burst=20 nodelay;
        
        # Only allow POST requests
        limit_except POST {
            deny all;
        }
        
        proxy_pass http://localhost:7432/hook;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
    }
}

---

Setup Steps

1. Install nginx (if not already installed)

# Ubuntu/Debian
sudo apt update
sudo apt install nginx

# CentOS/RHEL
sudo yum install nginx

2. Get SSL Certificate (Let's Encrypt - Free!)

# Install certbot
sudo apt install certbot python3-certbot-nginx

# Get certificate for your domain
sudo certbot --nginx -d studspider.sh -d www.studspider.sh

# Certbot will automatically:
# 1. Generate certificate
# 2. Configure nginx
# 3. Set up auto-renewal

3. Create nginx Configuration

# Create config file
sudo nano /etc/nginx/sites-available/studspider

# Paste one of the configurations above

# Create symbolic link to enable site
sudo ln -s /etc/nginx/sites-available/studspider /etc/nginx/sites-enabled/

# Test configuration
sudo nginx -t

# Reload nginx
sudo systemctl reload nginx

4. Start studspider (HTTP only, no SSL needed!)

# Simple - single port
python src/test_server.py --port 7432

# Or with separate webhook port
python src/test_server.py --port 7432 --hook-port 8080

5. Configure Square Webhook URL

In Square Developer Dashboard:

• Webhook URL: https://studspider.sh/hook
• Certificate: Upload the P7B file (see below)

6. Generate P7B for Square

# Convert your Let's Encrypt certificate to P7B for Square
python convert_to_p7b.py --cert /etc/letsencrypt/live/studspider.sh/fullchain.pem --output square.p7b

# Upload square.p7b to Square Dashboard

---

Systemd Service (Run studspider at boot)

Create /etc/systemd/system/studspider.service:

[Unit]
Description=studspider - LEGO Inventory Sync Service
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/opt/studspider
Environment="PATH=/opt/studspider/.venv/bin"
ExecStart=/opt/studspider/.venv/bin/python src/test_server.py --port 7432
Restart=always
RestartSec=10

# Logging
StandardOutput=append:/var/log/studspider/stdout.log
StandardError=append:/var/log/studspider/stderr.log

[Install]
WantedBy=multi-user.target

Enable and start:

# Create log directory
sudo mkdir -p /var/log/studspider
sudo chown www-data:www-data /var/log/studspider

# Enable service
sudo systemctl enable studspider

# Start service
sudo systemctl start studspider

# Check status
sudo systemctl status studspider

# View logs
sudo journalctl -u studspider -f

---

Testing

Test nginx configuration

sudo nginx -t

Test studspider locally

curl http://localhost:7432/hook -X POST -H "Content-Type: application/json" -d '{"test":"data"}'

Test through nginx

curl https://studspider.sh/hook -X POST -H "Content-Type: application/json" -d '{"test":"data"}'

Monitor logs

# nginx logs
sudo tail -f /var/log/nginx/studspider_access.log
sudo tail -f /var/log/nginx/studspider_error.log

# studspider logs
tail -f ./logs/studspider.log

---

Troubleshooting

502 Bad Gateway

• Check if studspider is running: ps aux | grep test_server.py
• Check the port matches nginx config
• Check firewall isn't blocking internal connections

Certificate Errors

• Ensure Let's Encrypt renewal is working: sudo certbot renew --dry-run
• Check certificate files exist and have correct permissions

Webhook Not Receiving

• Verify nginx is forwarding: Check nginx access logs
• Verify studspider is receiving: Check studspider logs
• Test locally first, then through nginx

Connection Refused

• Check studspider is listening: sudo netstat -tlnp | grep 7432
• Check nginx proxy_pass points to correct host:port

---

Summary

With nginx reverse proxy:

✓ studspider runs simple HTTP (no SSL certificates needed)

python src/test_server.py --port 7432

✓ nginx handles HTTPS (manages certificates automatically)

✓ Square gets HTTPS webhook (https://studspider.sh/hook)

✓ You upload P7B to Square (converted from nginx's certificate)

This is the recommended production setup!