docs are cool

doc/CERTIFICATE_FORMATS.md

@@ -0,0 +1,136 @@
+# Understanding PEM vs P7B Certificates
+
+## Quick Summary
+
+**You CANNOT use P7B files to run your HTTPS server!**
+
+- **PEM files** = Run your server with HTTPS ✓
+- **P7B files** = Upload to Square Dashboard ✓
+
+## The Complete Picture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Your HTTPS Server                        │
+│                                                             │
+│  Uses: cert.pem + key.pem (PEM format)                    │
+│  Command: python src/test_server.py                        │
+│           --ssl-cert cert.pem                              │
+│           --ssl-key key.pem                                │
+│           --hook-port 8443                                 │
+└─────────────────────────────────────────────────────────────┘
+                         │
+                         │ HTTPS traffic
+                         ▼
+┌─────────────────────────────────────────────────────────────┐
+│                  Square's Servers                           │
+│                                                             │
+│  They verify your server using: cert.p7b                   │
+│  (which YOU upload to Square Dashboard)                    │
+└─────────────────────────────────────────────────────────────┘
+```
+
+## Why Two Formats?
+
+### PEM Format (.pem files)
+- **Contains**: Certificate AND private key (separate files)
+- **Used by**: Your HTTPS server to encrypt traffic
+- **Files needed**: 
+  - `cert.pem` - Your public certificate
+  - `key.pem` - Your private key (NEVER share this!)
+
+### P7B Format (.p7b files)  
+- **Contains**: Certificate ONLY (no private key)
+- **Used by**: Square to verify webhook signatures
+- **File**: `cert.p7b` - Uploaded to Square Dashboard
+
+## Step-by-Step Workflow
+
+### 1. Generate Certificates (PEM format)
+```powershell
+python generate_cert.py
+```
+**Creates:**
+- `cert.pem` - Your certificate (public)
+- `key.pem` - Your private key (secret!)
+
+### 2. Start Your HTTPS Server (uses PEM)
+```powershell
+python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443 --hook-ssl-only
+```
+**Server runs at:**
+- `https://localhost:8443/hook`
+
+### 3. Convert Certificate to P7B for Square
+```powershell
+python convert_to_p7b.py
+```
+**Creates:**
+- `cert.p7b` - Certificate in P7B format (for Square)
+
+### 4. Upload P7B to Square
+1. Go to: https://developer.squareup.com/apps
+2. Your App → **Webhooks** → **Manage Notification Settings**
+3. Upload `cert.p7b`
+
+### 5. Configure Square Webhook URL
+- Enter: `https://yourdomain.com:8443/hook`
+- Square will verify using the P7B you uploaded
+
+## Common Mistakes
+
+### ❌ Trying to use P7B with your server
+```powershell
+# THIS WON'T WORK!
+python src/test_server.py --ssl-cert cert.p7b --ssl-key key.p7b
+```
+**Error:** P7B doesn't contain the private key needed for HTTPS!
+
+### ✓ Correct usage
+```powershell
+# Server uses PEM files
+python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443
+
+# P7B file is uploaded to Square Dashboard (not used by your server)
+```
+
+## If You Only Have a P7B File
+
+If someone gave you a P7B file, you can extract the certificate:
+
+```powershell
+python convert_from_p7b.py your_cert.p7b --output cert.pem
+```
+
+**BUT:** You still need the private key (`key.pem`) to run HTTPS!
+
+If you don't have the private key, you must:
+1. Generate a new certificate pair: `python generate_cert.py`
+2. Or get the private key from whoever created the P7B
+
+## File Checklist
+
+To run your HTTPS server with Square webhooks, you need:
+
+- [ ] `cert.pem` - Certificate (PEM format) - For your server
+- [ ] `key.pem` - Private key (PEM format) - For your server  
+- [ ] `cert.p7b` - Certificate (P7B format) - Upload to Square
+
+Generate all three with:
+```powershell
+python generate_cert.py      # Creates cert.pem and key.pem
+python convert_to_p7b.py      # Creates cert.p7b from cert.pem
+```
+
+## Security Notes
+
+- **Never share `key.pem`** - This is your private key!
+- **`cert.pem` is public** - Safe to share
+- **`cert.p7b` is public** - You upload this to Square
+- Self-signed certificates are fine for testing but use Let's Encrypt for production
+
+## Need Help?
+
+See the detailed guides:
+- `HTTPS_SETUP.md` - Complete HTTPS setup instructions
+- `SQUARE_WEBHOOK_SETUP.md` - Square webhook configuration

doc/HTTPS_SETUP.md

@@ -9,6 +9,14 @@ python generate_cert.py
 
 This creates `cert.pem` and `key.pem` files.
 
+### 1.5. Convert to P7B for Square (Required for Square Webhooks)
+```powershell
+python convert_to_p7b.py
+```
+
+This creates `cert.p7b` which Square needs for webhook signature verification.
+Upload this P7B file to Square Developer Dashboard → Webhooks → Manage Notification Settings.
+
 ### 2. Run Server with HTTPS
 
 #### Option A: HTTPS for webhook only (recommended for development)
@@ -115,6 +123,39 @@ python src/test_server.py
 
 ## Square Webhook Configuration
 
+### Certificate Requirements
+Square requires a **P7B (PKCS#7)** certificate file for webhook signature verification.
+
+### Setup Steps:
+
+1. **Generate your certificate** (if you haven't already):
+   ```powershell
+   python generate_cert.py
+   ```
+
+2. **Convert PEM to P7B format**:
+   ```powershell
+   # Basic conversion
+   python convert_to_p7b.py
+   
+   # With intermediate certificates (production)
+   python convert_to_p7b.py --cert fullchain.pem --chain intermediate.pem --output square.p7b
+   ```
+
+3. **Upload to Square Dashboard**:
+   - Go to https://developer.squareup.com/
+   - Navigate to your application
+   - Click on "Webhooks"
+   - Click "Manage Notification Settings"
+   - Upload the `cert.p7b` file
+   - Square will use this to verify webhook signatures
+
+4. **Configure webhook URL in Square**:
+   - Use your public HTTPS URL: `https://yourdomain.com:8443/hook`
+   - Square requires valid SSL (not self-signed for production)
+
+### Testing with Square Webhooks:
+
 When setting up webhooks in Square Dashboard:
 1. Use your HTTPS URL: `https://yourdomain.com:8443/hook`
 2. Square requires valid SSL (not self-signed)

doc/NGINX_SETUP.md

@@ -0,0 +1,356 @@
+# Nginx Reverse Proxy Configuration for studspider
+
+## Overview
+
+When using nginx as a reverse proxy:
+- **Nginx handles HTTPS** (SSL/TLS termination)
+- **studspider runs on HTTP** internally (no SSL needed)
+- **nginx forwards requests** to studspider
+
+## Architecture
+
+```
+Internet (HTTPS) → nginx (443) → studspider (7432/HTTP)
+                      ↓
+            Handles SSL certificates
+            Handles HTTPS traffic
+            Forwards to HTTP backend
+```
+
+## Benefits
+
+✓ **No SSL certificates needed in studspider**
+✓ **nginx handles SSL efficiently**
+✓ **Easy certificate renewal** (Let's Encrypt)
+✓ **Better performance** (nginx is optimized for this)
+✓ **Additional security** (rate limiting, IP filtering, etc.)
+
+---
+
+## Configuration Examples
+
+### Basic Configuration (HTTP → HTTPS redirect + webhook proxy)
+
+Create `/etc/nginx/sites-available/studspider`:
+
+```nginx
+# Redirect HTTP to HTTPS
+server {
+    listen 80;
+    server_name studspider.sh www.studspider.sh;
+    
+    location / {
+        return 301 https://$host$request_uri;
+    }
+}
+
+# HTTPS server
+server {
+    listen 443 ssl http2;
+    server_name studspider.sh www.studspider.sh;
+
+    # SSL certificates (Let's Encrypt)
+    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
+    
+    # SSL configuration (modern, secure)
+    ssl_protocols TLSv1.2 TLSv1.3;
+    ssl_ciphers HIGH:!aNULL:!MD5;
+    ssl_prefer_server_ciphers on;
+
+    # Logs
+    access_log /var/log/nginx/studspider_access.log;
+    error_log /var/log/nginx/studspider_error.log;
+
+    # Webhook endpoint (Square webhooks)
+    location /hook {
+        proxy_pass http://localhost:7432/hook;
+        
+        # Forward headers
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        
+        # Timeouts
+        proxy_connect_timeout 60s;
+        proxy_send_timeout 60s;
+        proxy_read_timeout 60s;
+        
+        # Disable buffering for real-time responses
+        proxy_buffering off;
+    }
+
+    # Main API endpoint (optional)
+    location / {
+        proxy_pass http://localhost:7432/;
+        
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+    }
+}
+```
+
+### Separate Port Configuration (webhook on different port)
+
+```nginx
+server {
+    listen 443 ssl http2;
+    server_name studspider.sh;
+
+    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
+
+    # Main API
+    location / {
+        proxy_pass http://localhost:7432/;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+    }
+}
+
+# Separate server for webhooks on different port
+server {
+    listen 8443 ssl http2;
+    server_name studspider.sh;
+
+    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
+
+    # Webhook only
+    location /hook {
+        proxy_pass http://localhost:8080/hook;  # studspider webhook port
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+    }
+}
+```
+
+### With Rate Limiting (recommended for webhooks)
+
+```nginx
+# Define rate limit zone (outside server block)
+limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=10r/s;
+
+server {
+    listen 443 ssl http2;
+    server_name studspider.sh;
+
+    ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
+
+    # Webhook with rate limiting
+    location /hook {
+        # Rate limit: 10 requests per second per IP
+        limit_req zone=webhook_limit burst=20 nodelay;
+        
+        # Only allow POST requests
+        limit_except POST {
+            deny all;
+        }
+        
+        proxy_pass http://localhost:7432/hook;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+    }
+}
+```
+
+---
+
+## Setup Steps
+
+### 1. Install nginx (if not already installed)
+
+```bash
+# Ubuntu/Debian
+sudo apt update
+sudo apt install nginx
+
+# CentOS/RHEL
+sudo yum install nginx
+```
+
+### 2. Get SSL Certificate (Let's Encrypt - Free!)
+
+```bash
+# Install certbot
+sudo apt install certbot python3-certbot-nginx
+
+# Get certificate for your domain
+sudo certbot --nginx -d studspider.sh -d www.studspider.sh
+
+# Certbot will automatically:
+# 1. Generate certificate
+# 2. Configure nginx
+# 3. Set up auto-renewal
+```
+
+### 3. Create nginx Configuration
+
+```bash
+# Create config file
+sudo nano /etc/nginx/sites-available/studspider
+
+# Paste one of the configurations above
+
+# Create symbolic link to enable site
+sudo ln -s /etc/nginx/sites-available/studspider /etc/nginx/sites-enabled/
+
+# Test configuration
+sudo nginx -t
+
+# Reload nginx
+sudo systemctl reload nginx
+```
+
+### 4. Start studspider (HTTP only, no SSL needed!)
+
+```bash
+# Simple - single port
+python src/test_server.py --port 7432
+
+# Or with separate webhook port
+python src/test_server.py --port 7432 --hook-port 8080
+```
+
+### 5. Configure Square Webhook URL
+
+In Square Developer Dashboard:
+- **Webhook URL**: `https://studspider.sh/hook`
+- **Certificate**: Upload the P7B file (see below)
+
+### 6. Generate P7B for Square
+
+```bash
+# Convert your Let's Encrypt certificate to P7B for Square
+python convert_to_p7b.py --cert /etc/letsencrypt/live/studspider.sh/fullchain.pem --output square.p7b
+
+# Upload square.p7b to Square Dashboard
+```
+
+---
+
+## Systemd Service (Run studspider at boot)
+
+Create `/etc/systemd/system/studspider.service`:
+
+```ini
+[Unit]
+Description=studspider - LEGO Inventory Sync Service
+After=network.target
+
+[Service]
+Type=simple
+User=www-data
+WorkingDirectory=/opt/studspider
+Environment="PATH=/opt/studspider/.venv/bin"
+ExecStart=/opt/studspider/.venv/bin/python src/test_server.py --port 7432
+Restart=always
+RestartSec=10
+
+# Logging
+StandardOutput=append:/var/log/studspider/stdout.log
+StandardError=append:/var/log/studspider/stderr.log
+
+[Install]
+WantedBy=multi-user.target
+```
+
+Enable and start:
+
+```bash
+# Create log directory
+sudo mkdir -p /var/log/studspider
+sudo chown www-data:www-data /var/log/studspider
+
+# Enable service
+sudo systemctl enable studspider
+
+# Start service
+sudo systemctl start studspider
+
+# Check status
+sudo systemctl status studspider
+
+# View logs
+sudo journalctl -u studspider -f
+```
+
+---
+
+## Testing
+
+### Test nginx configuration
+```bash
+sudo nginx -t
+```
+
+### Test studspider locally
+```bash
+curl http://localhost:7432/hook -X POST -H "Content-Type: application/json" -d '{"test":"data"}'
+```
+
+### Test through nginx
+```bash
+curl https://studspider.sh/hook -X POST -H "Content-Type: application/json" -d '{"test":"data"}'
+```
+
+### Monitor logs
+```bash
+# nginx logs
+sudo tail -f /var/log/nginx/studspider_access.log
+sudo tail -f /var/log/nginx/studspider_error.log
+
+# studspider logs
+tail -f ./logs/studspider.log
+```
+
+---
+
+## Troubleshooting
+
+### 502 Bad Gateway
+- Check if studspider is running: `ps aux | grep test_server.py`
+- Check the port matches nginx config
+- Check firewall isn't blocking internal connections
+
+### Certificate Errors
+- Ensure Let's Encrypt renewal is working: `sudo certbot renew --dry-run`
+- Check certificate files exist and have correct permissions
+
+### Webhook Not Receiving
+- Verify nginx is forwarding: Check nginx access logs
+- Verify studspider is receiving: Check studspider logs
+- Test locally first, then through nginx
+
+### Connection Refused
+- Check studspider is listening: `sudo netstat -tlnp | grep 7432`
+- Check nginx proxy_pass points to correct host:port
+
+---
+
+## Summary
+
+### With nginx reverse proxy:
+
+✓ **studspider runs simple HTTP** (no SSL certificates needed)
+```bash
+python src/test_server.py --port 7432
+```
+
+✓ **nginx handles HTTPS** (manages certificates automatically)
+
+✓ **Square gets HTTPS webhook** (`https://studspider.sh/hook`)
+
+✓ **You upload P7B to Square** (converted from nginx's certificate)
+
+### This is the recommended production setup!