docs are cool

This commit is contained in:
C committed 2025-12-21 16:02:09 -08:00
1 parent 8c145c0d01
commit be7a112038
4 files changed
+533

No files matched your search

+136
View File
@@ -0,0 +1,136 @@
# Understanding PEM vs P7B Certificates
## Quick Summary
**You CANNOT use P7B files to run your HTTPS server!**
- **PEM files** = Run your server with HTTPS ✓
- **P7B files** = Upload to Square Dashboard ✓
## The Complete Picture
```
┌─────────────────────────────────────────────────────────────┐
│ Your HTTPS Server │
│ │
│ Uses: cert.pem + key.pem (PEM format) │
│ Command: python src/test_server.py │
│ --ssl-cert cert.pem │
│ --ssl-key key.pem │
│ --hook-port 8443 │
└─────────────────────────────────────────────────────────────┘
│
│ HTTPS traffic
▼
┌─────────────────────────────────────────────────────────────┐
│ Square's Servers │
│ │
│ They verify your server using: cert.p7b │
│ (which YOU upload to Square Dashboard) │
└─────────────────────────────────────────────────────────────┘
```
## Why Two Formats?
### PEM Format (.pem files)
- **Contains**: Certificate AND private key (separate files)
- **Used by**: Your HTTPS server to encrypt traffic
- **Files needed**:
- `cert.pem` - Your public certificate
- `key.pem` - Your private key (NEVER share this!)
### P7B Format (.p7b files)
- **Contains**: Certificate ONLY (no private key)
- **Used by**: Square to verify webhook signatures
- **File**: `cert.p7b` - Uploaded to Square Dashboard
## Step-by-Step Workflow
### 1. Generate Certificates (PEM format)
```powershell
python generate_cert.py
```
**Creates:**
- `cert.pem` - Your certificate (public)
- `key.pem` - Your private key (secret!)
### 2. Start Your HTTPS Server (uses PEM)
```powershell
python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443 --hook-ssl-only
```
**Server runs at:**
- `https://localhost:8443/hook`
### 3. Convert Certificate to P7B for Square
```powershell
python convert_to_p7b.py
```
**Creates:**
- `cert.p7b` - Certificate in P7B format (for Square)
### 4. Upload P7B to Square
1. Go to: https://developer.squareup.com/apps
2. Your App → **Webhooks** → **Manage Notification Settings**
3. Upload `cert.p7b`
### 5. Configure Square Webhook URL
- Enter: `https://yourdomain.com:8443/hook`
- Square will verify using the P7B you uploaded
## Common Mistakes
### ❌ Trying to use P7B with your server
```powershell
# THIS WON'T WORK!
python src/test_server.py --ssl-cert cert.p7b --ssl-key key.p7b
```
**Error:** P7B doesn't contain the private key needed for HTTPS!
### ✓ Correct usage
```powershell
# Server uses PEM files
python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443
# P7B file is uploaded to Square Dashboard (not used by your server)
```
## If You Only Have a P7B File
If someone gave you a P7B file, you can extract the certificate:
```powershell
python convert_from_p7b.py your_cert.p7b --output cert.pem
```
**BUT:** You still need the private key (`key.pem`) to run HTTPS!
If you don't have the private key, you must:
1. Generate a new certificate pair: `python generate_cert.py`
2. Or get the private key from whoever created the P7B
## File Checklist
To run your HTTPS server with Square webhooks, you need:
- [ ] `cert.pem` - Certificate (PEM format) - For your server
- [ ] `key.pem` - Private key (PEM format) - For your server
- [ ] `cert.p7b` - Certificate (P7B format) - Upload to Square
Generate all three with:
```powershell
python generate_cert.py # Creates cert.pem and key.pem
python convert_to_p7b.py # Creates cert.p7b from cert.pem
```
## Security Notes
- **Never share `key.pem`** - This is your private key!
- **`cert.pem` is public** - Safe to share
- **`cert.p7b` is public** - You upload this to Square
- Self-signed certificates are fine for testing but use Let's Encrypt for production
## Need Help?
See the detailed guides:
- `HTTPS_SETUP.md` - Complete HTTPS setup instructions
- `SQUARE_WEBHOOK_SETUP.md` - Square webhook configuration
+166
View File
@@ -0,0 +1,166 @@
# HTTPS Webhook Setup Guide
## Quick Start (Self-Signed Certificate for Testing)
### 1. Generate Certificate
```powershell
python generate_cert.py
```
This creates `cert.pem` and `key.pem` files.
### 1.5. Convert to P7B for Square (Required for Square Webhooks)
```powershell
python convert_to_p7b.py
```
This creates `cert.p7b` which Square needs for webhook signature verification.
Upload this P7B file to Square Developer Dashboard → Webhooks → Manage Notification Settings.
### 2. Run Server with HTTPS
#### Option A: HTTPS for webhook only (recommended for development)
```powershell
python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443 --hook-ssl-only
```
- Main server: `http://localhost:7432/` (HTTP)
- Webhook: `https://localhost:8443/hook` (HTTPS)
#### Option B: HTTPS for everything
```powershell
python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443
```
- Main server: `https://localhost:7432/` (HTTPS)
- Webhook: `https://localhost:8443/hook` (HTTPS)
#### Option C: Single port with HTTPS
```powershell
python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem
```
- Everything on: `https://localhost:7432/`
### 3. Test the Webhook
Update `src/test_webhook.py` to use HTTPS:
```python
url = "https://localhost:8443/hook"
```
Then run:
```powershell
python src/test_webhook.py
```
## Production Setup (with Real Certificate)
### Option 1: Let's Encrypt (Free, Recommended)
On Linux/WSL:
```bash
# Install certbot
sudo apt-get install certbot
# Generate certificate (replace yourdomain.com)
sudo certbot certonly --standalone -d yourdomain.com
# Certificates will be at:
# /etc/letsencrypt/live/yourdomain.com/fullchain.pem
# /etc/letsencrypt/live/yourdomain.com/privkey.pem
# Run server
python src/test_server.py \
--ssl-cert /etc/letsencrypt/live/yourdomain.com/fullchain.pem \
--ssl-key /etc/letsencrypt/live/yourdomain.com/privkey.pem \
--host 0.0.0.0 \
--hook-port 443 \
--hook-ssl-only
```
### Option 2: Reverse Proxy (nginx/Caddy)
Run studspider on HTTP internally, use nginx/Caddy for HTTPS:
```nginx
# nginx config
server {
listen 443 ssl;
server_name yourdomain.com;
ssl_certificate /path/to/cert.pem;
ssl_certificate_key /path/to/key.pem;
location /hook {
proxy_pass http://localhost:7432/hook;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
```
Then run studspider normally:
```powershell
python src/test_server.py
```
## Troubleshooting
### "OpenSSL not found" Error
- **Windows**: Install from https://slproweb.com/products/Win32OpenSSL.html
- **Alternative**: Use Git Bash or WSL which include OpenSSL
### Self-Signed Certificate Warnings
- Normal for development
- Use `--insecure` flag with curl/tools for testing
- For production, get a proper certificate from Let's Encrypt
### Port 443 Already in Use
- Use a different port like 8443
- Or stop the service using port 443
### Certificate File Not Found
- Use absolute paths: `--ssl-cert C:\path\to\cert.pem`
- Or run from the directory containing the certificates
## Square Webhook Configuration
### Certificate Requirements
Square requires a **P7B (PKCS#7)** certificate file for webhook signature verification.
### Setup Steps:
1. **Generate your certificate** (if you haven't already):
```powershell
python generate_cert.py
```
2. **Convert PEM to P7B format**:
```powershell
# Basic conversion
python convert_to_p7b.py
# With intermediate certificates (production)
python convert_to_p7b.py --cert fullchain.pem --chain intermediate.pem --output square.p7b
```
3. **Upload to Square Dashboard**:
- Go to https://developer.squareup.com/
- Navigate to your application
- Click on "Webhooks"
- Click "Manage Notification Settings"
- Upload the `cert.p7b` file
- Square will use this to verify webhook signatures
4. **Configure webhook URL in Square**:
- Use your public HTTPS URL: `https://yourdomain.com:8443/hook`
- Square requires valid SSL (not self-signed for production)
### Testing with Square Webhooks:
When setting up webhooks in Square Dashboard:
1. Use your HTTPS URL: `https://yourdomain.com:8443/hook`
2. Square requires valid SSL (not self-signed)
3. Consider using ngrok for local testing:
```powershell
ngrok http 8443
```
Then use the ngrok HTTPS URL in Square Dashboard
+356
View File
@@ -0,0 +1,356 @@
# Nginx Reverse Proxy Configuration for studspider
## Overview
When using nginx as a reverse proxy:
- **Nginx handles HTTPS** (SSL/TLS termination)
- **studspider runs on HTTP** internally (no SSL needed)
- **nginx forwards requests** to studspider
## Architecture
```
Internet (HTTPS) → nginx (443) → studspider (7432/HTTP)
↓
Handles SSL certificates
Handles HTTPS traffic
Forwards to HTTP backend
```
## Benefits
✓ **No SSL certificates needed in studspider**
✓ **nginx handles SSL efficiently**
✓ **Easy certificate renewal** (Let's Encrypt)
✓ **Better performance** (nginx is optimized for this)
✓ **Additional security** (rate limiting, IP filtering, etc.)
---
## Configuration Examples
### Basic Configuration (HTTP → HTTPS redirect + webhook proxy)
Create `/etc/nginx/sites-available/studspider`:
```nginx
# Redirect HTTP to HTTPS
server {
listen 80;
server_name studspider.sh www.studspider.sh;
location / {
return 301 https://$host$request_uri;
}
}
# HTTPS server
server {
listen 443 ssl http2;
server_name studspider.sh www.studspider.sh;
# SSL certificates (Let's Encrypt)
ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
# SSL configuration (modern, secure)
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
# Logs
access_log /var/log/nginx/studspider_access.log;
error_log /var/log/nginx/studspider_error.log;
# Webhook endpoint (Square webhooks)
location /hook {
proxy_pass http://localhost:7432/hook;
# Forward headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
# Disable buffering for real-time responses
proxy_buffering off;
}
# Main API endpoint (optional)
location / {
proxy_pass http://localhost:7432/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
### Separate Port Configuration (webhook on different port)
```nginx
server {
listen 443 ssl http2;
server_name studspider.sh;
ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
# Main API
location / {
proxy_pass http://localhost:7432/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
}
# Separate server for webhooks on different port
server {
listen 8443 ssl http2;
server_name studspider.sh;
ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
# Webhook only
location /hook {
proxy_pass http://localhost:8080/hook; # studspider webhook port
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
}
```
### With Rate Limiting (recommended for webhooks)
```nginx
# Define rate limit zone (outside server block)
limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=10r/s;
server {
listen 443 ssl http2;
server_name studspider.sh;
ssl_certificate /etc/letsencrypt/live/studspider.sh/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/studspider.sh/privkey.pem;
# Webhook with rate limiting
location /hook {
# Rate limit: 10 requests per second per IP
limit_req zone=webhook_limit burst=20 nodelay;
# Only allow POST requests
limit_except POST {
deny all;
}
proxy_pass http://localhost:7432/hook;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
}
```
---
## Setup Steps
### 1. Install nginx (if not already installed)
```bash
# Ubuntu/Debian
sudo apt update
sudo apt install nginx
# CentOS/RHEL
sudo yum install nginx
```
### 2. Get SSL Certificate (Let's Encrypt - Free!)
```bash
# Install certbot
sudo apt install certbot python3-certbot-nginx
# Get certificate for your domain
sudo certbot --nginx -d studspider.sh -d www.studspider.sh
# Certbot will automatically:
# 1. Generate certificate
# 2. Configure nginx
# 3. Set up auto-renewal
```
### 3. Create nginx Configuration
```bash
# Create config file
sudo nano /etc/nginx/sites-available/studspider
# Paste one of the configurations above
# Create symbolic link to enable site
sudo ln -s /etc/nginx/sites-available/studspider /etc/nginx/sites-enabled/
# Test configuration
sudo nginx -t
# Reload nginx
sudo systemctl reload nginx
```
### 4. Start studspider (HTTP only, no SSL needed!)
```bash
# Simple - single port
python src/test_server.py --port 7432
# Or with separate webhook port
python src/test_server.py --port 7432 --hook-port 8080
```
### 5. Configure Square Webhook URL
In Square Developer Dashboard:
- **Webhook URL**: `https://studspider.sh/hook`
- **Certificate**: Upload the P7B file (see below)
### 6. Generate P7B for Square
```bash
# Convert your Let's Encrypt certificate to P7B for Square
python convert_to_p7b.py --cert /etc/letsencrypt/live/studspider.sh/fullchain.pem --output square.p7b
# Upload square.p7b to Square Dashboard
```
---
## Systemd Service (Run studspider at boot)
Create `/etc/systemd/system/studspider.service`:
```ini
[Unit]
Description=studspider - LEGO Inventory Sync Service
After=network.target
[Service]
Type=simple
User=www-data
WorkingDirectory=/opt/studspider
Environment="PATH=/opt/studspider/.venv/bin"
ExecStart=/opt/studspider/.venv/bin/python src/test_server.py --port 7432
Restart=always
RestartSec=10
# Logging
StandardOutput=append:/var/log/studspider/stdout.log
StandardError=append:/var/log/studspider/stderr.log
[Install]
WantedBy=multi-user.target
```
Enable and start:
```bash
# Create log directory
sudo mkdir -p /var/log/studspider
sudo chown www-data:www-data /var/log/studspider
# Enable service
sudo systemctl enable studspider
# Start service
sudo systemctl start studspider
# Check status
sudo systemctl status studspider
# View logs
sudo journalctl -u studspider -f
```
---
## Testing
### Test nginx configuration
```bash
sudo nginx -t
```
### Test studspider locally
```bash
curl http://localhost:7432/hook -X POST -H "Content-Type: application/json" -d '{"test":"data"}'
```
### Test through nginx
```bash
curl https://studspider.sh/hook -X POST -H "Content-Type: application/json" -d '{"test":"data"}'
```
### Monitor logs
```bash
# nginx logs
sudo tail -f /var/log/nginx/studspider_access.log
sudo tail -f /var/log/nginx/studspider_error.log
# studspider logs
tail -f ./logs/studspider.log
```
---
## Troubleshooting
### 502 Bad Gateway
- Check if studspider is running: `ps aux | grep test_server.py`
- Check the port matches nginx config
- Check firewall isn't blocking internal connections
### Certificate Errors
- Ensure Let's Encrypt renewal is working: `sudo certbot renew --dry-run`
- Check certificate files exist and have correct permissions
### Webhook Not Receiving
- Verify nginx is forwarding: Check nginx access logs
- Verify studspider is receiving: Check studspider logs
- Test locally first, then through nginx
### Connection Refused
- Check studspider is listening: `sudo netstat -tlnp | grep 7432`
- Check nginx proxy_pass points to correct host:port
---
## Summary
### With nginx reverse proxy:
✓ **studspider runs simple HTTP** (no SSL certificates needed)
```bash
python src/test_server.py --port 7432
```
✓ **nginx handles HTTPS** (manages certificates automatically)
✓ **Square gets HTTPS webhook** (`https://studspider.sh/hook`)
✓ **You upload P7B to Square** (converted from nginx's certificate)
### This is the recommended production setup!
+191
View File
@@ -0,0 +1,191 @@
# Square Webhook Setup - Quick Reference
## Complete Setup Process
### Step 1: Generate SSL Certificate
```powershell
# Generate PEM certificate
python generate_cert.py
# Creates: cert.pem, key.pem
```
### Step 2: Convert to P7B for Square
```powershell
# Convert PEM to P7B (PKCS#7) format
python convert_to_p7b.py
# Creates: cert.p7b
```
### Step 3: Start HTTPS Server
```powershell
# Start webhook server with HTTPS
python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443 --hook-ssl-only
# Server URLs:
# Main: http://localhost:7432/
# Webhook: https://localhost:8443/hook (or https://yourdomain.com:8443/hook)
```
### Step 4: Configure Square Webhook
1. **Upload P7B Certificate to Square:**
- Go to: https://developer.squareup.com/apps
- Select your application
- Navigate to: **Webhooks** → **Manage Notification Settings**
- Upload `cert.p7b` file
- This allows Square to verify webhook signatures
2. **Add Webhook Subscription:**
- Still in Webhooks section
- Click **"Add Subscription"**
- Enter URL: `https://yourdomain.com:8443/hook`
- Select events to subscribe to (e.g., `inventory.count.updated`)
- Click **Save**
### Step 5: Test the Webhook
```powershell
# Test from command line
python src/test_webhook.py
# Or manually with curl
curl -X POST https://localhost:8443/hook \
-H "Content-Type: application/json" \
-d '{"type":"inventory.count.updated","merchant_id":"test"}'
```
## Production Deployment
### Using Let's Encrypt (Recommended)
```bash
# 1. Get Let's Encrypt certificate (Linux/WSL)
sudo certbot certonly --standalone -d yourdomain.com
# 2. Convert to P7B for Square
python convert_to_p7b.py \
--cert /etc/letsencrypt/live/yourdomain.com/fullchain.pem \
--output square.p7b
# 3. Upload square.p7b to Square Dashboard
# 4. Run server
python src/test_server.py \
--ssl-cert /etc/letsencrypt/live/yourdomain.com/fullchain.pem \
--ssl-key /etc/letsencrypt/live/yourdomain.com/privkey.pem \
--host 0.0.0.0 \
--hook-port 443 \
--hook-ssl-only
```
### Using ngrok (Easy Testing)
```powershell
# Terminal 1: Start your server (HTTP is fine)
python src/test_server.py --hook-port 8080
# Terminal 2: Expose with ngrok (provides HTTPS automatically)
ngrok http 8080
# Use the ngrok HTTPS URL in Square:
# Example: https://abc123.ngrok.io/hook
# For P7B certificate with ngrok:
# ngrok provides its own certificate, so you don't need to upload one to Square
```
## Webhook Event Types
Common Square webhook events you can subscribe to:
- `inventory.count.updated` - Inventory quantity changes
- `catalog.version.updated` - Catalog item updates
- `payment.created` - New payment received
- `payment.updated` - Payment status changes
- `order.created` - New order created
- `order.updated` - Order status changes
## Troubleshooting
### Certificate Upload Fails
- Ensure file is in P7B format (not PEM)
- File must be valid PKCS#7 structure
- Check file isn't corrupted: `openssl pkcs7 -print_certs -in cert.p7b`
### Webhook Not Receiving Data
1. Check server is running and accessible from internet
2. Verify URL in Square Dashboard is correct
3. Check firewall allows incoming traffic on webhook port
4. Review server logs: `./logs/studspider.log`
### SSL Certificate Errors
- Self-signed certificates won't work with Square in production
- Use Let's Encrypt or commercial CA for production
- Use ngrok for quick testing (provides valid HTTPS)
### Testing Locally
```powershell
# Square can't reach localhost - use one of these:
# 1. ngrok (easiest)
ngrok http 8443
# 2. Cloudflare Tunnel
cloudflared tunnel --url http://localhost:8443
# 3. Deploy to cloud (AWS, Azure, DigitalOcean, etc.)
```
## Verifying Webhook Signatures
Square signs webhook requests. Your server should verify signatures:
```python
# TODO: Add to hook_handler in test_server.py
import hmac
import hashlib
def verify_square_signature(payload: bytes, signature: str, webhook_signature_key: str) -> bool:
"""Verify Square webhook signature"""
computed_signature = hmac.new(
webhook_signature_key.encode('utf-8'),
payload,
hashlib.sha256
).digest()
expected_signature = base64.b64decode(signature)
return hmac.compare_digest(computed_signature, expected_signature)
```
Get your webhook signature key from Square Dashboard → Webhooks → Signature Key.
## Environment Variables
Add to your `.env` file:
```env
# Square Webhook Configuration
SQUARE_WEBHOOK_SIGNATURE_KEY=your_signature_key_from_square_dashboard
# SSL Certificate Paths (production)
SSL_CERT_PATH=/path/to/fullchain.pem
SSL_KEY_PATH=/path/to/privkey.pem
```
## Commands Quick Reference
```powershell
# Generate certificate
python generate_cert.py
# Convert to P7B
python convert_to_p7b.py
# Start HTTPS server
python src/test_server.py --ssl-cert cert.pem --ssl-key key.pem --hook-port 8443 --hook-ssl-only
# Test webhook
python src/test_webhook.py
# Start with ngrok
ngrok http 8443
```