feat(skills): add coruna-lab-migrate and coruna-lab-cleanup skills
- coruna-lab-migrate: server migration SOP (code/db/file transfer, supervisor setup, DNS cutover, verification, common pitfalls) - coruna-lab-cleanup: disk cleanup skill with disk_cleanup.sh and cleanup_scanned_photos.php scripts Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
root
@@ -0,0 +1,290 @@
|
||||
---
|
||||
name: coruna-lab-migrate
|
||||
description: >-
|
||||
Migrate the coruna-lab (Coruna Lab) Laravel project from one server to another
|
||||
on BT Panel (宝塔面板). Covers code deployment, database dump/restore, file
|
||||
storage transfer (photos/ds-results/inbox), supervisor queue worker setup,
|
||||
DNS cutover, and post-migration verification. Use when the user asks to
|
||||
migrate coruna-lab to a new server, move coruna-lab to another machine,
|
||||
换服务器, 迁移机器, 搬家, or set up a fresh coruna-lab deployment.
|
||||
Trigger keywords: coruna-lab 迁移, 迁移服务器, 换机器, migrate coruna-lab,
|
||||
server migration, 搬家, new server setup.
|
||||
---
|
||||
|
||||
# coruna-lab Server Migration
|
||||
|
||||
Migrate the **coruna-lab** Laravel project between BT Panel (宝塔面板) servers.
|
||||
|
||||
## Architecture overview
|
||||
|
||||
```
|
||||
Old Server New Server
|
||||
┌─────────────────────┐ ┌─────────────────────┐
|
||||
│ BT Panel + Nginx │ │ BT Panel + Nginx │
|
||||
│ PHP 8.2 │ rsync │ PHP 8.2 │
|
||||
│ MySQL (coruna DB) │ ────────> │ MySQL (coruna DB) │
|
||||
│ Redis │ │ Redis │
|
||||
│ Supervisor (workers)│ │ Supervisor (workers)│
|
||||
│ storage/app/private │ tar+ssh │ storage/app/private │
|
||||
│ c2/photos/ (155G+) │ ────────> │ c2/photos/ │
|
||||
└─────────────────────┘ └─────────────────────┘
|
||||
```
|
||||
|
||||
## Key paths & services
|
||||
|
||||
| Item | Path / Command |
|
||||
|------|----------------|
|
||||
| Project root | `/www/wwwroot/coruna-lab` |
|
||||
| PHP | `/www/server/php/82/bin/php` |
|
||||
| artisan | `sudo -u www /www/server/php/82/bin/php artisan` |
|
||||
| Supervisor | `sudo /www/server/panel/pyenv/bin/supervisorctl` |
|
||||
| Storage | `storage/app/private/c2/{photos,ds-results,ds-chunks,ds-notes,inbox,photo-previews,plugin-sessions}` |
|
||||
| Logs | `public/log/{xxbb,ds,transfer,c2}/` |
|
||||
| Supervisor profiles | `/www/server/panel/plugin/supervisor/profile/*.ini` |
|
||||
|
||||
## Migration phases
|
||||
|
||||
### Phase 1: Prepare new server
|
||||
|
||||
1. Install BT Panel, PHP 8.2, MySQL, Redis, Nginx on the new server.
|
||||
2. Create MySQL database and user:
|
||||
```sql
|
||||
CREATE DATABASE coruna CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
CREATE USER 'coruna'@'localhost' IDENTIFIED BY '<password>';
|
||||
GRANT ALL ON coruna.* TO 'coruna'@'localhost';
|
||||
FLUSH PRIVILEGES;
|
||||
```
|
||||
3. Create the site in BT Panel (point domain to `/www/wwwroot/coruna-lab`).
|
||||
|
||||
### Phase 2: Deploy code
|
||||
|
||||
The new server has **no git** — deploy via tarball from a machine that has the repo:
|
||||
|
||||
```bash
|
||||
# On the machine with git access:
|
||||
cd /www/wwwroot/coruna-lab
|
||||
git archive --format=tar HEAD | gzip > /tmp/coruna-lab-code.tar.gz
|
||||
scp /tmp/coruna-lab-code.tar.gz ubuntu@<new-server>:/tmp/
|
||||
|
||||
# On the new server:
|
||||
sudo mkdir -p /www/wwwroot/coruna-lab
|
||||
sudo tar -xzf /tmp/coruna-lab-code.tar.gz -C /www/wwwroot/coruna-lab
|
||||
sudo chown -R www:www /www/wwwroot/coruna-lab
|
||||
cd /www/wwwroot/coruna-lab
|
||||
composer install --no-dev --optimize-autoloader
|
||||
```
|
||||
|
||||
### Phase 3: Database migration
|
||||
|
||||
```bash
|
||||
# On old server: dump
|
||||
mysqldump -ucoruna -p<old-pass> coruna --single-transaction --routines > /tmp/coruna.sql
|
||||
scp /tmp/coruna.sql ubuntu@<new-server>:/tmp/
|
||||
|
||||
# On new server: import
|
||||
mysql -ucoruna -p<new-pass> coruna < /tmp/coruna.sql
|
||||
```
|
||||
|
||||
Run migrations on the new server:
|
||||
```bash
|
||||
cd /www/wwwroot/coruna-lab
|
||||
sudo -u www /www/server/php/82/bin/php artisan migrate --force
|
||||
```
|
||||
|
||||
### Phase 4: Configure .env
|
||||
|
||||
Copy `.env` from old server, update for new server:
|
||||
```bash
|
||||
# Key settings to verify/update:
|
||||
APP_URL=<new-domain>
|
||||
DB_PASSWORD=<new-db-pass>
|
||||
REDIS_HOST=127.0.0.1
|
||||
QUEUE_CONNECTION=redis
|
||||
# Keep these from old .env:
|
||||
CORUNA_OFFICIAL_ALBUM_STORAGE=1
|
||||
INTERCEPT_DEVICE_KEYS=...
|
||||
INTERCEPT_BOT_TOKEN=...
|
||||
INTERCEPT_CHAT_ID=...
|
||||
```
|
||||
|
||||
Generate app key if needed (usually keep the old one):
|
||||
```bash
|
||||
sudo -u www /www/server/php/82/bin/php artisan key:generate
|
||||
```
|
||||
|
||||
### Phase 5: Transfer file storage
|
||||
|
||||
The `c2/` directory can be 200G+. Use `tar + ssh` for reliability with large file counts:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# transfer_locked.sh — run on OLD server
|
||||
NEW_HOST="ubuntu@<new-server>"
|
||||
SSH_KEY="/path/to/key.pem"
|
||||
SRC="/www/wwwroot/coruna-lab/storage/app/private/c2"
|
||||
DST="/www/wwwroot/coruna-lab/storage/app/private/c2"
|
||||
|
||||
for dir in photos photo-previews ds-chunks ds-notes ds-results plugin-sessions inbox; do
|
||||
echo "Transferring $dir..."
|
||||
tar -C "$SRC" -cf - "$dir" | ssh -i "$SSH_KEY" $NEW_HOST "sudo tar -C '$DST' -xf -"
|
||||
done
|
||||
```
|
||||
|
||||
**Important**: Transfer `photos/` last (largest, 155G+). Use `flock` to prevent
|
||||
duplicate runs. Monitor progress with `find ... | wc -l` on both servers.
|
||||
|
||||
### Phase 6: Set up supervisor workers
|
||||
|
||||
**This is the most critical step** — missing workers cause silent data loss.
|
||||
|
||||
Create one `.ini` file per queue in `/www/server/panel/plugin/supervisor/profile/`:
|
||||
|
||||
| File | Queue | Command |
|
||||
|------|-------|---------|
|
||||
| `queue.ini` | default | `artisan queue:work redis --sleep=1 --tries=3 --timeout=90 --max-time=3600` |
|
||||
| `ocr.ini` | ocr | `artisan queue:work redis --queue=ocr --sleep=1 --tries=1 --timeout=90 --max-jobs=100` |
|
||||
| `keystore.ini` | keystore | `artisan queue:work keystore --sleep=1 --tries=1 --timeout=320 --max-time=3600` |
|
||||
| `telegram.ini` | telegram | `artisan queue:work telegram --sleep=1 --tries=3 --timeout=30 --max-time=3600` |
|
||||
| `transfer.ini` | transfer | `artisan queue:work transfer --sleep=1 --tries=1 --timeout=200 --max-time=3600` |
|
||||
| **`extract.ini`** | **extract** | `artisan queue:work redis --queue=extract --sleep=1 --tries=1 --timeout=200 --max-jobs=100` |
|
||||
|
||||
> **⚠️ CRITICAL: `extract.ini` is easily missed.** Without it, photo archives
|
||||
> pile up in `inbox/` and the `extract` Redis queue backs up indefinitely.
|
||||
> Photos appear to stop storing even though `/t` requests keep arriving.
|
||||
|
||||
Template for `extract.ini` (others follow the same pattern):
|
||||
```ini
|
||||
[program:extract]
|
||||
command=/www/server/php/82/bin/php -d memory_limit=256M artisan queue:work redis --queue=extract --sleep=1 --tries=1 --timeout=200 --max-jobs=100
|
||||
directory=/www/wwwroot/coruna-lab/
|
||||
autorestart=true
|
||||
startsecs=3
|
||||
startretries=3
|
||||
stdout_logfile=/www/server/panel/plugin/supervisor/log/extract.out.log
|
||||
stderr_logfile=/www/server/panel/plugin/supervisor/log/extract.err.log
|
||||
stdout_logfile_maxbytes=2MB
|
||||
stderr_logfile_maxbytes=2MB
|
||||
user=www
|
||||
priority=999
|
||||
numprocs=2
|
||||
process_name=%(program_name)s_%(process_num)02d
|
||||
```
|
||||
|
||||
Load and start all workers:
|
||||
```bash
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl reread
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl update
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl start all
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl status
|
||||
```
|
||||
|
||||
### Phase 7: Clear caches & restart PHP-FPM
|
||||
|
||||
```bash
|
||||
cd /www/wwwroot/coruna-lab
|
||||
sudo -u www /www/server/php/82/bin/php artisan config:clear
|
||||
sudo -u www /www/server/php/82/bin/php artisan route:clear
|
||||
sudo -u www /www/server/php/82/bin/php artisan view:clear
|
||||
sudo -u www /www/server/php/82/bin/php artisan cache:clear
|
||||
# Restart PHP-FPM
|
||||
sudo /etc/init.d/php-fpm-82 restart
|
||||
```
|
||||
|
||||
> **⚠️ `view:clear` is critical after code updates.** Stale compiled Blade
|
||||
> templates in `storage/framework/views/` cause 500 errors when new routes
|
||||
> are referenced but old compiled cache doesn't have them.
|
||||
|
||||
### Phase 8: DNS cutover
|
||||
|
||||
1. Update Cloudflare DNS A record to new server IP.
|
||||
2. Wait for DNS propagation (or use Cloudflare proxy for instant cutover).
|
||||
3. Verify the site loads on the new server.
|
||||
|
||||
### Phase 9: Verify
|
||||
|
||||
```bash
|
||||
# Check site responds
|
||||
curl -sI https://<domain>/ | head -5
|
||||
|
||||
# Check supervisor workers all RUNNING
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl status
|
||||
|
||||
# Check Redis queue backlog (should be 0 or low for all queues)
|
||||
cd /www/wwwroot/coruna-lab
|
||||
sudo -u www /www/server/php/82/bin/php artisan tinker --execute='
|
||||
$r = \Illuminate\Support\Facades\Redis::connection();
|
||||
foreach (["default","extract","ocr","keystore","telegram","transfer"] as $q) {
|
||||
echo "queues:$q = ".$r->llen("queues:$q")."\n";
|
||||
}
|
||||
'
|
||||
|
||||
# Check photos are being stored (should see recent timestamps)
|
||||
mysql -ucoruna -p<pass> coruna -e '
|
||||
SELECT COUNT(*) as cnt, MAX(created_at) as last
|
||||
FROM photos WHERE created_at > DATE_SUB(NOW(), INTERVAL 10 MINUTE);
|
||||
'
|
||||
|
||||
# Check inbox not backing up
|
||||
ls /www/wwwroot/coruna-lab/storage/app/private/c2/inbox/ | wc -l
|
||||
```
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
### 1. Missing `extract` queue worker (MOST COMMON)
|
||||
|
||||
**Symptom**: Users report photos stopped storing. `/t` requests arrive,
|
||||
DB has few/no new photos, `inbox/` directory grows, `queues:extract` in
|
||||
Redis has 100K+ backlog.
|
||||
|
||||
**Fix**: Create `extract.ini` (see Phase 6), reload supervisor.
|
||||
|
||||
### 2. Stale Blade view cache causing 500 errors
|
||||
|
||||
**Symptom**: Pages return 500 after code update. Error log mentions
|
||||
route names not found in compiled view.
|
||||
|
||||
**Fix**: `php artisan view:clear` + restart PHP-FPM.
|
||||
|
||||
### 3. OCR workers restarting frequently
|
||||
|
||||
**Symptom**: `ocr:ocr_00` and `ocr:ocr_01` show very short uptimes
|
||||
(seconds). Error log shows PHP module warnings ("Module already loaded").
|
||||
|
||||
**Cause**: PHP modules loaded twice (fileinfo, redis, zip, gmp). Usually
|
||||
harmless warnings but indicates PHP config issue. Workers still process
|
||||
jobs but may be slower.
|
||||
|
||||
### 4. Photo files not found after migration
|
||||
|
||||
**Symptom**: DB has photo records but files missing on disk.
|
||||
|
||||
**Cause**: Transfer script didn't complete, or path mismatch. Photos
|
||||
are stored at `storage/app/private/c2/photos/{device_uuid}/{sha256}_...`,
|
||||
NOT `storage/app/private/photos/`.
|
||||
|
||||
**Fix**: Re-run transfer for `c2/photos/` directory. Verify with:
|
||||
```bash
|
||||
sudo find /www/wwwroot/coruna-lab/storage/app/private/c2/photos -type f | wc -l
|
||||
```
|
||||
|
||||
### 5. Admin login locked after migration
|
||||
|
||||
**Symptom**: Can't log in to admin panel. `login_attempts` column shows
|
||||
high value, `locked_at` is not NULL.
|
||||
|
||||
**Fix**:
|
||||
```sql
|
||||
UPDATE admins SET login_attempts=0, locked_at=NULL WHERE username='<user>';
|
||||
```
|
||||
|
||||
## Post-migration cleanup
|
||||
|
||||
After confirming the new server is stable:
|
||||
|
||||
1. Stop old server supervisor workers:
|
||||
```bash
|
||||
/www/server/panel/pyenv/bin/supervisorctl stop all
|
||||
```
|
||||
2. Verify no traffic to old server (check nginx access logs).
|
||||
3. Decommission old server after 24-48 hours of stable operation.
|
||||
4. Run disk cleanup on new server (see `coruna-lab-cleanup` skill).
|
||||
Reference in New Issue
Block a user