bttoxin-pipeline/specs/requirements.md

# Technical Specifications

## BtToxin Pipeline - Technical Requirements

### 1. System Architecture

#### 1.1 Overview
BtToxin Pipeline is a web-based genomic analysis platform consisting of:
- **Frontend**: Vue 3 SPA with Element Plus components
- **Backend**: FastAPI REST API with async task processing
- **Task Queue**: Redis-backed queue with semaphore-based concurrency control
- **Analysis Engine**: BtToxin_Digger and BtToxin_Shoter modules

#### 1.2 Component Architecture
```
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   Vue 3 SPA     │────▶│  FastAPI API    │────▶│  Task Queue     │
│  (Frontend)     │     │  (Backend)      │     │  (Redis)        │
└─────────────────┘     └─────────────────┘     └─────────────────┘
                                │                        │
                                │                        ▼
                        ┌───────┴────────┐     ┌─────────────────┐
                        │  Pixi/Conda    │     │  Task Workers   │
                        │  Environments  │     │  (16 concurrent)│
                        └────────────────┘     └─────────────────┘
                                │
                                ▼
                       ┌─────────────────┐
                       │  BtToxin Tools  │
                       │  (Digger/Shoter)│
                       └─────────────────┘
```

### 2. Frontend Specifications

#### 2.1 Technology Stack
| Component | Version/Requirement |
|-----------|---------------------|
| Vue 3 | Composition API + script setup |
| Vite | Latest stable |
| Element Plus | Latest compatible |
| Pinia | Latest stable |
| Vue Router | v4 |
| vue-i18n | v9+ |
| HTTP Client | fetch API (no axios) |

#### 2.2 Page Structure
| Page | Route | Description |
|------|-------|-------------|
| Home | `/` | System introduction, quick start |
| About | `/about` | Features, usage, limitations |
| Submit | `/submit` | File upload, parameters, submit |
| Status | `/status` | Task progress, results |
| Tools | `/tools` | BtToxin_Shoter methodology |

#### 2.3 File Upload Component Requirements
- Drag and drop zone
- File type auto-detection
- Size limit: 100MB
- Pre-upload format validation
- Progress indicator during upload

#### 2.4 Internationalization (i18n)
- Languages: Chinese (zh), English (en)
- Language switcher in header/nav
- Persist selection via localStorage
- Refresh page on language change

### 3. Backend Specifications

#### 3.1 Technology Stack
| Component | Version/Requirement |
|-----------|---------------------|
| FastAPI | Latest stable |
| Uvicorn | Latest stable |
| Python | 3.9+ |
| Redis | Latest stable |
| pixi | Latest (conda alternative) |

#### 3.2 API Specifications

##### 3.2.1 Create Task
```
POST /api/tasks
Content-Type: multipart/form-data

Request Parameters:
| Name                    | Type    | Required | Default | Description |
|-------------------------|---------|----------|---------|-------------|
| file                    | File    | Yes      | -       | Uploaded file |
| file_type               | string  | Yes      | -       | genome/protein |
| min_identity            | float   | No       | 0.8     | Min similarity (0-1) |
| min_coverage            | float   | No       | 0.6     | Min coverage (0-1) |
| allow_unknown_families  | boolean | No       | false   | Allow unknown families |
| require_index_hit       | boolean | No       | true    | Require index hit |
| lang                    | string  | No       | zh      | Report language (zh/en) |

Response:
{
  "task_id": "uuid-string",
  "status": "pending",
  "created_at": "ISO-timestamp",
  "expires_at": "ISO-timestamp"
}
```

##### 3.2.2 Get Task Status
```
GET /api/tasks/{task_id}

Response:
{
  "task_id": "uuid-string",
  "status": "queued|running|completed|failed",
  "progress": 0-100,
  "current_stage": "digger|shoter|plots|bundle",
  "submission_time": "ISO-timestamp",
  "start_time": "ISO-timestamp|null",
  "completion_time": "ISO-timestamp|null",
  "filename": "original-filename",
  "error": "error-message|null",
  "estimated_remaining_seconds": number|null,
  "queue_position": number|null
}
```

##### 3.2.3 Download Results
```
GET /api/tasks/{task_id}/download

Response: .tar.gz file (Content-Disposition: attachment)
```

##### 3.2.4 Delete Task
```
DELETE /api/tasks/{task_id}

Response: 204 No Content
```

#### 3.3 Task Queue Specifications

##### Concurrency Control
- Maximum concurrent tasks: 16
- Implementation: asyncio.Semaphore(16)
- Queue overflow: Tasks wait in Redis queue
- Queue position: Track and display position for queued tasks

##### Task Lifecycle
```
pending → queued → running → completed
                      → failed
```

##### Task Status Values
| Status | Description |
|--------|-------------|
| pending | Created, waiting to enter queue |
| queued | Waiting for available slot (has queue_position) |
| running | Currently processing (has progress, current_stage) |
| completed | Successfully finished (has download URL) |
| failed | Error occurred (has error message) |

##### Pipeline Stages
| Stage | Description |
|-------|-------------|
| digger | BtToxin_Digger gene identification |
| shoter | BtToxin_Shoter toxicity assessment |
| plots | Heatmap generation |
| bundle | Result packaging (.tar.gz) |

#### 3.4 Redis Data Structures

| Key Pattern | Type | Description |
|-------------|------|-------------|
| `task:{task_id}:status` | Hash | Task status and metadata |
| `task:{task_id}:result` | String | Result bundle path |
| `queue:waiting` | List | Waiting task IDs |
| `queue:running` | Set | Currently running task IDs |
| `queue:position:{task_id}` | String | Individual queue position |

### 4. File Format Support

| Extension | File Type | MIME Type | sequence_type |
|-----------|-----------|-----------|---------------|
| .fna | Genome (nucleotide) | application/fasta | nucl |
| .fa | Genome (nucleotide) | application/fasta | nucl |
| .fasta | Auto-detect | application/fasta | auto |
| .faa | Protein | application/fasta | prot |

### 5. Database Specifications

#### 5.1 BPPRC Specificity Database
- File: `toxicity-data.csv`
- Contains: Historical toxin-insect activity records
- Used by: BtToxin_Shoter for activity prediction

#### 5.2 BtToxin Database
- Directory: `external_dbs/bt_toxin`
- Contains: Known Bt toxin sequences
- Used by: BtToxin_Digger for gene identification

### 6. Analysis Pipeline Specifications

#### 6.1 BtToxin_Digger
- Environment: digger (pixi)
- Dependencies: BtToxin_Digger, BLAST, Perl
- Input: Genome (.fna/.fa/.fasta) or protein (.faa) file
- Output: Identified toxin genes with coordinates

#### 6.2 BtToxin_Shoter
- Environment: pipeline (pixi)
- Dependencies: Python 3.9+, pandas, matplotlib, seaborn
- Input: Digger output, scoring parameters
- Output: Toxin-target activity predictions

#### 6.3 Scoring Parameters
| Parameter | Type | Range | Default | Description |
|-----------|------|-------|---------|-------------|
| min_identity | float | 0-1 | 0.8 | Minimum sequence identity |
| min_coverage | float | 0-1 | 0.6 | Minimum coverage |
| allow_unknown_families | boolean | - | false | Allow unknown toxin families |
| require_index_hit | boolean | - | true | Require database index hit |

### 7. Reserved Features

#### 7.1 CRISPR-Cas Analysis Module
- Directory: `crispr_cas/`
- Environment: Additional pixi environment
- Integration: Weighted scoring with Shotter
- Modes: Additive or subtractive weight adjustment

#### 7.2 Direct Protein Analysis
- Digger mode: sequence_type=prot
- Shoter: Process protein sequence hits normally

### 8. Performance Requirements

| Metric | Requirement |
|--------|-------------|
| Task timeout | 6 hours |
| API response time | < 1 second (excluding task execution) |
| Max concurrent tasks | 16 |
| Max file size | 100MB |
| Result retention | 30 days |

### 9. Security Requirements

- **Task isolation**: Each task has independent working directory
- **Input validation**: File format and size validation
- **Result protection**: 30-day automatic cleanup
- **File permissions**: Restricted access to task directories

### 10. Deployment Specifications

#### 10.1 Docker Configuration
- Docker Compose for orchestration
- Services: frontend, backend, redis, traefik
- Volume mounts for data persistence

#### 10.2 Traefik Configuration
- Domain: bttiaw.hzau.edu.cn
- HTTP/HTTPS support
- Automatic certificate management (Let's Encrypt)
- Router rules for each service

### 11. Environment Variables

| Variable | Description | Required |
|----------|-------------|----------|
| REDIS_HOST | Redis server hostname | Yes |
| REDIS_PORT | Redis server port | Yes |
| PIXI_ENV_PATH | Path to pixi environments | Yes |
| API_BASE_URL | Backend API base URL | Yes |
| MAX_CONCURRENT_TASKS | Maximum concurrent tasks | No (default: 16) |
| TASK_TIMEOUT_HOURS | Task timeout in hours | No (default: 6) |
| RESULT_RETENTION_DAYS | Result retention days | No (default: 30) |

### 12. Success Criteria Validation

| Criterion | Validation Method |
|-----------|-------------------|
| Genome/protein upload | Test with .fna and .faa files |
| 16 concurrent tasks | Load test with 20 simultaneous requests |
| Language switching | Verify zh/en toggle works on all pages |
| Heatmap visualization | Compare output with expected results |
| Docker deployment | Access via bttiaw.hzau.edu.cn |