Commit Graph

30 Commits

Author SHA1 Message Date
zenfun
ae2f4d7819 feat(model): add IPBan entity for global IP blocking
Introduces the IPBan model to support global IP/CIDR ban rules enforced by the data plane. Includes fields for CIDR, status, expiration, and hit counts, and registers the model for auto-migration in the server startup.
2026-01-04 00:55:00 +08:00
zenfun
481f616704 refactor(stats): remove daily stats aggregation
Remove the DailyStatsJob, DailyStat model, and associated database
migrations. This eliminates the pre-aggregation layer and updates the
dashboard handler to remove dependencies on the daily_stats table.
2026-01-02 23:08:50 +08:00
zenfun
5b2b176a55 feat(stats): add daily statistics aggregation job and model
- Create `DailyStat` model for immutable daily metrics including
  request counts, tokens, latency, and top models.
- Implement `DailyStatsJob` to aggregate `log_records` from the previous
  day, running daily at 00:05 UTC.
- Register database migrations and schedule the job in the server.
- Add `last7d` and `last30d` period support to stats handler.
2026-01-02 22:20:37 +08:00
zenfun
4cda273f7b feat(alerts): add MasterID to log records and improve traffic spike detection
- Add MasterID field with index to LogRecord model for efficient queries
- Fix threshold config loading to use fixed ID=1 with FirstOrCreate
- Allow traffic spike detection to work without Redis for log-based checks
- Add traffic_spike to API documentation for alert type filter
- Add comprehensive tests for RPM/RPD/TPM spike detection scenarios
2025-12-31 18:01:09 +08:00
zenfun
ba54abd424 feat(alerts): add traffic spike detection with configurable thresholds
Introduce traffic_spike alert type for monitoring system and per-master
traffic levels with configurable thresholds stored in database.

- Add AlertThresholdConfig model for persistent threshold configuration
- Implement GET/PUT /admin/alerts/thresholds endpoints for threshold management
- Add traffic spike detection in alert detector cron job:
  - Global QPS monitoring across all masters
  - Per-master RPM/TPM checks with minimum sample thresholds
  - Per-master RPD/TPD checks for daily limits
- Use warning severity at threshold, critical at 2x threshold
- Include metric metadata (value, threshold, window) in alert details
- Update API documentation with new endpoints and alert type
2025-12-31 15:56:17 +08:00
zenfun
bfba16bbd4 feat(api): add internal alerts reporting endpoint with deduplication
Add ReportAlerts endpoint for Data Plane to report alerts to Control Plane
with fingerprint-based deduplication using a 5-minute cooldown period.

Changes:
- Add POST /internal/alerts/report endpoint with validation
- Add Fingerprint field to Alert model for deduplication
- Extend GetAPIKeyStatsSummary with optional time range filtering
  using since/until query parameters to query from log records
2025-12-31 14:18:09 +08:00
zenfun
2b5e657b3d feat(api): add alert system with CRUD endpoints and statistics
Introduce a comprehensive alert management system for monitoring
system events and notifications.

Changes include:
- Add Alert model with type, severity, status, and metadata fields
- Implement AlertHandler with full CRUD operations (create, list,
  get, acknowledge, resolve, dismiss)
- Add alert statistics endpoint for counts by status and severity
- Register Alert model in database auto-migration
- Add minute-level aggregation to log stats (limited to 6-hour range)
2025-12-31 13:43:48 +08:00
zenfun
1a2cc5b798 feat(api): add API key stats flush and summary endpoints
Introduce internal endpoint for flushing accumulated APIKey statistics
from data plane to control plane database, updating both individual
API keys and their parent provider groups with request counts and
success/failure rates.

Add admin endpoint to retrieve aggregated API key statistics summary
across all provider groups, including total requests, success/failure
counts, and calculated rates.
2025-12-30 00:11:52 +08:00
zenfun
5156ca9cec feat(model): add request statistics fields to ProviderGroup and APIKey 2025-12-29 23:40:01 +08:00
zenfun
f0fe9f0dad feat(api): add OAuth token fields and new provider types support
Add support for OAuth-based authentication with access/refresh tokens
and expiration tracking for API keys. Extend provider groups with
static headers configuration and headers profile options.

Changes include:
- Add AccessToken, RefreshToken, ExpiresAt, AccountID, ProjectID to APIKey model
- Add StaticHeaders and HeadersProfile to ProviderGroup model
- Add TokenRefresh configuration for background token management
- Support new provider types: ClaudeCode, Codex, GeminiCLI, Antigravity
- Update sync service to include new fields in provider snapshots
2025-12-28 02:49:54 +08:00
zenfun
6a16712b9d feat(core): implement sync outbox mechanism and refactor provider validation
- Introduce `SyncOutboxService` and model to retry failed CP-to-Redis sync operations
- Update `SyncService` to handle sync failures by enqueuing tasks to the outbox
- Centralize provider group and API key validation logic into `ProviderGroupManager`
- Refactor API handlers to utilize the new manager and robust sync methods
- Add configuration options for sync outbox (interval, batch size, retries)
2025-12-25 01:24:19 +08:00
zenfun
dff2fc7562 chore(test): clean up unused code and fix test helper usage
- Update TestBatchBindings_Status to use newTestHandlerWithRedis helper
- Remove unused jsonID helper function from sync_bindings_spec_test.go
- Add time import to models.go
2025-12-24 02:27:11 +08:00
zenfun
dea8363e41 refactor(api): split Provider into ProviderGroup and APIKey models
Restructure the provider management system by separating the monolithic
Provider model into two distinct entities:

- ProviderGroup: defines shared upstream configuration (type, base_url,
  google settings, models, status)
- APIKey: represents individual credentials within a group (api_key,
  weight, status, auto_ban, ban settings)

This change also updates:
- Binding model to reference GroupID instead of RouteGroup string
- All CRUD handlers for the new provider-group and api-key endpoints
- Sync service to rebuild provider snapshots from joined tables
- Model registry to aggregate capabilities across group/key pairs
- Access handler to validate namespace existence and subset constraints
- Migration importer to handle the new schema structure
- All related tests to use the new model relationships

BREAKING CHANGE: Provider API endpoints replaced with /provider-groups
and /api-keys endpoints; Binding.RouteGroup replaced with Binding.GroupID
2025-12-24 02:15:52 +08:00
zenfun
2c5ccd56ee feat(api): add realtime stats endpoints for masters
Introduce StatsService integration to admin and master handlers,
exposing realtime metrics (requests, tokens, QPS, rate limit status)
via new endpoints:
- GET /admin/masters/:id/realtime
- GET /v1/realtime

Also embed realtime stats in the existing GET /admin/masters/:id
response and change GlobalQPS default to 0 with validation to
reject negative values.
2025-12-22 12:02:27 +08:00
zenfun
c2ed2f3f9e feat(api): add namespaces, batch ops, and admin logs 2025-12-21 23:16:27 +08:00
zenfun
ac9f0cd0a7 feat(stats): add usage stats and quota reset 2025-12-19 21:50:28 +08:00
zenfun
524f8c5a4e feat(key): extend key metadata and validation 2025-12-19 21:24:24 +08:00
zenfun
96e1fe41a5 feat(models): add kind and models_meta snapshot 2025-12-17 23:15:12 +08:00
zenfun
174735f152 feat(api): add provider creation endpoints and weight support
- Add `POST /admin/providers/preset` for streamlined creation of official providers (OpenAI, Anthropic, Gemini)
- Add `POST /admin/providers/custom` for generic OpenAI-compatible providers
- Add `weight` field to provider model and DTOs to enable weighted routing
- Update sync service to propagate provider weights
- Add unit tests for new creation handlers
2025-12-17 21:40:54 +08:00
zenfun
16fceec8e7 style(core): align struct tags and fields
Standardize alignment of struct tags in models and field assignments in
services for better readability. Additionally, include the Binding model
in the test database auto-migration.
2025-12-17 00:51:02 +08:00
zenfun
2e3b471533 feat(core): implement namespace support and routing bindings
Introduce namespace-aware routing capabilities through a new Binding
model and updates to Master/Key entities.

- Add Binding model for configuring model routes per namespace
- Add DefaultNamespace and Namespaces fields to Master and Key models
- Update auto-migration to include Binding table
- Implement Redis synchronization for binding configurations
- Propagate namespace settings during master creation and key issuance
2025-12-17 00:33:36 +08:00
zenfun
aa69ce3659 feat(api): add admin endpoint to issue keys for masters
Add `POST /admin/masters/{id}/keys` allowing admins to issue child keys
on behalf of a master. Introduce an `issued_by` field in the Key model
to audit whether a key was issued by the master or an admin.

Refactor master service to use typed errors for consistent HTTP status
mapping and ensure validation logic (active status, group check) is
shared.
2025-12-15 15:59:33 +08:00
zenfun
67df74e09a feat(provider): add google project and location fields
Add support for Google Cloud-specific configuration in provider models.
New fields `GoogleProject` and `GoogleLocation` are now included in the
Provider DTO, database model, API handlers, and sync service snapshots.

- Extend Provider struct in model with gorm/json tags
- Update ProviderDTO with omitempty JSON tags
- Include fields in handler create/update logic with trim
- Add fields to providerSnapshot for sync operations
2025-12-13 20:24:44 +08:00
zenfun
2407afe0e6 feat(provider): add status and banning mechanism
Introduce fields for managing provider status (active, auto_disabled,
manual_disabled) and banning logic.

- Add Status, AutoBan, BanReason, and BanUntil to Provider model and DTO
- Update CreateProvider handler to process new fields with defaults
- Extend sync service to propagate status and ban information in snapshots
- Serialize BanUntil as Unix timestamp for sync compatibility
2025-12-12 23:40:18 +08:00
zenfun
25e5e105b3 feat(auth): enhance security with token hashing and sync integration
- Add token hash fields to Master and Key models for indexed lookups
- Implement SyncService integration in admin and master handlers
- Update master key validation with backward-compatible digest lookup
- Hash child keys in database and store token digests for Redis sync
- Add master metadata sync to Redis for balancer validation
- Ensure backward compatibility with legacy rows during migration
2025-12-05 00:17:22 +08:00
zenfun
8645b22b83 feat(auth): implement master key authentication system with child key issuance
Add admin and master authentication layers with JWT support. Replace direct
key creation with hierarchical master/child key system. Update database
schema to support master accounts with configurable limits and epoch-based
key revocation. Add health check endpoint with system status monitoring.

BREAKING CHANGE: Removed direct POST /keys endpoint in favor of master-based
key issuance through /v1/tokens. Database migration requires dropping old User
table and creating Master table with new relationships.
2025-12-05 00:16:47 +08:00
b8bd613f77 feat(provider): add Models field to ProviderDTO and update provider handling 2025-12-02 16:32:03 +08:00
zenfun
aa57af874c feat(core): implement group-based routing and optimize sync
Replace direct provider linkage with group-based routing for Keys and
Providers. This allows for more flexible load balancing and tiering
strategies.

Changes include:
- Remove `ProviderID` from Key model and DTO
- Add `Group` field to Key and Provider models
- Refactor Redis sync to use Hashes for O(1) partial updates
- Update API handlers to perform incremental syncs

BREAKING CHANGE: Key API payload no longer accepts `provider_id`. Redis
configuration storage format has changed from JSON strings to Hashes.
2025-12-02 14:49:03 +08:00
zenfun
64d71953a6 feat(server): implement management endpoints and redis sync
Enable full resource management via API and support data plane
synchronization.

- Add CRUD handlers for Providers, Models, and Keys using DTOs
- Implement LogWriter service for asynchronous, batched audit logging
- Update SyncService to snapshot full configuration state to Redis
- Register new API routes and initialize background services
- Add configuration options for logging performance tuning
2025-12-02 14:26:16 +08:00
zenfun
58dfe5e9ac feat(server): initialize project skeleton with db and api setup
Establish the foundational structure for the ez-api server.

Key changes include:
- Set up main entry point with graceful shutdown and Gin router
- Configure database connections for PostgreSQL (GORM) and Redis
- Define core data models (User, Provider, Key, Model)
- Implement configuration loading and basic key creation handler
- Add Dockerfile for multi-stage builds and .gitignore
2025-12-02 13:35:17 +08:00