Commit Graph

81 Commits

Author SHA1 Message Date
7929b2a872 refactor(api): rename auth/whoami to users/info and simplify response
Rename endpoint from /auth/whoami to /users/info to align with
ez-contract schema. Simplify WhoamiResponse by removing:
- Realtime stats (requests, tokens, qps, rate_limited)
- Key-specific fields (allow_ips, deny_ips, expires_at, model_limits,
  quota fields, usage stats)

The endpoint now returns only essential identity information as
defined in ez-contract/schemas/auth/auth.yaml.

BREAKING CHANGE: /auth/whoami endpoint moved to /users/info with
reduced response fields
2026-01-13 15:57:52 +08:00
c990fa6610 docs(swagger): fix response type annotations for swagger generation
Update DeleteMasterKey endpoint to use MapData instead of dto.DeleteResponse
and add blank references for model registry DTOs to ensure proper swagger
documentation generation.
2026-01-10 15:36:46 +08:00
zenfun
2098bc4abe refactor(api): standardize DTOs and update swagger
Decouple API contract from internal models by introducing dedicated DTOs for requests and responses.
- Add Response DTOs for all resources (API Keys, Bindings, Models, Namespaces, etc.)
- Update Swagger annotations to use DTOs with field examples instead of internal models
- Refactor handlers to bind and return DTO structures
- Consolidate request/response definitions in the dto package
2026-01-10 02:05:55 +08:00
zenfun
e7db9f319f fix: delete keys and seed only active ones
Ensure admin key deletion removes the DB record and returns a
"deleted" status. Update seeder idempotency to count only active keys
when deciding whether to skip or create new keys.
2026-01-10 01:18:04 +08:00
zenfun
5349c9c833 feat(api): add admin master key listing/revoke
Add admin endpoints to list and revoke child keys under a master.
Standardize OpenAPI responses to use ResponseEnvelope with MapData
for error payloads, and regenerate swagger specs accordingly.
2026-01-10 01:10:36 +08:00
zenfun
ac6a1858cf feat(api): align response envelope schema
Switch response envelope business code to numeric and make message
consistently present. Add trace_id and optional details, and remove the
duplicate DTO envelope definition. Improve middleware path exclusion
handling and add a time-based trace ID fallback if crypto RNG fails.

BREAKING CHANGE: response envelope `code` is now `int` (was `string`) and
`message` semantics/defaults changed; clients must update parsing.
2026-01-10 01:09:05 +08:00
zenfun
dd32ee817d test(api): align tests with new envelope codes
Update API handler tests to expect numeric `code`, `success` messages, and
new envelope fields (`trace_id`, `details`), matching recent response
envelope changes.
2026-01-10 00:39:35 +08:00
zenfun
33838b1e2c feat(api): wrap JSON responses in envelope
Add response envelope middleware to standardize JSON responses as
`{code,data,message}` with consistent business codes across endpoints.
Update Swagger annotations and tests to reflect the new response shape.

BREAKING CHANGE: API responses are now wrapped in a response envelope; clients must read payloads from `data` and handle `code`/`message` fields.
2026-01-10 00:15:08 +08:00
zenfun
f400ffde95 refactor(api): update traffic chart response structure
Change the traffic chart API response from bucket-based to series-based
to better support frontend visualization libraries. The new format
provides a shared X-axis and aligned data arrays for each model series.

- Replace `buckets` with `x` and `series` in response
- Implement data alignment and zero-filling for time slots
- Update Swagger documentation including pending definitions

BREAKING CHANGE: The `GET /admin/logs/stats/traffic-chart` response
schema has changed. `buckets` and `models` fields are removed.
2026-01-08 18:40:44 +08:00
341b54b185 docs(api): add ModelMetricsMap type and improve breakdown field documentation
Extract ModelMetricsMap as a named type with documentation comments
explaining the map structure and providing examples. Update the
TrafficBucket.Breakdown field to use the new type and enhance the
GetTrafficChart endpoint description with detailed breakdown field
documentation including example usage.
2026-01-08 16:31:34 +08:00
b71c7f2541 style(api): align struct field tags and fix string concatenation
- Fix JSON field alignment in error response and TrafficBucket struct
- Add documentation comment and swagger annotations for Breakdown field
- Remove unnecessary string concatenation spacing in SQL select
2026-01-08 16:27:37 +08:00
a2bc994100 docs(api): expand whoami endpoint documentation with detailed response fields
Add comprehensive API documentation for the whoami endpoint including:
- Detailed response structure for each token type (Admin, Master, Child Key)
- Complete field listings for master keys and child keys
- All possible error responses with status codes
- Usage guidance for frontend initialization
2026-01-06 09:21:02 +08:00
1ee6bea413 feat(api): enhance whoami endpoint with realtime stats and extended key info
Add realtime statistics (requests, tokens, QPS, rate limiting) to whoami
response for both master and key authentication types. Extend key response
with additional fields including master name, model limits, quota tracking,
and usage statistics.

- Inject StatsService into AuthHandler for realtime stats retrieval
- Add WhoamiRealtimeView struct for realtime statistics
- Include admin permissions field in admin response
- Add comprehensive key metadata (quotas, model limits, usage stats)
- Add test for expired key returning 401 Unauthorized
2026-01-06 09:15:49 +08:00
zenfun
a7571dd4ad feat(server): integrate ip ban cron and refine updates
- Initialize and schedule IP ban maintenance tasks in server entry point
- Perform initial IP ban sync to Redis on startup
- Implement optional JSON unmarshalling to handle null `expires_at` in API
- Add CIDR overlap validation when updating rule status to active
2026-01-04 01:44:45 +08:00
zenfun
7393bf6dc3 feat(api): add IP ban handler for CRUD operations
Implement the HTTP handler for managing global IP/CIDR bans. This
includes endpoints for creating, listing, retrieving, updating, and
deleting IP ban rules, complete with Swagger documentation and error
handling.
2026-01-04 01:00:44 +08:00
zenfun
03ec45c84c fix(auth): improve token metadata parsing and validation
Safeguard integer parsing in the `Whoami` handler by trimming whitespace and handling errors explicitly for `master_id`, `issued_at_epoch`, and `expires_at`. This prevents potential validation bypasses or incorrect behavior due to malformed metadata.

Add unit tests to verify invalid epoch handling and response correctness.
2026-01-03 16:19:42 +08:00
zenfun
4cd9b66a84 feat(auth): enhance token validation and internal access control
Refactor the `Whoami` handler to validate token metadata (status, expiration,
revocation) against Redis before database lookup, ensuring consistency with
balancer logic. Add `allow_ips`, `deny_ips`, and `expires_at` fields to
authentication responses.

Update internal middleware to support explicit anonymous access configuration
and harden security for unconfigured tokens.

Remove legacy fallback logic for master keys without digests.

BREAKING CHANGE: Internal endpoints now reject requests by default if no stats token is configured. To allow unauthenticated access, set `internal.allow_anonymous` to true.
BREAKING CHANGE: Support for legacy master keys without stored digests has been removed.
2026-01-03 16:04:04 +08:00
zenfun
5c01497ce0 fix(api): handle zero-baseline edge cases in trend calculation
Introduce `CalculateTrendFloatWithBaseline` to correctly handle scenarios where previous period metrics (Error Rate, Latency) are zero or missing. This prevents arithmetic errors and distinguishes between "new" data and actual increases ("up") when starting from zero.

Also updates the admin panel dashboard documentation to reflect current project status.
2026-01-02 23:17:55 +08:00
zenfun
481f616704 refactor(stats): remove daily stats aggregation
Remove the DailyStatsJob, DailyStat model, and associated database
migrations. This eliminates the pre-aggregation layer and updates the
dashboard handler to remove dependencies on the daily_stats table.
2026-01-02 23:08:50 +08:00
zenfun
08a8a1e42f feat(api): add trend analysis and extended periods to dashboard summary
- Add `include_trends` query parameter to enable trend calculation
- Implement trend comparison logic (delta % and direction) against previous periods
- Add support for `last7d`, `last30d`, and `all` time period options
- Update `DashboardSummaryResponse` to include optional `trends` field
- Add helper functions for custom time window aggregation
- Add unit tests for trend calculation and period window logic
- Update feature documentation with new parameters and response schemas
2026-01-02 22:30:38 +08:00
zenfun
5b2b176a55 feat(stats): add daily statistics aggregation job and model
- Create `DailyStat` model for immutable daily metrics including
  request counts, tokens, latency, and top models.
- Implement `DailyStatsJob` to aggregate `log_records` from the previous
  day, running daily at 00:05 UTC.
- Register database migrations and schedule the job in the server.
- Add `last7d` and `last30d` period support to stats handler.
2026-01-02 22:20:37 +08:00
zenfun
bfb19ca23e test(api): add validation tests for traffic chart endpoint
Add TestTrafficChart_MinuteGranularityValidation to verify input
parameters including granularity, time range limits, and top_n
constraints. Include skipped placeholders for PostgreSQL-specific
aggregation tests.
2026-01-02 21:35:42 +08:00
zenfun
9d082ff375 feat(api): add admin traffic chart statistics endpoint
Add new endpoint GET /admin/logs/stats/traffic-chart to provide
aggregated traffic metrics grouped by time and model. Features include:
- Time granularity selection (hour/minute)
- Top-N model breakdown with "other" aggregation
- Metrics for request counts and token usage

Includes generated Swagger documentation.
2026-01-02 21:24:56 +08:00
zenfun
4cda273f7b feat(alerts): add MasterID to log records and improve traffic spike detection
- Add MasterID field with index to LogRecord model for efficient queries
- Fix threshold config loading to use fixed ID=1 with FirstOrCreate
- Allow traffic spike detection to work without Redis for log-based checks
- Add traffic_spike to API documentation for alert type filter
- Add comprehensive tests for RPM/RPD/TPM spike detection scenarios
2025-12-31 18:01:09 +08:00
zenfun
f714a314a9 test(alerts): add comprehensive tests for alert handler and detector
Add unit tests for alert-related functionality:

- alert_handler_test.go: tests for threshold CRUD operations,
  alert creation with traffic_spike type, filtering, and stats
- alert_detector_test.go: tests for threshold config loading,
  traffic spike severity calculation, deduplication logic,
  error rate severity, and nil-safety checks

Also fix format string issues:
- Use %d instead of %.2f for integer QPS in alert messages
- Wrap error description with format directive to avoid linter warning
2025-12-31 16:09:02 +08:00
zenfun
ba54abd424 feat(alerts): add traffic spike detection with configurable thresholds
Introduce traffic_spike alert type for monitoring system and per-master
traffic levels with configurable thresholds stored in database.

- Add AlertThresholdConfig model for persistent threshold configuration
- Implement GET/PUT /admin/alerts/thresholds endpoints for threshold management
- Add traffic spike detection in alert detector cron job:
  - Global QPS monitoring across all masters
  - Per-master RPM/TPM checks with minimum sample thresholds
  - Per-master RPD/TPD checks for daily limits
- Use warning severity at threshold, critical at 2x threshold
- Include metric metadata (value, threshold, window) in alert details
- Update API documentation with new endpoints and alert type
2025-12-31 15:56:17 +08:00
zenfun
bfba16bbd4 feat(api): add internal alerts reporting endpoint with deduplication
Add ReportAlerts endpoint for Data Plane to report alerts to Control Plane
with fingerprint-based deduplication using a 5-minute cooldown period.

Changes:
- Add POST /internal/alerts/report endpoint with validation
- Add Fingerprint field to Alert model for deduplication
- Extend GetAPIKeyStatsSummary with optional time range filtering
  using since/until query parameters to query from log records
2025-12-31 14:18:09 +08:00
zenfun
2b5e657b3d feat(api): add alert system with CRUD endpoints and statistics
Introduce a comprehensive alert management system for monitoring
system events and notifications.

Changes include:
- Add Alert model with type, severity, status, and metadata fields
- Implement AlertHandler with full CRUD operations (create, list,
  get, acknowledge, resolve, dismiss)
- Add alert statistics endpoint for counts by status and severity
- Register Alert model in database auto-migration
- Add minute-level aggregation to log stats (limited to 6-hour range)
2025-12-31 13:43:48 +08:00
zenfun
53c18c3867 feat(api): add dashboard summary and system realtime endpoints
Add new admin API endpoints for dashboard metrics and system-wide
realtime statistics:

- Add /admin/dashboard/summary endpoint with aggregated metrics
  including requests, tokens, latency, masters, keys, and provider
  keys statistics with time period filtering
- Add /admin/realtime endpoint for system-level realtime stats
  aggregated across all masters
- Add status filter parameter to ListAPIKeys endpoint
- Add hour grouping option to log stats aggregation
- Update OpenAPI documentation with new endpoints and schemas
2025-12-31 13:17:23 +08:00
zenfun
1a2cc5b798 feat(api): add API key stats flush and summary endpoints
Introduce internal endpoint for flushing accumulated APIKey statistics
from data plane to control plane database, updating both individual
API keys and their parent provider groups with request counts and
success/failure rates.

Add admin endpoint to retrieve aggregated API key statistics summary
across all provider groups, including total requests, success/failure
counts, and calculated rates.
2025-12-30 00:11:52 +08:00
zenfun
f0fe9f0dad feat(api): add OAuth token fields and new provider types support
Add support for OAuth-based authentication with access/refresh tokens
and expiration tracking for API keys. Extend provider groups with
static headers configuration and headers profile options.

Changes include:
- Add AccessToken, RefreshToken, ExpiresAt, AccountID, ProjectID to APIKey model
- Add StaticHeaders and HeadersProfile to ProviderGroup model
- Add TokenRefresh configuration for background token management
- Support new provider types: ClaudeCode, Codex, GeminiCLI, Antigravity
- Update sync service to include new fields in provider snapshots
2025-12-28 02:49:54 +08:00
zenfun
f51cd63c82 test(api): remove health handler test file 2025-12-27 13:51:25 +08:00
zenfun
637bfa8210 feat(api): add public status endpoints with version injection
Replace health_handler with status_handler providing public /status and
/about endpoints. Add build-time version injection via ldflags in
Makefile, and support --version/-v CLI flag.

- Add /status endpoint returning runtime status, uptime, and version
- Add /about endpoint with system metadata (name, description, repo)
- Configure VERSION variable with git describe fallback
- Update swagger docs and api.md for new public endpoints
- Remove deprecated /api/status/test endpoint
2025-12-27 13:24:13 +08:00
zenfun
c83fe03892 docs(swagger): update API documentation for log endpoints
Add new response types and parameters for logs API:
- Add GroupedStatsItem and GroupedStatsResponse definitions
- Add ListLogsResponse and LogView definitions for detailed log records
- Add group_by enum parameter (model/day/month) to stats endpoint
- Update endpoint descriptions to clarify response types and request_body inclusion
- Update response schema references to use correct types
2025-12-26 11:12:47 +08:00
zenfun
3f1d006006 feat(api): add grouped statistics and request_body to log endpoints
Add group_by parameter to /admin/logs/stats endpoint supporting:
- group_by=model: aggregate stats per model with avg latency
- group_by=day: daily aggregation with token counts
- group_by=month: monthly aggregation with token counts

Also include request_body field in admin ListLogs response for
full visibility into logged requests.
2025-12-26 11:06:39 +08:00
8e6d86edd7 feat(api): add /auth/whoami endpoint for identity detection 2025-12-25 14:41:38 +08:00
zenfun
6a16712b9d feat(core): implement sync outbox mechanism and refactor provider validation
- Introduce `SyncOutboxService` and model to retry failed CP-to-Redis sync operations
- Update `SyncService` to handle sync failures by enqueuing tasks to the outbox
- Centralize provider group and API key validation logic into `ProviderGroupManager`
- Refactor API handlers to utilize the new manager and robust sync methods
- Add configuration options for sync outbox (interval, batch size, retries)
2025-12-25 01:24:19 +08:00
zenfun
dff2fc7562 chore(test): clean up unused code and fix test helper usage
- Update TestBatchBindings_Status to use newTestHandlerWithRedis helper
- Remove unused jsonID helper function from sync_bindings_spec_test.go
- Add time import to models.go
2025-12-24 02:27:11 +08:00
zenfun
dea8363e41 refactor(api): split Provider into ProviderGroup and APIKey models
Restructure the provider management system by separating the monolithic
Provider model into two distinct entities:

- ProviderGroup: defines shared upstream configuration (type, base_url,
  google settings, models, status)
- APIKey: represents individual credentials within a group (api_key,
  weight, status, auto_ban, ban settings)

This change also updates:
- Binding model to reference GroupID instead of RouteGroup string
- All CRUD handlers for the new provider-group and api-key endpoints
- Sync service to rebuild provider snapshots from joined tables
- Model registry to aggregate capabilities across group/key pairs
- Access handler to validate namespace existence and subset constraints
- Migration importer to handle the new schema structure
- All related tests to use the new model relationships

BREAKING CHANGE: Provider API endpoints replaced with /provider-groups
and /api-keys endpoints; Binding.RouteGroup replaced with Binding.GroupID
2025-12-24 02:15:52 +08:00
zenfun
de1d23cce5 docs: regenerate swagger docs with new API endpoints
Add comprehensive API documentation for new endpoints including:
- Bindings CRUD and batch operations
- Namespaces management
- Master tenant CRUD, access settings, and realtime stats
- Provider CRUD with preset/custom/google variants
- Model registry status, refresh, check, and rollback
- Log management with stats and webhook config
- Key access settings endpoints
- Internal stats flush endpoint
- Health check endpoint

Also adds swagger annotations for FlushStats internal handler.
2025-12-22 13:03:52 +08:00
zenfun
2c5ccd56ee feat(api): add realtime stats endpoints for masters
Introduce StatsService integration to admin and master handlers,
exposing realtime metrics (requests, tokens, QPS, rate limit status)
via new endpoints:
- GET /admin/masters/:id/realtime
- GET /v1/realtime

Also embed realtime stats in the existing GET /admin/masters/:id
response and change GlobalQPS default to 0 with validation to
reject negative values.
2025-12-22 12:02:27 +08:00
zenfun
fa7f92c6e3 feat(api): allow batch status updates 2025-12-21 23:32:07 +08:00
zenfun
c2ed2f3f9e feat(api): add namespaces, batch ops, and admin logs 2025-12-21 23:16:27 +08:00
zenfun
73147fc55a feat(api): add model delete, pagination, and cors config 2025-12-21 23:03:12 +08:00
zenfun
816ea93339 feat(arch): add log partitioning and provider delete sync 2025-12-21 20:45:16 +08:00
zenfun
f819f89ba2 test(ops): cover status test and model fetch 2025-12-21 20:23:04 +08:00
zenfun
f7baa0f08f feat(ops): add status test and model fetch 2025-12-21 20:22:56 +08:00
zenfun
6e6c669ea0 test(log): expand handler, config, and metrics coverage 2025-12-21 16:18:38 +08:00
zenfun
c2c65e774b feat(log): wire log db, metrics, and body toggle 2025-12-21 16:18:22 +08:00
zenfun
4c1e03f83d feat(api): add log webhook notification service
Implement webhook notifications for log error threshold alerts with
configurable thresholds, time windows, and cooldown periods.

- Add LogWebhookService with Redis-backed configuration storage
- Add admin endpoints for webhook config management (GET/PUT)
- Trigger webhook notifications when error count exceeds threshold
- Support status code threshold and error message detection
- Include sample log record data in webhook payload
2025-12-21 14:13:35 +08:00