Commit Graph

141 Commits

Author SHA1 Message Date
zenfun
3d39c591fd fix(sync): use nanosecond precision for bindings version to ensure uniqueness
Use UnixNano for version field while keeping Unix seconds for updated_at
timestamp. This ensures version changes are detected even when multiple
syncs occur within the same second.
2025-12-26 11:20:53 +08:00
zenfun
c83fe03892 docs(swagger): update API documentation for log endpoints
Add new response types and parameters for logs API:
- Add GroupedStatsItem and GroupedStatsResponse definitions
- Add ListLogsResponse and LogView definitions for detailed log records
- Add group_by enum parameter (model/day/month) to stats endpoint
- Update endpoint descriptions to clarify response types and request_body inclusion
- Update response schema references to use correct types
2025-12-26 11:12:47 +08:00
zenfun
3f1d006006 feat(api): add grouped statistics and request_body to log endpoints
Add group_by parameter to /admin/logs/stats endpoint supporting:
- group_by=model: aggregate stats per model with avg latency
- group_by=day: daily aggregation with token counts
- group_by=month: monthly aggregation with token counts

Also include request_body field in admin ListLogs response for
full visibility into logged requests.
2025-12-26 11:06:39 +08:00
30f15a84b5 feat(api): add /auth/whoami endpoint and build automation 2025-12-25 14:54:52 +08:00
8e6d86edd7 feat(api): add /auth/whoami endpoint for identity detection 2025-12-25 14:41:38 +08:00
b566eb8058 fix(swagger): restore apikey security definition with Bearer usage description 2025-12-25 11:32:55 +08:00
c8fced4cf1 fix(swagger): fix swagger authorization header 2025-12-25 11:26:56 +08:00
ecdff89364 feat(k8s): add missing env configs for outbox and timeout 2025-12-25 11:14:02 +08:00
6473ac1c81 feat: update kubernetes deployment 2025-12-25 11:09:32 +08:00
9a07e0f468 feat: update kubernetes test environment configurations 2025-12-25 11:02:37 +08:00
41998a3584 feat(swagger): support dynamic host via EZ_SWAGGER_HOST env var
- Add ServerConfig.SwaggerHost field
- Set docs.SwaggerInfo.Host dynamically at startup
- Update docker-compose, k8s manifests, and .env.example
2025-12-25 10:59:57 +08:00
zenfun
6a16712b9d feat(core): implement sync outbox mechanism and refactor provider validation
- Introduce `SyncOutboxService` and model to retry failed CP-to-Redis sync operations
- Update `SyncService` to handle sync failures by enqueuing tasks to the outbox
- Centralize provider group and API key validation logic into `ProviderGroupManager`
- Refactor API handlers to utilize the new manager and robust sync methods
- Add configuration options for sync outbox (interval, batch size, retries)
2025-12-25 01:24:19 +08:00
zenfun
44a82fa252 delete: remove png 2025-12-24 17:01:56 +08:00
zenfun
72d7920534 doc: add admin panel map 2025-12-24 15:12:06 +08:00
zenfun
38d2329991 doc: 增加 .env 的注释 2025-12-24 14:51:57 +08:00
7f160a8d2a feat: 添加kubernetes部署 2025-12-24 11:49:34 +08:00
b97705532e chore:update foundation version 2025-12-24 11:31:49 +08:00
zenfun
dff2fc7562 chore(test): clean up unused code and fix test helper usage
- Update TestBatchBindings_Status to use newTestHandlerWithRedis helper
- Remove unused jsonID helper function from sync_bindings_spec_test.go
- Add time import to models.go
2025-12-24 02:27:11 +08:00
zenfun
d731e42ae5 docs: update swagger specs for provider-groups and api-keys refactor
Regenerate OpenAPI documentation to reflect the architectural split
of Provider into ProviderGroup and APIKey models:

- Replace /admin/providers endpoints with /admin/provider-groups
- Add new /admin/api-keys CRUD and batch endpoints
- Update BindingDTO to use group_id instead of route_group
- Remove provider-specific DTOs (ProviderCustomCreateDTO, ProviderPresetCreateDTO, ProviderGoogleCreateDTO)
- Add APIKeyDTO and ProviderGroupDTO definitions
- Update model definitions for APIKey, Binding, and ProviderGroup
2025-12-24 02:20:32 +08:00
zenfun
dea8363e41 refactor(api): split Provider into ProviderGroup and APIKey models
Restructure the provider management system by separating the monolithic
Provider model into two distinct entities:

- ProviderGroup: defines shared upstream configuration (type, base_url,
  google settings, models, status)
- APIKey: represents individual credentials within a group (api_key,
  weight, status, auto_ban, ban settings)

This change also updates:
- Binding model to reference GroupID instead of RouteGroup string
- All CRUD handlers for the new provider-group and api-key endpoints
- Sync service to rebuild provider snapshots from joined tables
- Model registry to aggregate capabilities across group/key pairs
- Access handler to validate namespace existence and subset constraints
- Migration importer to handle the new schema structure
- All related tests to use the new model relationships

BREAKING CHANGE: Provider API endpoints replaced with /provider-groups
and /api-keys endpoints; Binding.RouteGroup replaced with Binding.GroupID
2025-12-24 02:15:52 +08:00
zenfun
cd5616dc26 feat(migrate): add import CLI command and importer for migration data
Introduce a new `import` subcommand to the server binary that reads
exported JSON files and imports masters, providers, keys, bindings,
and namespaces into the database.

Key features:
- Support for dry-run mode to validate without writing
- Conflict policies: skip existing or overwrite
- Optional binding import via --include-bindings flag
- Auto-generation of master keys with secure hashing
- Namespace auto-creation for referenced namespaces
- Detailed import summary with warnings and created credentials
2025-12-23 20:13:45 +08:00
zenfun
ee6c28afc9 docs(api): update and expand business documentation
Rewrite docs/api.md to provide a more structured overview of business
logic, core models, and authentication mechanisms. Include detailed
cURL examples for typical operations and add a new management
relationship diagram asset.

- Significant rewrite of docs/api.md with better formatting and content
- Add mermaid diagram for resource relationships
- Update README.md to reference the expanded documentation
- Add docs/管理关系图.png asset
2025-12-22 14:45:18 +08:00
zenfun
d8682acfe5 add doc 2025-12-22 14:34:22 +08:00
zenfun
9518ae3fec feat(docker): add dedicated PostgreSQL instance for logs
Add separate log-postgres service to docker-compose stack for storing
API logs independently from the main database.

- Add log-postgres service with PostgreSQL 15 Alpine image
- Configure environment variables for log database credentials
- Update ez-api service to depend on log-postgres health check
- Set default EZ_LOG_PG_DSN to connect to the new log database
- Update .env.example and README with new configuration options
2025-12-22 14:32:27 +08:00
zenfun
5770e496ff chore(deps): bump foundation to v0.3.0 2025-12-22 14:03:06 +08:00
zenfun
c325c2e820 update .env 2025-12-22 13:29:21 +08:00
zenfun
dc43e3ad4c update docker yml 2025-12-22 13:28:40 +08:00
zenfun
ca0be28aba doc : update readme 2025-12-22 13:27:02 +08:00
zenfun
de1d23cce5 docs: regenerate swagger docs with new API endpoints
Add comprehensive API documentation for new endpoints including:
- Bindings CRUD and batch operations
- Namespaces management
- Master tenant CRUD, access settings, and realtime stats
- Provider CRUD with preset/custom/google variants
- Model registry status, refresh, check, and rollback
- Log management with stats and webhook config
- Key access settings endpoints
- Internal stats flush endpoint
- Health check endpoint

Also adds swagger annotations for FlushStats internal handler.
2025-12-22 13:03:52 +08:00
zenfun
2c5ccd56ee feat(api): add realtime stats endpoints for masters
Introduce StatsService integration to admin and master handlers,
exposing realtime metrics (requests, tokens, QPS, rate limit status)
via new endpoints:
- GET /admin/masters/:id/realtime
- GET /v1/realtime

Also embed realtime stats in the existing GET /admin/masters/:id
response and change GlobalQPS default to 0 with validation to
reject negative values.
2025-12-22 12:02:27 +08:00
zenfun
fa7f92c6e3 feat(api): allow batch status updates 2025-12-21 23:32:07 +08:00
zenfun
c2ed2f3f9e feat(api): add namespaces, batch ops, and admin logs 2025-12-21 23:16:27 +08:00
zenfun
73147fc55a feat(api): add model delete, pagination, and cors config 2025-12-21 23:03:12 +08:00
zenfun
816ea93339 feat(arch): add log partitioning and provider delete sync 2025-12-21 20:45:16 +08:00
zenfun
f819f89ba2 test(ops): cover status test and model fetch 2025-12-21 20:23:04 +08:00
zenfun
f7baa0f08f feat(ops): add status test and model fetch 2025-12-21 20:22:56 +08:00
zenfun
6e6c669ea0 test(log): expand handler, config, and metrics coverage 2025-12-21 16:18:38 +08:00
zenfun
c2c65e774b feat(log): wire log db, metrics, and body toggle 2025-12-21 16:18:22 +08:00
zenfun
4c1e03f83d feat(api): add log webhook notification service
Implement webhook notifications for log error threshold alerts with
configurable thresholds, time windows, and cooldown periods.

- Add LogWebhookService with Redis-backed configuration storage
- Add admin endpoints for webhook config management (GET/PUT)
- Trigger webhook notifications when error count exceeds threshold
- Support status code threshold and error message detection
- Include sample log record data in webhook payload
2025-12-21 14:13:35 +08:00
zenfun
00192f937e feat(api): add log_request_body_enabled feature flag support
Add runtime feature flag to control whether request bodies are stored
in logs. The Handler now accepts a Redis client to check the
log_request_body_enabled feature flag before persisting log records.

- Add logRequestBodyFeatureKey constant for feature flag
- Inject Redis client into Handler for feature flag lookups
- Strip request body from log records when feature is disabled
- Update tests to pass Redis client to NewHandler
2025-12-21 13:26:16 +08:00
zenfun
1089f9490e test(api): add feature handler tests for regular and mixed keys
Add tests for UpdateFeatures endpoint covering:
- Regular feature flag keys with various value types
- Mixed keys combining feature flags with log retention settings
- Redis storage verification for both scenarios
2025-12-21 13:07:00 +08:00
zenfun
2f58dd76a7 feat(api): add log retention and max records feature overrides
Add special handling for log_retention_days and log_max_records features
that are stored as separate Redis keys instead of in the features hash.

- Store log overrides in dedicated keys (meta:log:retention_days,
  meta:log:max_records) for runtime access by log cleanup cron
- Include log override values in ListFeatures response
- Support clearing overrides by setting value to 0
- Add comprehensive tests for log override CRUD operations
2025-12-21 12:26:48 +08:00
zenfun
25795a79d6 feat(cron): add automatic log cleanup with retention policy
Implement LogCleaner cron job to automatically clean up old log records
based on configurable retention period and maximum record count.

- Add LogCleaner with retention_days and max_records configuration
- Add EZ_LOG_RETENTION_DAYS and EZ_LOG_MAX_RECORDS environment variables
- Default to 30 days retention and 1,000,000 max records
- Include unit tests for log cleaner functionality
2025-12-21 12:01:52 +08:00
zenfun
369c204c16 feat: add admin log deletion 2025-12-21 00:53:52 +08:00
zenfun
88289015fc feat: add internal stats flush API 2025-12-19 23:26:33 +08:00
zenfun
ac9f0cd0a7 feat(stats): add usage stats and quota reset 2025-12-19 21:50:28 +08:00
zenfun
524f8c5a4e feat(key): extend key metadata and validation 2025-12-19 21:24:24 +08:00
zenfun
5e98368428 model-registry: default supports_stream true 2025-12-18 16:48:43 +08:00
zenfun
7dd3fac24e model-registry: add upstream check endpoint 2025-12-18 16:43:12 +08:00
zenfun
a61eff27e7 feat(admin/master): provider+master CRUD, token mgmt, logs APIs 2025-12-18 16:21:46 +08:00