Commit Graph

71 Commits

Author SHA1 Message Date
zenfun
cd5616dc26 feat(migrate): add import CLI command and importer for migration data
Introduce a new `import` subcommand to the server binary that reads
exported JSON files and imports masters, providers, keys, bindings,
and namespaces into the database.

Key features:
- Support for dry-run mode to validate without writing
- Conflict policies: skip existing or overwrite
- Optional binding import via --include-bindings flag
- Auto-generation of master keys with secure hashing
- Namespace auto-creation for referenced namespaces
- Detailed import summary with warnings and created credentials
2025-12-23 20:13:45 +08:00
zenfun
ee6c28afc9 docs(api): update and expand business documentation
Rewrite docs/api.md to provide a more structured overview of business
logic, core models, and authentication mechanisms. Include detailed
cURL examples for typical operations and add a new management
relationship diagram asset.

- Significant rewrite of docs/api.md with better formatting and content
- Add mermaid diagram for resource relationships
- Update README.md to reference the expanded documentation
- Add docs/管理关系图.png asset
2025-12-22 14:45:18 +08:00
zenfun
d8682acfe5 add doc 2025-12-22 14:34:22 +08:00
zenfun
9518ae3fec feat(docker): add dedicated PostgreSQL instance for logs
Add separate log-postgres service to docker-compose stack for storing
API logs independently from the main database.

- Add log-postgres service with PostgreSQL 15 Alpine image
- Configure environment variables for log database credentials
- Update ez-api service to depend on log-postgres health check
- Set default EZ_LOG_PG_DSN to connect to the new log database
- Update .env.example and README with new configuration options
2025-12-22 14:32:27 +08:00
zenfun
5770e496ff chore(deps): bump foundation to v0.3.0 2025-12-22 14:03:06 +08:00
zenfun
c325c2e820 update .env 2025-12-22 13:29:21 +08:00
zenfun
dc43e3ad4c update docker yml 2025-12-22 13:28:40 +08:00
zenfun
ca0be28aba doc : update readme 2025-12-22 13:27:02 +08:00
zenfun
de1d23cce5 docs: regenerate swagger docs with new API endpoints
Add comprehensive API documentation for new endpoints including:
- Bindings CRUD and batch operations
- Namespaces management
- Master tenant CRUD, access settings, and realtime stats
- Provider CRUD with preset/custom/google variants
- Model registry status, refresh, check, and rollback
- Log management with stats and webhook config
- Key access settings endpoints
- Internal stats flush endpoint
- Health check endpoint

Also adds swagger annotations for FlushStats internal handler.
2025-12-22 13:03:52 +08:00
zenfun
2c5ccd56ee feat(api): add realtime stats endpoints for masters
Introduce StatsService integration to admin and master handlers,
exposing realtime metrics (requests, tokens, QPS, rate limit status)
via new endpoints:
- GET /admin/masters/:id/realtime
- GET /v1/realtime

Also embed realtime stats in the existing GET /admin/masters/:id
response and change GlobalQPS default to 0 with validation to
reject negative values.
2025-12-22 12:02:27 +08:00
zenfun
fa7f92c6e3 feat(api): allow batch status updates 2025-12-21 23:32:07 +08:00
zenfun
c2ed2f3f9e feat(api): add namespaces, batch ops, and admin logs 2025-12-21 23:16:27 +08:00
zenfun
73147fc55a feat(api): add model delete, pagination, and cors config 2025-12-21 23:03:12 +08:00
zenfun
816ea93339 feat(arch): add log partitioning and provider delete sync 2025-12-21 20:45:16 +08:00
zenfun
f819f89ba2 test(ops): cover status test and model fetch 2025-12-21 20:23:04 +08:00
zenfun
f7baa0f08f feat(ops): add status test and model fetch 2025-12-21 20:22:56 +08:00
zenfun
6e6c669ea0 test(log): expand handler, config, and metrics coverage 2025-12-21 16:18:38 +08:00
zenfun
c2c65e774b feat(log): wire log db, metrics, and body toggle 2025-12-21 16:18:22 +08:00
zenfun
4c1e03f83d feat(api): add log webhook notification service
Implement webhook notifications for log error threshold alerts with
configurable thresholds, time windows, and cooldown periods.

- Add LogWebhookService with Redis-backed configuration storage
- Add admin endpoints for webhook config management (GET/PUT)
- Trigger webhook notifications when error count exceeds threshold
- Support status code threshold and error message detection
- Include sample log record data in webhook payload
2025-12-21 14:13:35 +08:00
zenfun
00192f937e feat(api): add log_request_body_enabled feature flag support
Add runtime feature flag to control whether request bodies are stored
in logs. The Handler now accepts a Redis client to check the
log_request_body_enabled feature flag before persisting log records.

- Add logRequestBodyFeatureKey constant for feature flag
- Inject Redis client into Handler for feature flag lookups
- Strip request body from log records when feature is disabled
- Update tests to pass Redis client to NewHandler
2025-12-21 13:26:16 +08:00
zenfun
1089f9490e test(api): add feature handler tests for regular and mixed keys
Add tests for UpdateFeatures endpoint covering:
- Regular feature flag keys with various value types
- Mixed keys combining feature flags with log retention settings
- Redis storage verification for both scenarios
2025-12-21 13:07:00 +08:00
zenfun
2f58dd76a7 feat(api): add log retention and max records feature overrides
Add special handling for log_retention_days and log_max_records features
that are stored as separate Redis keys instead of in the features hash.

- Store log overrides in dedicated keys (meta:log:retention_days,
  meta:log:max_records) for runtime access by log cleanup cron
- Include log override values in ListFeatures response
- Support clearing overrides by setting value to 0
- Add comprehensive tests for log override CRUD operations
2025-12-21 12:26:48 +08:00
zenfun
25795a79d6 feat(cron): add automatic log cleanup with retention policy
Implement LogCleaner cron job to automatically clean up old log records
based on configurable retention period and maximum record count.

- Add LogCleaner with retention_days and max_records configuration
- Add EZ_LOG_RETENTION_DAYS and EZ_LOG_MAX_RECORDS environment variables
- Default to 30 days retention and 1,000,000 max records
- Include unit tests for log cleaner functionality
2025-12-21 12:01:52 +08:00
zenfun
369c204c16 feat: add admin log deletion 2025-12-21 00:53:52 +08:00
zenfun
88289015fc feat: add internal stats flush API 2025-12-19 23:26:33 +08:00
zenfun
ac9f0cd0a7 feat(stats): add usage stats and quota reset 2025-12-19 21:50:28 +08:00
zenfun
524f8c5a4e feat(key): extend key metadata and validation 2025-12-19 21:24:24 +08:00
zenfun
5e98368428 model-registry: default supports_stream true 2025-12-18 16:48:43 +08:00
zenfun
7dd3fac24e model-registry: add upstream check endpoint 2025-12-18 16:43:12 +08:00
zenfun
a61eff27e7 feat(admin/master): provider+master CRUD, token mgmt, logs APIs 2025-12-18 16:21:46 +08:00
zenfun
b2d2df18c5 feat(model-registry): models.dev updater + admin endpoints 2025-12-17 23:59:34 +08:00
zenfun
96e1fe41a5 feat(models): add kind and models_meta snapshot 2025-12-17 23:15:12 +08:00
zenfun
2b0ed3d3d5 feat(api): align provider creation with presets/custom/google sdk 2025-12-17 22:19:17 +08:00
zenfun
174735f152 feat(api): add provider creation endpoints and weight support
- Add `POST /admin/providers/preset` for streamlined creation of official providers (OpenAI, Anthropic, Gemini)
- Add `POST /admin/providers/custom` for generic OpenAI-compatible providers
- Add `weight` field to provider model and DTOs to enable weighted routing
- Update sync service to propagate provider weights
- Add unit tests for new creation handlers
2025-12-17 21:40:54 +08:00
zenfun
5a4e63b75d test(service): add specs for binding synchronization
Add unit tests for the SyncBindings service covering exact matching,
regex selectors, and normalized matching logic using in-memory
dependencies (SQLite and MiniRedis) to ensure correct upstream
resolution.
2025-12-17 01:18:02 +08:00
zenfun
6d8a12df34 feat(api): add access control and binding endpoints
Add endpoints for master and key access management to configure default
and allowed namespaces, including propagation options.
Implement GET and DELETE operations for individual bindings.
Update sync service to persist bindings snapshots even when no upstreams
are available.
2025-12-17 01:08:01 +08:00
zenfun
16fceec8e7 style(core): align struct tags and fields
Standardize alignment of struct tags in models and field assignments in
services for better readability. Additionally, include the Binding model
in the test database auto-migration.
2025-12-17 00:51:02 +08:00
zenfun
ed6446e586 feat(api): add admin endpoints for binding management
Implement handlers for creating, listing, and updating model bindings.
Register new routes in the admin server group and add DTO definitions.
Update provider handlers to trigger binding synchronization on changes
to ensure upstream mappings remain current.
2025-12-17 00:37:02 +08:00
zenfun
2e3b471533 feat(core): implement namespace support and routing bindings
Introduce namespace-aware routing capabilities through a new Binding
model and updates to Master/Key entities.

- Add Binding model for configuring model routes per namespace
- Add DefaultNamespace and Namespaces fields to Master and Key models
- Update auto-migration to include Binding table
- Implement Redis synchronization for binding configurations
- Propagate namespace settings during master creation and key issuance
2025-12-17 00:33:36 +08:00
zenfun
51e55cb036 feat(api): add update endpoints for providers and models
Register admin routes to handle updates for providers and models via
their respective handlers.
2025-12-15 16:22:54 +08:00
zenfun
ca03ac1b79 docs(api): update swagger documentation for admin endpoints
Reflect recent API changes in Swagger documentation:
- Add endpoints for feature flag management (/admin/features)
- Add endpoint for issuing child keys to masters (/admin/masters/{id}/keys)
- Add endpoint for updating providers (/admin/providers/{id})
- Update provider model definitions with new fields (auto_ban, google_* attributes, status)
2025-12-15 16:12:56 +08:00
zenfun
aa69ce3659 feat(api): add admin endpoint to issue keys for masters
Add `POST /admin/masters/{id}/keys` allowing admins to issue child keys
on behalf of a master. Introduce an `issued_by` field in the Key model
to audit whether a key was issued by the master or an admin.

Refactor master service to use typed errors for consistent HTTP status
mapping and ensure validation logic (active status, group check) is
shared.
2025-12-15 15:59:33 +08:00
zenfun
11f6e81798 feat(api): add feature flag management endpoints
Add FeatureHandler to manage lightweight runtime configuration toggles
stored in the Redis `meta:features` hash. This enables dynamic control
over system behavior (e.g., storage backends) via the admin API.

- Add `GET /admin/features` to list flags
- Add `PUT /admin/features` to update flags
- Update README with feature flag documentation
2025-12-15 15:01:01 +08:00
zenfun
fb68e0f155 docs(testing): point provider snapshot golden to foundation 2025-12-14 23:54:57 +08:00
zenfun
85223cc31d chore(deps): bump foundation to v0.2.0 2025-12-14 23:54:12 +08:00
zenfun
d1d1b1c42a refactor(deps): use foundation shared utilities 2025-12-14 23:52:46 +08:00
zenfun
71c183a480 test(service): add golden file validation and master key tests
- Update TESTING.md to reflect current testing status and future plans
- Add golden file comparison for provider snapshot validation in sync_test.go
- Introduce master_test.go for testing Master/Key functionality
- Add testdata directory for contract testing snapshots
2025-12-14 23:37:16 +08:00
zenfun
d0011f3eb2 test: add comprehensive unit tests for provider, middleware, and sync service
- Add TESTING.md documentation explaining unit test conventions and integration testing setup
- Add miniredis and sqlite dependencies to go.mod for in-memory testing
- Add provider_handler_test.go ensuring Vertex providers default google_location to "global"
- Add request_id_test.go verifying request ID generation and header preservation
- Add sync_test.go validating Redis snapshot writes and routing key generation
- Update README.md with testing section referencing new documentation
2025-12-14 00:35:35 +08:00
zenfun
50da2ff668 chore: bump foundation to v0.1.0 2025-12-14 00:03:49 +08:00
zenfun
531ac6e5a8 feat(telemetry): implement request tracing identifier
Introduce a middleware layer to attach a unique identifier to each HTTP request for observability purposes. The identifier is propagated via the X-Request-ID header, allowing for correlation of logs and events across distributed system components.
2025-12-13 23:49:58 +08:00