Commit Graph

165 Commits

Author SHA1 Message Date
zenfun
e7db9f319f fix: delete keys and seed only active ones
Ensure admin key deletion removes the DB record and returns a
"deleted" status. Update seeder idempotency to count only active keys
when deciding whether to skip or create new keys.
2026-01-10 01:18:04 +08:00
zenfun
5349c9c833 feat(api): add admin master key listing/revoke
Add admin endpoints to list and revoke child keys under a master.
Standardize OpenAPI responses to use ResponseEnvelope with MapData
for error payloads, and regenerate swagger specs accordingly.
2026-01-10 01:10:36 +08:00
zenfun
ac6a1858cf feat(api): align response envelope schema
Switch response envelope business code to numeric and make message
consistently present. Add trace_id and optional details, and remove the
duplicate DTO envelope definition. Improve middleware path exclusion
handling and add a time-based trace ID fallback if crypto RNG fails.

BREAKING CHANGE: response envelope `code` is now `int` (was `string`) and
`message` semantics/defaults changed; clients must update parsing.
2026-01-10 01:09:05 +08:00
zenfun
26733be020 feat(api): standardize response envelope behavior
Add shared response DTOs and enhance the response envelope middleware with
excluded paths, trace ID generation fallback, and automatic extraction of
error details from handler responses. Update default business code mapping
for 503 and 504, and adjust idempotency detection to only treat the new
envelope format as already-wrapped.

BREAKING CHANGE: responses using the old envelope format (e.g., string
`code`) are now wrapped into the new standard envelope.
2026-01-10 00:59:45 +08:00
zenfun
6af938448e fix(seeder): improve key idempotency and log names
Trim whitespace in provider model lists, format provider names as `group#keyID`
to match DP logs, and skip existing API keys during seeding (deleting on reset)
to keep runs idempotent and summaries accurate
2026-01-10 00:58:02 +08:00
zenfun
5431e24923 fix(seeder): correct log generation fields
- Parse provider group models from API response string and expose as slice
- Send `model` field (not `model_name`) when creating logs
- Use API key ID as `provider_id` instead of provider group ID
- Restrict reset behavior to resources matching seeder tag/prefix
- Refactor usage sample generation to accept a context struct
2026-01-10 00:46:03 +08:00
zenfun
dd32ee817d test(api): align tests with new envelope codes
Update API handler tests to expect numeric `code`, `success` messages, and
new envelope fields (`trace_id`, `details`), matching recent response
envelope changes.
2026-01-10 00:39:35 +08:00
zenfun
cb3b7e8230 feat(api): enrich response envelope metadata
Add numeric business codes, include `trace_id`, and support custom
error messages and `details` for error responses while keeping envelope
wrapping idempotent across old and new formats.

BREAKING CHANGE: response envelope `code` changes from string to int and
envelope format now includes `trace_id` (and may include `details`).
2026-01-10 00:33:46 +08:00
zenfun
24cf1a6a30 chore(config): ignore seeder directory 2026-01-10 00:27:06 +08:00
zenfun
18b9846f83 feat(seeder): add control plane data seeder
Introduce a `cmd/seeder` CLI to generate deterministic demo datasets and
seed them into the Control Plane via admin endpoints, supporting reset,
dry-run, profiles, and usage sample generation.

Add Cobra/Viper dependencies to support the new command.
2026-01-10 00:26:48 +08:00
zenfun
33838b1e2c feat(api): wrap JSON responses in envelope
Add response envelope middleware to standardize JSON responses as
`{code,data,message}` with consistent business codes across endpoints.
Update Swagger annotations and tests to reflect the new response shape.

BREAKING CHANGE: API responses are now wrapped in a response envelope; clients must read payloads from `data` and handle `code`/`message` fields.
2026-01-10 00:15:08 +08:00
zenfun
f400ffde95 refactor(api): update traffic chart response structure
Change the traffic chart API response from bucket-based to series-based
to better support frontend visualization libraries. The new format
provides a shared X-axis and aligned data arrays for each model series.

- Replace `buckets` with `x` and `series` in response
- Implement data alignment and zero-filling for time slots
- Update Swagger documentation including pending definitions

BREAKING CHANGE: The `GET /admin/logs/stats/traffic-chart` response
schema has changed. `buckets` and `models` fields are removed.
2026-01-08 18:40:44 +08:00
341b54b185 docs(api): add ModelMetricsMap type and improve breakdown field documentation
Extract ModelMetricsMap as a named type with documentation comments
explaining the map structure and providing examples. Update the
TrafficBucket.Breakdown field to use the new type and enhance the
GetTrafficChart endpoint description with detailed breakdown field
documentation including example usage.
2026-01-08 16:31:34 +08:00
b71c7f2541 style(api): align struct field tags and fix string concatenation
- Fix JSON field alignment in error response and TrafficBucket struct
- Add documentation comment and swagger annotations for Breakdown field
- Remove unnecessary string concatenation spacing in SQL select
2026-01-08 16:27:37 +08:00
zenfun
e35e801839 feat(service): add provider metadata tracking
Write 'meta:providers_meta' to Redis during provider synchronization,
including version, timestamp, and configuration checksum. This aligns
provider sync with model metadata handling and enables better cache
invalidation.
2026-01-07 13:43:12 +08:00
c19c26817d build: add docker build and push script
Add a shell script to automate the Docker image build and push process
targeting the configured Harbor registry.
2026-01-07 09:41:01 +08:00
a2bc994100 docs(api): expand whoami endpoint documentation with detailed response fields
Add comprehensive API documentation for the whoami endpoint including:
- Detailed response structure for each token type (Admin, Master, Child Key)
- Complete field listings for master keys and child keys
- All possible error responses with status codes
- Usage guidance for frontend initialization
2026-01-06 09:21:02 +08:00
1ee6bea413 feat(api): enhance whoami endpoint with realtime stats and extended key info
Add realtime statistics (requests, tokens, QPS, rate limiting) to whoami
response for both master and key authentication types. Extend key response
with additional fields including master name, model limits, quota tracking,
and usage statistics.

- Inject StatsService into AuthHandler for realtime stats retrieval
- Add WhoamiRealtimeView struct for realtime statistics
- Include admin permissions field in admin response
- Add comprehensive key metadata (quotas, model limits, usage stats)
- Add test for expired key returning 401 Unauthorized
2026-01-06 09:15:49 +08:00
zenfun
f99e4a15ab test(service): remove legacy routing assertions
Updates TestSyncProviders_WritesSnapshot by removing assertions for
routing keys that are no longer generated following the legacy routing
removal.
2026-01-04 13:49:07 +08:00
zenfun
05cba292d4 refactor(service): remove legacy routing and unused code
Removes the legacy route table maintenance logic from the sync service
that populated deprecated Redis keys. Additionally, deletes the unused
TokenService and KeyDTO files to reduce technical debt.
2026-01-04 12:07:37 +08:00
zenfun
4b22b759e7 test(service): add tests for IP ban update edge cases
Add integration tests for `IPBanService.Update` to verify:
- Reactivating an expired ban correctly detects overlaps with existing active bans.
- Explicitly clearing the `expires_at` field (setting to null) works as expected.
2026-01-04 01:45:31 +08:00
zenfun
a7571dd4ad feat(server): integrate ip ban cron and refine updates
- Initialize and schedule IP ban maintenance tasks in server entry point
- Perform initial IP ban sync to Redis on startup
- Implement optional JSON unmarshalling to handle null `expires_at` in API
- Add CIDR overlap validation when updating rule status to active
2026-01-04 01:44:45 +08:00
zenfun
830c6fa6e7 feat(cron): implement IP ban maintenance tasks
Add IPBanManager to handle periodic background jobs including:
- Expiring outdated bans
- Syncing hit counts from Redis to DB
- Performing full Redis state synchronization

Additionally, update the service expiration logic to use system time
and add unit tests for CIDR normalization and overlap checking.
2026-01-04 01:28:43 +08:00
zenfun
63d43db39d feat(server): register IP ban routes in admin group
Initialize the IP ban service and handler, and wire up the CRUD
endpoints to the admin router group.
2026-01-04 01:26:55 +08:00
zenfun
7393bf6dc3 feat(api): add IP ban handler for CRUD operations
Implement the HTTP handler for managing global IP/CIDR bans. This
includes endpoints for creating, listing, retrieving, updating, and
deleting IP ban rules, complete with Swagger documentation and error
handling.
2026-01-04 01:00:44 +08:00
zenfun
2359603666 feat(service): implement IP ban service logic
Add IPBanService to manage global IP bans with Redis synchronization
for high-performance filtering. Includes logic for CIDR normalization,
overlap detection, hit count tracking, and rule expiration.
2026-01-04 00:59:03 +08:00
zenfun
ae2f4d7819 feat(model): add IPBan entity for global IP blocking
Introduces the IPBan model to support global IP/CIDR ban rules enforced by the data plane. Includes fields for CIDR, status, expiration, and hit counts, and registers the model for auto-migration in the server startup.
2026-01-04 00:55:00 +08:00
zenfun
f4d6f64068 docs(config): add trusted proxy and cdn details to .env.example
Update comments for EZ_BALANCER_TRUSTED_PROXIES to include:
- Header resolution priority (CF-Connecting-IP, Ali-CDN-Real-IP, etc.)
- Current Cloudflare IPv4/IPv6 CIDR lists for easier reference
- Specific notes on production configuration guidelines
2026-01-03 23:52:02 +08:00
zenfun
8a52d58674 docs(admin): add dashboard feature design and mockup assets
Organize admin panel feature documentation into a dedicated directory
and include an interactive HTML mockup along with a reference
screenshot for the EZ-API Control Plane Dashboard.
2026-01-03 17:11:02 +08:00
zenfun
f938abbdfa docs(config): update .env.example with production guidelines
Add detailed comments explaining production vs development configuration
recommendations, particularly regarding security and retention policies.
Update default values for balancer log sink and stats flush to enabled.
2026-01-03 17:07:57 +08:00
zenfun
ce8d68c897 docs(config): add details for balancer log and stats
Add explanatory comments for EZ_BALANCER_LOG_SINK_ENABLED and
EZ_BALANCER_STATS_FLUSH_ENABLED to clarify default behavior and
production recommendations regarding monitoring and quota management.
2026-01-03 17:07:23 +08:00
zenfun
776eb98ebd docs(config): improve .env.example organization and docs
Reorganize configuration variables into logical sections with clear
headers and detailed comments to enhance readability. Add missing log
buffering settings (EZ_LOG_QUEUE, EZ_LOG_BATCH_SIZE, EZ_LOG_FLUSH_MS)
and expand descriptions for authentication and network options.
2026-01-03 16:57:21 +08:00
zenfun
9cfc1b2c97 docs(config): document new internal auth and balancer variables
Update .env.example with new configuration options:
- EZ_INTERNAL_ALLOW_ANON for controlling anonymous internal access
- EZ_BALANCER_ENABLE_TEST_KEYS for testing auth bypass
- EZ_BALANCER_TRUSTED_PROXIES for real IP resolution

Add security configuration section to README explaining internal endpoint
authentication logic and default behaviors.
2026-01-03 16:52:23 +08:00
zenfun
03ec45c84c fix(auth): improve token metadata parsing and validation
Safeguard integer parsing in the `Whoami` handler by trimming whitespace and handling errors explicitly for `master_id`, `issued_at_epoch`, and `expires_at`. This prevents potential validation bypasses or incorrect behavior due to malformed metadata.

Add unit tests to verify invalid epoch handling and response correctness.
2026-01-03 16:19:42 +08:00
zenfun
8b63aa6ec0 test(middleware): add unit tests for internal auth middleware
Add comprehensive test coverage for InternalAuthMiddleware including scenarios
for allowed anonymous access, missing tokens, invalid tokens, and empty token
configuration to ensure access control logic correctness.
2026-01-03 16:18:59 +08:00
zenfun
4cd9b66a84 feat(auth): enhance token validation and internal access control
Refactor the `Whoami` handler to validate token metadata (status, expiration,
revocation) against Redis before database lookup, ensuring consistency with
balancer logic. Add `allow_ips`, `deny_ips`, and `expires_at` fields to
authentication responses.

Update internal middleware to support explicit anonymous access configuration
and harden security for unconfigured tokens.

Remove legacy fallback logic for master keys without digests.

BREAKING CHANGE: Internal endpoints now reject requests by default if no stats token is configured. To allow unauthenticated access, set `internal.allow_anonymous` to true.
BREAKING CHANGE: Support for legacy master keys without stored digests has been removed.
2026-01-03 16:04:04 +08:00
zenfun
295faa8e01 docs(admin): update dashboard feature specification
- Add detailed interaction tables for system status, metrics, and charts
- Update API field mappings to match backend implementation
- Clarify error handling, loading states, and edge cases
2026-01-02 23:39:18 +08:00
zenfun
47991bd5f3 docs(api): update swagger documentation for traffic charts and trends
Regenerate API documentation to reflect recent statistics features:
- Add definition for new `/admin/logs/stats/traffic-chart` endpoint
- Update dashboard summary with `include_trends` parameter and new time periods
- Add `DashboardTrends` and `TrafficChartResponse` data structures
- Update alert types to include `traffic_spike
2026-01-02 23:22:36 +08:00
zenfun
5c01497ce0 fix(api): handle zero-baseline edge cases in trend calculation
Introduce `CalculateTrendFloatWithBaseline` to correctly handle scenarios where previous period metrics (Error Rate, Latency) are zero or missing. This prevents arithmetic errors and distinguishes between "new" data and actual increases ("up") when starting from zero.

Also updates the admin panel dashboard documentation to reflect current project status.
2026-01-02 23:17:55 +08:00
zenfun
481f616704 refactor(stats): remove daily stats aggregation
Remove the DailyStatsJob, DailyStat model, and associated database
migrations. This eliminates the pre-aggregation layer and updates the
dashboard handler to remove dependencies on the daily_stats table.
2026-01-02 23:08:50 +08:00
zenfun
08a8a1e42f feat(api): add trend analysis and extended periods to dashboard summary
- Add `include_trends` query parameter to enable trend calculation
- Implement trend comparison logic (delta % and direction) against previous periods
- Add support for `last7d`, `last30d`, and `all` time period options
- Update `DashboardSummaryResponse` to include optional `trends` field
- Add helper functions for custom time window aggregation
- Add unit tests for trend calculation and period window logic
- Update feature documentation with new parameters and response schemas
2026-01-02 22:30:38 +08:00
zenfun
5b2b176a55 feat(stats): add daily statistics aggregation job and model
- Create `DailyStat` model for immutable daily metrics including
  request counts, tokens, latency, and top models.
- Implement `DailyStatsJob` to aggregate `log_records` from the previous
  day, running daily at 00:05 UTC.
- Register database migrations and schedule the job in the server.
- Add `last7d` and `last30d` period support to stats handler.
2026-01-02 22:20:37 +08:00
zenfun
7b20c35fba docs(admin): clarify dashboard key metrics and limitations
Update dashboard summary specification to distinguish between provider
keys (upstream) and internal keys. Change summary metrics to use
`provider_keys` fields for better clarity.

Add section on known limitations regarding time period logic and
missing trend data.
2026-01-02 22:07:18 +08:00
zenfun
0ba94026d4 docs(admin): update dashboard alert severity color mapping
Update the UI specification to distinguish warning severity with orange
color instead of grouping it with critical (red). Also remove redundant
project overview table.
2026-01-02 21:53:50 +08:00
zenfun
e5624c62f1 docs(admin): update dashboard integration specs
Restructure documentation into tables for improved readability and
include detailed specifications for the new traffic chart endpoint.
Define explicit data refresh strategies, error handling guidelines, and
response structures for admin panel components.
2026-01-02 21:45:46 +08:00
zenfun
b1fe1ecb97 docs(admin): add dashboard integration documentation
Add detailed specification for the EZ-API Control Plane Dashboard
frontend-backend integration. This document defines the data mapping,
API endpoints, and UI logic for global navigation, real-time metrics,
traffic analysis, and alert summaries.
2026-01-02 21:40:55 +08:00
zenfun
bfb19ca23e test(api): add validation tests for traffic chart endpoint
Add TestTrafficChart_MinuteGranularityValidation to verify input
parameters including granularity, time range limits, and top_n
constraints. Include skipped placeholders for PostgreSQL-specific
aggregation tests.
2026-01-02 21:35:42 +08:00
zenfun
9d082ff375 feat(api): add admin traffic chart statistics endpoint
Add new endpoint GET /admin/logs/stats/traffic-chart to provide
aggregated traffic metrics grouped by time and model. Features include:
- Time granularity selection (hour/minute)
- Top-N model breakdown with "other" aggregation
- Metrics for request counts and token usage

Includes generated Swagger documentation.
2026-01-02 21:24:56 +08:00
zenfun
bae3d9bd5b refactor(scheduler): add base context for graceful shutdown
- Create application-level context with cancel function
- Pass base context to scheduler via WithBaseContext option
- Move scheduler.Stop() to explicit shutdown sequence after context cancellation
- Upgrade foundation dependency to v0.7.0 for new scheduler options
2026-01-01 01:44:49 +08:00
zenfun
31914b9ab5 refactor(scheduler): migrate outbox and model registry to scheduler-based execution
Replace internal goroutine-based timing loops with scheduler integration
for SyncOutboxService and ModelRegistryService. Both services now expose
RunOnce() methods called by the central scheduler instead of managing
their own background loops.

- Add Interval() and RunOnce() methods to SyncOutboxService
- Add RefreshEvery() and RunOnce() methods to ModelRegistryService
- Remove started flag from SyncOutboxService struct
- Move scheduler.Start() after all services are initialized
- Ensure initial model registry refresh before scheduler starts
2026-01-01 00:55:51 +08:00