Replace the manual WebRTC signaling implementation with LiveKit SFU. This includes:
- Adding LiveKit service, handler, and configuration.
- Updating Docker Compose to include LiveKit server, Redis, and PostgreSQL.
- Refactoring `CallService` and `WSHandler` to support LiveKit room readiness instead of raw SDP/ICE relaying.
- Adding new API endpoints for LiveKit token generation and webhooks.
- Removing deprecated WebRTC configuration and manual signaling DTOs.
Introduce several performance and synchronization enhancements:
- Implement `SeqBufferManager` to allow sequence number pre-allocation via Redis Lua scripts, reducing atomic increment overhead.
- Add `PushWorker` to handle asynchronous message pushing using Redis Streams.
- Implement incremental conversation synchronization via `ConversationVersionLog` to allow clients to fetch only recent changes.
- Add support for Gzip compression in WebSocket communications to reduce bandwidth usage.
- Update dependency injection and configuration to support these new components.
Introduce a new WebSocket messaging architecture that supports both standalone and cluster modes. This allows for horizontal scaling of WebSocket servers by using Redis Pub/Sub to synchronize messages across multiple instances.
Key changes:
- Added `ws.MessagePublisher` interface to abstract message distribution.
- Implemented `ws.Bus` to handle cluster-mode messaging via Redis.
- Added `ws.OnlineTracker` to manage user online status across the cluster.
- Refactored multiple services (Chat, Group, Push, Call, etc.) to use the new `MessagePublisher` instead of a concrete `ws.Hub`.
- Added WebSocket configuration options (mode, instance ID, channel, TTL, heartbeat) to `config.yaml` and `config.go`.
- Updated dependency injection with Wire to support the new publisher and Redis client.
- Improved logging by replacing standard `log` with `zap` in several service components.
This commit introduces several significant architectural improvements to enhance system performance, scalability, and reliability:
- **Performance Optimization (N+1 Problem Resolution)**: Implemented batching mechanisms across `MessageHandler`, `ChatService`, `GroupService`, `MessageService`, and `UserService`. This replaces multiple individual database/cache queries with single batch operations for fetching unread counts, last messages, participants, and user information.
- **Enhanced Unread Count Management**: Migrated unread count tracking to a Redis Hash-based approach (`unread#️⃣{userID}`). This allows for atomic increments/decrements and efficient retrieval of both individual conversation unread counts and total unread counts for a user.
- **Improved Message Sequencing**: Integrated Redis-based sequence (`seq`) pre-allocation for messages to ensure strict ordering and reduce database contention during high-concurrency message creation.
- **Push Service Reliability**: Refactored the push notification system to include persistent `PushRecord` tracking. Added a recovery mechanism (`recoverPendingPushes`) to reload pending notifications from the database upon service startup, ensuring better delivery guarantees.
- **WebSocket Reliability**: Updated the WebSocket hub to move away from in-memory history replay in favor of a client-driven synchronization model (`sync_required` event and `ack` handling), reducing memory overhead and improving connection stability.
- **Cache Layer Enhancements**: Added `HIncrBy` and `IncrBySeq` to the `Cache` interface and its implementations (`RedisCache`, `LayeredCache`) to support the new unread and sequence management logic.
This commit introduces several architectural improvements and optimizations across the codebase:
- **Performance & Reliability**:
- Implemented Redis pipelining in `ConversationCache.CacheMessage` to reduce network round-trips.
- Added a circuit breaker to the JPush client to prevent cascading failures.
- Introduced batch deletion and batch member addition capabilities in repositories.
- Added message idempotency support using `client_msg_id` and a Redis-based cache.
- Optimized WebSocket handling with connection limits (total and per-user) and improved error logging.
- **Code Refactoring**:
- Refactored `Router` to use a `RouterDeps` struct, simplifying the constructor and improving maintainability.
- Unified model ID generation logic using new `id_helper.go` (supporting UUID and Snowflake).
- Standardized JSON serialization/deserialization in models using `json_helper.go`.
- Refactored DTO conversion logic, specifically for `UserResponse` (using functional options) and `Report` responses.
- Removed redundant/deprecated DTOs like `PostDetailResponse` and `TradeItemDetailResponse`.
- **Cache Improvements**:
- Enhanced `LayeredCache` with `SetRaw` to avoid double-encoding when promoting values from Redis to local cache.
- Added `DeleteBatch` support to the cache interface.
- **Other Changes**:
- Cleaned up `config.go` by removing redundant default values and explicit environment variable overrides.
- Improved WebSocket registration flow to handle connection limits gracefully.
- Replace `interface{}` with `any` type alias across all packages
- Use built-in `min()`/`max()` for parameter clamping
- Use `slices.SortFunc`, `slices.Min`, `slices.Max` for cleaner code
- Use `strings.Cut()` for simpler string parsing in auth middleware
- Use `errors.Is()` for proper error comparison in handlers
- Update dependencies: golang.org/x/image 0.37.0 -> 0.38.0
- Add Wire code generation guidelines to ARCHITECTURE.md
- Disable Go cache in CI build workflow
- Updated CallService to handle user connection and disconnection events, allowing for automatic management of active calls based on user status.
- Introduced methods to retrieve active calls for users and send pending call invites when users reconnect.
- Modified the Invite method to return the callee's online status, improving user experience during call invitations.
- Enhanced WSHandler to support connection and disconnection event handlers, ensuring accurate call state management.
- Introduced a new method, PublishToUserOnlineReliable, to ensure critical signaling messages are reliably sent to online users.
- Updated CallService to include a cleanup mechanism for expired calls, improving resource management and user experience.
- Enhanced call acceptance logic to utilize optimistic locking for state updates, ensuring accurate call status handling across devices.
- Refactored wire generation to accommodate the new database dependency in CallService, streamlining service initialization.
- Added error handling for call invite and answer processes to differentiate between callee offline and call already answered scenarios.
- Introduced a new method to publish messages only to online users, improving real-time communication during call signaling.
- Updated call service to check callee's online status before sending call invites, ensuring better user experience.
- Enhanced call acceptance logic to notify both caller and callee about call status changes, including when a call is answered on another device.
- Added WebSocket support by introducing a new WSHandler and related infrastructure.
- Replaced SSE implementations with WebSocket equivalents across message and QR code handlers.
- Updated service and repository layers to utilize WebSocket for real-time messaging.
- Removed obsolete SSE hub and related code, streamlining the application for WebSocket usage.
- Enhanced router to include WebSocket endpoints for real-time communication.
