Move the responsibility of sequence (`seq`) allocation from the application/cache layer to the database layer to ensure absolute consistency. Previously, sequences were pre-allocated via Redis/Lua, which introduced complexity in managing synchronization between the cache and the database.
Key changes:
- Implement `CreateMessageWithSeq` in `message_repo.go` using `SELECT FOR UPDATE` to lock the conversation row and atomically increment the `last_seq` within a single transaction.
- Simplify `SeqBufferManager` by removing the complex local buffering and Lua-based pre-allocation logic, reverting to a simpler model.
- Update `chat_service.go`, `group_service.go`, and `message_service.go` to remove manual sequence retrieval, relying instead on the repository's transactional allocation.
- Introduce `SyncConvSeq` in `conversation_cache.go` to perform "write-through" updates to Redis, ensuring the cache remains synchronized with the database's source of truth.
- Improve cold-start handling in `conversation_cache.go` with a new `syncSeqLua` script to prevent stale sequence reads.
Implement Lua scripts for atomic sequence incrementing and cold-start initialization in Redis to prevent race conditions. Improve the sequence buffer manager with CAS-like logic to prevent interval leakage during concurrent allocations. Update the message repository to use `SELECT ... FOR UPDATE` and conditional updates for `last_seq` to ensure database consistency during high concurrency.
- Add `nextSeqLua` and `initSeqLua` for atomic Redis operations
- Implement thread-safe buffer management in `seq_buffer.go`
- Add row-level locking in `message_repo.go`
- Update service layers to support group-aware sequence retrieval
Introduce several performance and synchronization enhancements:
- Implement `SeqBufferManager` to allow sequence number pre-allocation via Redis Lua scripts, reducing atomic increment overhead.
- Add `PushWorker` to handle asynchronous message pushing using Redis Streams.
- Implement incremental conversation synchronization via `ConversationVersionLog` to allow clients to fetch only recent changes.
- Add support for Gzip compression in WebSocket communications to reduce bandwidth usage.
- Update dependency injection and configuration to support these new components.
Remove the fallback mechanism that queried the database when Redis keys were missing or unavailable. The `GetConvMaxSeq` and `GetConvMaxSeqBatch` methods now strictly rely on the Redis cache and return `ErrKeyNotFound` if the data is not present.
Improve the reliability of conversation sequence generation by implementing a fallback mechanism to the database when Redis is unavailable or when keys are missing.
- Add `GetNextSeq` to `MessageRepository` and implement it in `MessageRepositoryAdapter`.
- Update `ConversationCache` to fallback to DB in `GetConvMaxSeq`, `GetConvMaxSeqBatch`, and `GetNextSeq`.
- Ensure Redis TTL is refreshed after every `INCR` operation to prevent sequence counter expiration.
- Refactor service layers to handle sequence generation errors more explicitly.
- Add `IsEnabled` helper to `LayeredCache` for cleaner availability checks.
Clean up the codebase by removing unused and deprecated unread count
functionality in `ConversationCache` and associated Redis key
generators. This follows the transition to sequence-based unread
calculation.
- Remove `IncrementUnread`, `ClearUnread`, `GetUnreadCountFromHash`,
`GetAllUnreadCounts`, and `GetTotalUnread` from `ConversationCache`.
- Remove `PrefixUnreadHash` and `PrefixUnreadTotal` constants and
`UnreadHashKey`/`UnreadTotalKey` helper functions.
- Remove unused `isValidContentType` helper in `MessageHandler`.
Refactor unread count logic to use arithmetic calculation (maxSeq - readSeq)
instead of manual incrementing/decrementing. This simplifies the cache
management and improves consistency.
New features:
- Batch mark-as-read functionality for multiple conversations.
- Lightweight sync data endpoint to retrieve conversation metadata
(maxSeq and last message timestamp) for client synchronization.
- Optimized batch retrieval of conversation max sequences using Redis MGet.
Technical changes:
- Deprecated `IncrementUnread` and `ClearUnread` in `ConversationCache`.
- Added `GetConvMaxSeqBatch` to reduce network round-trips.
- Added `HandleMarkReadAll` and `HandleGetSyncData` handlers.
- Updated `ChatService` to support batch operations and sync data retrieval.
Add new functionality for managing and synchronizing academic grades and exam schedules. This includes new models, repositories, services, and HTTP handlers, along with updated gRPC definitions and dependency injection configuration.
Additionally, improve the reliability of unread message count tracking by implementing an atomic Lua script for Redis operations in the conversation cache.
Introduce a new mechanism for tracking read positions using message
sequences (hasReadSeq), similar to OpenIM, to allow for O(1) unread
count calculations.
- Implement `UserReadSeq` caching in Redis with Lua scripts to prevent
sequence regression (ensuring updates only occur if the new sequence
is greater than the current one).
- Update `ConversationCache` to support sequence-based unread count
computation (`maxSeq - hasReadSeq`).
- Refactor `MessageRepository` to use conditional updates for
`last_read_seq` in the database.
- Update `ChatService` to automatically update the sender's read
sequence when sending a message.
- Optimize `MarkAsRead` logic to update both database and Redis
sequence caches and refine notification targets for private vs
group chats.
- Update `jpush` client to use pointer types for boolean fields to
properly handle optional values in JSON.
This commit introduces several significant architectural improvements to enhance system performance, scalability, and reliability:
- **Performance Optimization (N+1 Problem Resolution)**: Implemented batching mechanisms across `MessageHandler`, `ChatService`, `GroupService`, `MessageService`, and `UserService`. This replaces multiple individual database/cache queries with single batch operations for fetching unread counts, last messages, participants, and user information.
- **Enhanced Unread Count Management**: Migrated unread count tracking to a Redis Hash-based approach (`unread#️⃣{userID}`). This allows for atomic increments/decrements and efficient retrieval of both individual conversation unread counts and total unread counts for a user.
- **Improved Message Sequencing**: Integrated Redis-based sequence (`seq`) pre-allocation for messages to ensure strict ordering and reduce database contention during high-concurrency message creation.
- **Push Service Reliability**: Refactored the push notification system to include persistent `PushRecord` tracking. Added a recovery mechanism (`recoverPendingPushes`) to reload pending notifications from the database upon service startup, ensuring better delivery guarantees.
- **WebSocket Reliability**: Updated the WebSocket hub to move away from in-memory history replay in favor of a client-driven synchronization model (`sync_required` event and `ack` handling), reducing memory overhead and improving connection stability.
- **Cache Layer Enhancements**: Added `HIncrBy` and `IncrBySeq` to the `Cache` interface and its implementations (`RedisCache`, `LayeredCache`) to support the new unread and sequence management logic.
This commit introduces several architectural improvements and optimizations across the codebase:
- **Performance & Reliability**:
- Implemented Redis pipelining in `ConversationCache.CacheMessage` to reduce network round-trips.
- Added a circuit breaker to the JPush client to prevent cascading failures.
- Introduced batch deletion and batch member addition capabilities in repositories.
- Added message idempotency support using `client_msg_id` and a Redis-based cache.
- Optimized WebSocket handling with connection limits (total and per-user) and improved error logging.
- **Code Refactoring**:
- Refactored `Router` to use a `RouterDeps` struct, simplifying the constructor and improving maintainability.
- Unified model ID generation logic using new `id_helper.go` (supporting UUID and Snowflake).
- Standardized JSON serialization/deserialization in models using `json_helper.go`.
- Refactored DTO conversion logic, specifically for `UserResponse` (using functional options) and `Report` responses.
- Removed redundant/deprecated DTOs like `PostDetailResponse` and `TradeItemDetailResponse`.
- **Cache Improvements**:
- Enhanced `LayeredCache` with `SetRaw` to avoid double-encoding when promoting values from Redis to local cache.
- Added `DeleteBatch` support to the cache interface.
- **Other Changes**:
- Cleaned up `config.go` by removing redundant default values and explicit environment variable overrides.
- Improved WebSocket registration flow to handle connection limits gracefully.
The Go module name has been changed from `carrot_bbs` to `with_you`, which requires
all import paths to be updated throughout the codebase and by external consumers.
BREAKING CHANGE: Module name changed from carrot_bbs to with_you. All imports
and dependencies must be updated from carrot_bbs/* to with_you/*.
- Migrate all log.Printf/log.Println calls to structured zap logging
- Use cmp.Or for cleaner default value handling
- Replace manual loops with slices.ContainsFunc/IndexFunc
- Add batch query methods (IsLikedBatch, IsFavoritedBatch) to solve N+1 problem
- Update JSON tags from omitempty to omitzero for numeric fields
- Use errgroup-style wg.Go for worker goroutines
- Remove duplicate casbin/v2 dependency (keeping v3)
- Add plans/ to gitignore
- Introduced new logging services: operation logs, login logs, and data change logs.
- Added support for message content encryption in the Message model, including methods for encrypting and decrypting message segments.
- Updated router to include admin log handling and integrated log service for access logging.
- Modified configuration to support encryption settings.
Add complete schedule functionality including:
- Schedule screen with weekly course table view
- Course detail screen with transparent modal presentation
- New ScheduleStack navigator integrated into main tab bar
- Schedule service for API interactions
- Type definitions for course entities
Also includes bug fixes for group invite/request handlers
to include required groupId parameter.
