Add new functionality for managing and synchronizing academic grades and exam schedules. This includes new models, repositories, services, and HTTP handlers, along with updated gRPC definitions and dependency injection configuration.
Additionally, improve the reliability of unread message count tracking by implementing an atomic Lua script for Redis operations in the conversation cache.
Introduce a new mechanism for tracking read positions using message
sequences (hasReadSeq), similar to OpenIM, to allow for O(1) unread
count calculations.
- Implement `UserReadSeq` caching in Redis with Lua scripts to prevent
sequence regression (ensuring updates only occur if the new sequence
is greater than the current one).
- Update `ConversationCache` to support sequence-based unread count
computation (`maxSeq - hasReadSeq`).
- Refactor `MessageRepository` to use conditional updates for
`last_read_seq` in the database.
- Update `ChatService` to automatically update the sender's read
sequence when sending a message.
- Optimize `MarkAsRead` logic to update both database and Redis
sequence caches and refine notification targets for private vs
group chats.
- Update `jpush` client to use pointer types for boolean fields to
properly handle optional values in JSON.
- fix unread count calculation logic in message repository
- implement logic to retrieve other participant's last read sequence in conversation handlers
- mask real names in verification record responses for privacy
- improve error reporting in main entry point by using stderr instead of zap logger during initialization
GetLastMessagesBatch was returning empty segments because it skipped
the decryption step that all other message query methods perform.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit introduces several significant architectural improvements to enhance system performance, scalability, and reliability:
- **Performance Optimization (N+1 Problem Resolution)**: Implemented batching mechanisms across `MessageHandler`, `ChatService`, `GroupService`, `MessageService`, and `UserService`. This replaces multiple individual database/cache queries with single batch operations for fetching unread counts, last messages, participants, and user information.
- **Enhanced Unread Count Management**: Migrated unread count tracking to a Redis Hash-based approach (`unread#️⃣{userID}`). This allows for atomic increments/decrements and efficient retrieval of both individual conversation unread counts and total unread counts for a user.
- **Improved Message Sequencing**: Integrated Redis-based sequence (`seq`) pre-allocation for messages to ensure strict ordering and reduce database contention during high-concurrency message creation.
- **Push Service Reliability**: Refactored the push notification system to include persistent `PushRecord` tracking. Added a recovery mechanism (`recoverPendingPushes`) to reload pending notifications from the database upon service startup, ensuring better delivery guarantees.
- **WebSocket Reliability**: Updated the WebSocket hub to move away from in-memory history replay in favor of a client-driven synchronization model (`sync_required` event and `ack` handling), reducing memory overhead and improving connection stability.
- **Cache Layer Enhancements**: Added `HIncrBy` and `IncrBySeq` to the `Cache` interface and its implementations (`RedisCache`, `LayeredCache`) to support the new unread and sequence management logic.
This commit introduces several architectural improvements and optimizations across the codebase:
- **Performance & Reliability**:
- Implemented Redis pipelining in `ConversationCache.CacheMessage` to reduce network round-trips.
- Added a circuit breaker to the JPush client to prevent cascading failures.
- Introduced batch deletion and batch member addition capabilities in repositories.
- Added message idempotency support using `client_msg_id` and a Redis-based cache.
- Optimized WebSocket handling with connection limits (total and per-user) and improved error logging.
- **Code Refactoring**:
- Refactored `Router` to use a `RouterDeps` struct, simplifying the constructor and improving maintainability.
- Unified model ID generation logic using new `id_helper.go` (supporting UUID and Snowflake).
- Standardized JSON serialization/deserialization in models using `json_helper.go`.
- Refactored DTO conversion logic, specifically for `UserResponse` (using functional options) and `Report` responses.
- Removed redundant/deprecated DTOs like `PostDetailResponse` and `TradeItemDetailResponse`.
- **Cache Improvements**:
- Enhanced `LayeredCache` with `SetRaw` to avoid double-encoding when promoting values from Redis to local cache.
- Added `DeleteBatch` support to the cache interface.
- **Other Changes**:
- Cleaned up `config.go` by removing redundant default values and explicit environment variable overrides.
- Improved WebSocket registration flow to handle connection limits gracefully.
Add comprehensive security improvements across the application:
- **IP-based login protection**: Implement IP ban system tracking login failures, auto-banning after threshold exceeded
- **Ownership verification**: Add userID parameter to Delete/Update operations for posts and comments to prevent unauthorized modifications
- **SSRF protection**: Add URL and resolved host validation for image URLs in chat and OpenAI client
- **SQL injection prevention**: Add EscapeLikeWildcard utility to escape special characters in LIKE queries
- **HTTP security**: Configure server timeouts and add security headers middleware
- **Rate limiting**: Refactor to support configurable duration and per-endpoint rate limits for auth routes
- **Error handling**: Standardize error responses using HandleError and proper error types
- Remove BOM from all Go source files
- Standardize import ordering and whitespace across handlers and services
- Replace PostgreSQL full-text search with ILIKE pattern matching in post and user repositories
- Enhance JPush client with TLS transport, rate limit monitoring, and simplified API
- Refactor PushService to include userID in device management methods
- Add MaxDevicesPerUser limit and extract helper functions for push payload construction
Add JPush (极光推送) integration to enable push notifications for offline users. This includes:
- New jpush client package with API for batch push notifications
- Configuration for jpush (enabled, app_key, master_secret, production mode)
- Updated push service to send messages via JPush when users are offline
- Added PushChatMessage method with do-not-disturb checking
- Integrated push service with chat service to notify offline users of new messages
- Updated deployment workflow with JPush and related environment variables
Add complete trade functionality including:
- TradeItem, TradeImage, TradeFavorite models with auto-migration
- TradeRepository with CRUD and favorite operations
- TradeService with business logic for listing, creating, updating, status management
- TradeHandler with RESTful endpoints for trade operations
- DTOs and converters for request/response handling
Routes include: list, get by id, create, update, delete, status update, view recording, favorite/unfavorite.
Add channel-specific hot post filtering and ranking across the posts
listing flow. Introduce refreshChannelRanks to compute per-channel hot
scores, new Redis cache keys for channel hot ranks, and update
repository/service/handler layers to accept channelID filtering
parameters. Update HotRankWorker dependency injection to include
ChannelRepository.
Add support for referencing other posts within messages through a new post_ref segment type. Includes new PostReference model to track relationships, PostRefService for processing references, and new endpoints for post suggestions, related posts, and reference click tracking.
- Rename Muted field to NotificationMuted across model, DTOs and converters
- Add UpdateNotificationMuted repository method with upsert support
- Add SetConversationNotificationMuted service method
- Add PUT /:id/notification_muted route for setting mute status
- Improve S3 storage resilience by skipping bucket check on access denied
Update conversation list queries to distinguish between private and group chats:
private conversations are only shown when they have messages (last_seq > 0), while group conversations are always displayed even without messages. This applies to both the paginated GetConversations method and the cursor-based GetConversationsByCursor method.
Add functionality to filter out posts from blocked users in post listing endpoints. Includes a new `IsBlockedBatch` method for efficient batch checking of blocked relationships and `is_blocked` field in API responses when viewing a blocked user's posts.
Add rich message segment support enabling structured content (text, images, votes, mentions) in posts and comments. Includes segments field in models, DTOs, handlers, and services. Also adds @mention notification processing and migrates vote data to embedded segments.
The Go module name has been changed from `carrot_bbs` to `with_you`, which requires
all import paths to be updated throughout the codebase and by external consumers.
BREAKING CHANGE: Module name changed from carrot_bbs to with_you. All imports
and dependencies must be updated from carrot_bbs/* to with_you/*.
Add admin comment moderation endpoints (single and batch) and introduce a three-tier moderation system (pass/review/block) for content flagged by AI. Content with unclear violations is now held for manual review instead of being auto-rejected. Deleted the legacy audit_service.go in favor of the unified hook-based moderation system.
Add PreviewURL and PreviewURLLarge fields to PostImage model and responses. Image data now supports format "url|preview_url|preview_url_large" for storing multiple image variants. Also change comment cursor pagination to sort by created_at ASC (oldest first) to display earliest comments as first floor, and make AdminReviewVerificationRequest.Approve field optional.
Add privacy settings allowing users to control visibility of their followers, following, posts, and favorites lists with three visibility levels: everyone, following, and self. Also implement account deletion workflow with 30-day cooldown period.
- Add PrivacySettings model with visibility levels
- Add DTOs for privacy settings and account deletion
- Add repository methods for privacy settings and deletion
- Add service methods to check visibility permissions
- Add handlers for get/update privacy settings and account deletion
- Add new routes for privacy settings and account management
- Add account cleanup service for hard-deleting expired accounts
- Modify login to cancel deletion when user logs in
BREAKING CHANGE: Users with pending_deletion status can no longer log in; logging in now automatically cancels deletion requests.
Implement sensitive word filtering feature with configurable database and Redis
support. Add security enhancements including WebSocket origin validation, improved
password strength requirements, timing attack prevention, and production JWT secret
validation. Add batch operation validation limits and optimize user blocking
queries with subqueries.
BREAKING CHANGE: Password validation now requires minimum 8 characters with uppercase,
lowercase, and digit (was 6-50 characters without complexity requirements)
Implement a complete user identity verification system with support for multiple identity types (student, teacher, staff, alumni). The system includes user-facing verification submission and status checking endpoints, admin endpoints for reviewing verification requests, middleware for protecting verification-required routes, and integration with the user model to track verification status.
- Replace `interface{}` with `any` type alias across all packages
- Use built-in `min()`/`max()` for parameter clamping
- Use `slices.SortFunc`, `slices.Min`, `slices.Max` for cleaner code
- Use `strings.Cut()` for simpler string parsing in auth middleware
- Use `errors.Is()` for proper error comparison in handlers
- Update dependencies: golang.org/x/image 0.37.0 -> 0.38.0
- Add Wire code generation guidelines to ARCHITECTURE.md
- Disable Go cache in CI build workflow
- Updated CallService to handle user connection and disconnection events, allowing for automatic management of active calls based on user status.
- Introduced methods to retrieve active calls for users and send pending call invites when users reconnect.
- Modified the Invite method to return the callee's online status, improving user experience during call invitations.
- Enhanced WSHandler to support connection and disconnection event handlers, ensuring accurate call state management.
- Added CallHandler and related services for managing call sessions and participants.
- Enhanced WSHandler to support call signaling, including invite, answer, reject, and end functionalities.
- Updated router to include call-related routes for history and ICE server retrieval.
- Introduced WebRTC configuration in the application settings for call management.
- Refactored wire generation to include CallService and CallRepository for improved dependency injection.
- Refactored repository structures to use interfaces instead of concrete types, enhancing flexibility and testability.
- Updated various repository methods to accept interfaces, allowing for better dependency management.
- Modified wire generation to accommodate new repository interfaces, ensuring proper service injection throughout the application.
- Enhanced handler methods to utilize DTOs for filtering and data handling, improving code clarity and maintainability.
- Updated Like and Unlike methods in CommentRepository to use transactions for data consistency.
- Introduced IsLikedBatch method for batch checking if comments are liked, addressing N+1 query issues.
- Enhanced BatchDelete method in PostRepository to perform batch deletions within a single transaction, improving efficiency.
- Added BatchUpdateStatus method for posts to utilize a single SQL update for status changes, reducing database load.
- Implemented BatchUpdateSortOrder in StickerRepository using CASE WHEN for efficient sorting updates.
- Added IsFollowingBatch and IsBlockedBatch methods in UserRepository for batch checking follow and block relationships, optimizing performance.
- Introduced MaterialHandler for managing learning materials and subjects.
- Updated router to include routes for public and admin material management.
- Enhanced database initialization with material subject and file models.
- Implemented seeding logic for material subjects and files.
- Updated wire generation to include MaterialHandler and related services.
- Updated ChannelService to include caching for channel lists, improving performance and reducing database load.
- Introduced cache invalidation methods to ensure channel list consistency after modifications.
- Modified ChannelRepository to remove unused ListByIDs method, streamlining the repository interface.
- Updated wire generation to inject cache into ChannelService for enhanced functionality.
- Added new cache key constants for channel-related data management.
- Updated PostHandler to include channel information in post responses.
- Introduced PostChannelBrief DTO to represent channel details associated with posts.
- Modified post conversion functions to support channel data, ensuring proper mapping of channel IDs to channel names.
- Enhanced ChannelRepository and ChannelService to facilitate batch retrieval of channels by IDs.
- Updated wire generation to inject channel service into post handler for improved functionality.
- Added Channel support in Post model, including ChannelID field in PostRequest and PostResponse DTOs.
- Updated PostService and PostRepository to handle channel-specific logic for creating and listing posts.
- Introduced ChannelRepository and ChannelService, along with corresponding handlers for managing channels.
- Enhanced router to include routes for channel management and public channel listing.
- Seeded default channels in the database during initialization.
- Added RecordShare method in PostHandler to handle sharing posts and increment share count.
- Introduced IncrementShares method in PostRepository to update shares_count for published posts.
- Updated PostService to include RecordShare, ensuring cache invalidation for post details and lists.
- Integrated new share route in the router for handling share requests.
- Removed Gorse-related configurations, handlers, and dependencies from the codebase.
- Introduced HotRank feature with configuration options for ranking posts based on recent activity.
- Updated application structure to support HotRank processing, including new caching mechanisms and database interactions.
- Cleaned up related DTOs and repository methods to reflect the removal of Gorse and the addition of HotRank functionality.
- Added NODE_NO_WARNINGS environment variable to build and deploy workflows.
- Upgraded actions/upload-artifact and actions/download-artifact to version 4 in the build workflow.
- Introduced ContentEditedAt field in PostResponse and PostDetailResponse DTOs to track content modification timestamps.
- Updated post conversion functions to include ContentEditedAt in responses.
- Ensured ContentEditedAt is set during post updates in the repository.
- Changed JSON field names from "items" to "list" in various cursor pagination response DTOs for consistency.
- Updated moderation status update methods in CommentRepository and PostRepository to use UpdateColumn for better clarity and to ensure that moderation actions do not affect content update timestamps.
- Add new GetByReceiverIDCursor method with context support and generic cursor types
- Rename old implementation to GetByReceiverIDCursorLegacy for backward compatibility
- Update handler to use legacy interface temporarily during migration
- Use CursorBuilder pattern for consistent pagination across services
Add support for different post feed types via tab parameter:
- follow: posts from followed users (requires authentication)
- hot: trending posts using Gorse most_liked_weekly or database fallback
- recommend: personalized recommendations with random offset for diversity
- latest: default chronological feed
The cursor pagination detection is also improved to check for parameter
existence rather than non-empty string value.
Implement cursor-based pagination across multiple API endpoints to improve performance and enable efficient infinite scrolling. This replaces traditional offset-based pagination for high-volume data retrieval.
Changes:
- Add cursor pagination DTOs for all entity types (Post, Comment, Message, Conversation, Group, Notification, GroupMember, GroupAnnouncement)
- Implement cursor pagination methods in repositories with support for various sort orders (created_at, updated_at, join_time, seq)
- Add cursor pagination handlers that maintain backward compatibility with existing offset pagination
- Add new API routes for cursor-based endpoints (/cursor suffix)
- Add helper converter functions for pointer slice types in DTO conversions
- Migrate all log.Printf/log.Println calls to structured zap logging
- Use cmp.Or for cleaner default value handling
- Replace manual loops with slices.ContainsFunc/IndexFunc
- Add batch query methods (IsLikedBatch, IsFavoritedBatch) to solve N+1 problem
- Update JSON tags from omitempty to omitzero for numeric fields
- Use errgroup-style wg.Go for worker goroutines
- Remove duplicate casbin/v2 dependency (keeping v3)
- Add plans/ to gitignore
- Add admin handlers and services for users, posts, comments, groups, and dashboard
- Implement role permission management with ability to view and update permissions
- Add database seeding for roles and Casbin permission policies
- Upgrade Casbin from v2 to v3 with GORM adapter for persistent policy storage
- Add admin-specific repository methods with filtering and pagination
- Register new admin API routes under /api/v1/admin/*
Integrate Casbin for role-based access control with admin routes for role management. Add user activity logging service to track login and refresh events. Refactor audit service to use AI-based content moderation.
Add gRPC server support with configurable TLS, environment-based settings, and Wire injection. Implement schedule synchronization service with task runner integration for external course data fetching.
- Add gRPC configuration with env var overrides (APP_GRPC_ENABLED, APP_GRPC_PORT, etc.)
- Create gRPC server infrastructure with runner task management
- Implement ScheduleSyncService for course data synchronization
- Add sync endpoint to schedule handler for external system integration
- Update repository with context-aware batch delete operation
- Wire all dependencies including zap logger for structured logging
- Add Google Wire for compile-time dependency injection
- Replace concrete service types with interfaces across handlers
- Remove global state (DB, cache) in favor of constructor injection
- Split monolithic files into focused modules:
- config: separate files for each config domain
- dto: converters split by domain (user, post, message, group)
- cache: separate metrics.go and redis_cache.go
- Introduce unified apperrors package with string-based error codes
- Add transaction support with context-aware repository methods
- Upgrade golang.org/x/crypto from 0.17.0 to 0.26.0
- Simplify InviteMembers to always send invite card directly to invitee
- Modify RespondInvite to check group join type (anyone vs need approval) before adding member
- Update SetGroupAddRequest to properly handle invite flow with invitee agreement and admin approval stages
- Add case-insensitive search using LOWER() in user repository
- Add random offset (0-50) for first page of recommended posts
- Update dependencies: remove gorilla/websocket, promote gomail to direct dependency
