TailorMind: Towards Preference-Aligned Multimodal Content Generation
Abstract Overview
This paper studies personalized multimodal content generation, aiming to create user-tailored image-text posts and videos when suitable user-generated content is unavailable. The proposed TailorMind framework connects behavior-based preference modeling with controllable generation through hypergraph collaborative filtering, LLM-based multimodal profile construction, and iterative textual-gradient refinement using ranking feedback from held-out interactions. For generation, it uses retrieval-augmented style control to ground outputs in authentic UGC patterns and a cross-modal cohesion reflection mechanism to reduce drift between textual and visual components. The authors also introduce TailorBench, a benchmark built from Rednote, Bilibili, and Hupu with evaluation over coherence, novelty, aesthetic quality, hallucination, and profiling.
Novelty
The distinctive aspect of the work is the explicit linkage between collaborative preference modeling and multimodal generation, rather than treating recommendation and generation as separate problems. It also introduces textual-gradient profile refinement driven by ranking errors and combines retrieval-based style grounding with cross-modal reflection for preference-aligned content synthesis.
Results
Across Rednote, Hupu, and Bilibili, TailorMind achieves competitive or stronger coherence than the compared generation baselines while improving novelty and aesthetic scores. In reranking-based profiling evaluation, it posts the best Recall results across datasets, with the abstract highlighting up to 29% Recall gains. Ablation and human evaluation further support the contributions of the retrieval grounding and cohesion modules, while reported hallucination rates remain low or zero in several settings.
Key Points
- TailorMind derives generation-ready user profiles from sparse multimodal histories by combining hypergraph collaborative filtering with iterative textual profile optimization.
- The generation stage is controlled by retrieved user-aligned exemplars and a reflection mechanism that checks text-visual consistency and relevance to user history.
- The paper introduces TailorBench and shows that TailorMind improves user alignment, novelty, and aesthetic quality across multiple platforms and content modalities.