Daily Paper Briefing

Abstract Overview

HOMIE is a human-object centric video personalization framework designed to handle both inter-subject references, where different images depict different entities, and intra-subject references, where multiple inputs describe the same subject such as OCR maps or multi-view views. The method keeps the text encoder and uses an MLLM primarily to extract relationships among prompts and references, rather than replacing the standard text-conditioning pathway. Its two main components are Global Multimodal Guidance, which injects pooled MLLM semantics into video-token self-attention, and Modality-Reference Embedding, which marks token modality and ties together references that belong to the same subject. The paper also describes a curated training pipeline built from existing subject-driven video datasets plus an additional high-quality human-object dataset.

Novelty

The paper's main novelty is a unified HOCVP framework that addresses both inter-subject and intra-subject personalization with a specific MLLM integration strategy that preserves the original text encoder. It introduces Global Multimodal Guidance for injecting MLLM-derived semantic cues into self-attention and Modality-Reference Embedding for distinguishing modalities while linking same-subject references.

Results

In quantitative comparisons, HOMIE reports state-of-the-art video quality and text-following among the evaluated open-source methods, while remaining competitive on subject-consistency metrics across diverse HOCVP settings. The reported gains include higher OCR accuracy than prior baselines and competitive results for multi-view intra-subject generation, and qualitative examples show stronger logo placement reasoning, text fidelity, and viewpoint consistency. Ablations and a 40-participant user study further support the contribution of the GMG and MRE components.

Key Points

1. HOMIE unifies inter-subject and intra-subject human-object video personalization within one framework instead of treating these settings separately.
2. The method integrates MLLM information without replacing the text encoder, using GMG to modulate video-token attention and MRE to encode modality and reference identity structure.
3. Experiments indicate strong open-source performance, especially for OCR-based text fidelity, abstract logo personalization, and multi-view consistency in generated interactions.

References

• arXiv: https://arxiv.org/abs/2607.18217v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2607.18217v1
• Hugging Face Papers: https://huggingface.co/papers/2607.18217
• Project: https://yiyangcai.github.io/homie-page.github.io/