README.md
9.0 KB · 173 lines · markdown Raw
1 ---
2 license: apache-2.0
3 language:
4 - en
5 - zh
6 pipeline_tag: image-to-video
7 tags:
8 - video-generation
9 - text-to-video
10 - image-to-video
11 - video-to-video
12 - reference-to-video
13 - minimax-h3
14 - comfyui
15 - fine-tuned
16 - hdr
17 - singularity
18 - int8
19 base_model:
20 - MiniMaxAI/MiniMax-H3
21 ---
22
23 # 🚀 V2 INT8 — Latest Update
24
25 **Minimax-h3_Singularity V2 INT8** builds on V1 with expanded action coverage, stronger prompt adherence, a fix for audio reference conditioning, and enhanced visual style rendering.
26
27 ## ✨ What's New in V2
28
29 * ⚔️ **Expanded Action Coverage**: Further optimized on top of V1 to support a wider range of action scenes and more varied motion sequences, extending the model's versatility across dynamic video generation scenarios.
30 * 📝 **Stronger Prompt Adherence**: V2 follows prompt instructions more closely, improving control over subjects, actions, environments, and visual styles. We recommend keeping prompts within approximately **5,000 characters**, using clear, focused descriptions and avoiding excessive or repetitive instructions.
31 * 🎵 **Audio Reference Fix**: Fixed the audio reference conditioning issue. Audio references can now be used to help adjust the character's color tone.
32 * 🎨 **Enhanced Visual Styles**: Strengthened visual style rendering, with strong results across **photorealistic, East Asian realism, 3D CG, and cinematic CG** scenes. For best results, explicitly include the desired style keywords in your prompt.
33 * 📦 **INT8 Release**: The updated V2 model is available in **INT8 format**.
34
35 ### 💡 Recommended Prompting for V2
36
37 Clearly describe the **subject, action, environment, camera movement, and intended visual style**. Keep instructions consistent and prioritize the details that matter most.
38
39 Include style keywords that match your desired result. Examples:
40
41 | Visual Style | Suggested Prompt Keywords |
42 | --- | --- |
43 | Photorealistic | `photorealistic, natural lighting, realistic textures` |
44 | East Asian Realism | `East Asian realism, realistic East Asian aesthetics, natural lighting` |
45 | 3D CG | `3D CG, detailed materials, stylized 3D rendering` |
46 | Cinematic CG | `cinematic CG, cinematic lighting, film-quality rendering` |
47
48 **Recommended prompt length: within approximately 5,000 characters.** This is a practical recommendation, not a target to fill. Shorter, well-structured prompts are welcome; avoid unnecessary details, repetition, and conflicting style instructions.
49
50 ---
51
52 # Minimax-h3_Singularity
53
54 <p align="center">
55 <a href="https://www.runninghub.ai/zh-cn/post/2112912294795993090/?inviteCode=rh-v1559">
56 <img src="https://img.shields.io/badge/🚀_Online_Demo-RunningHub-blue.svg" alt="Online Demo">
57 </a>
58 <a href="https://youtube.com/@AIGC-Singularity">
59 <img src="https://img.shields.io/badge/YouTube-AIGC--Singularity-red.svg" alt="YouTube">
60 </a>
61 <a href="https://space.bilibili.com">
62 <img src="https://img.shields.io/badge/Bilibili-AIGC--Singularity_Space-00a1d6.svg" alt="Bilibili">
63 </a>
64 </p>
65
66 ## 📖 Model Overview
67
68 **Minimax-h3_Singularity** is a comprehensive fine-tuned fusion model designed to enhance the capabilities of **MiniMax-H3**. As a versatile **multimodal video generation model**, it supports **Text-to-Video (T2V)**, **Image-to-Video (I2V)**, **Reference-to-Video (Ref2V)**, and **Video-to-Video (V2V)** workflows within **ComfyUI**.
69
70 Built upon a strategic fusion of key checkpoints, including `ref`, `fl`, and `b25-49`, the model underwent extensive high-step fine-tuning. To preserve the original model's foundational strengths and broad generalization while addressing artifacts introduced during training, we spent **3 full days on precise model pruning and weight optimization**. The result is a clean, sharp, and highly dynamic video generation model.
71
72 The latest **V2 INT8** release builds on this foundation with broader action coverage, improved prompt adherence, corrected audio reference conditioning, and stronger visual style rendering.
73
74 ---
75
76 ## ✨ Key Improvements & Features
77
78 * 🎬 **HDR Image Quality & Blur Reduction**: Fine-tuned on high-dynamic-range (HDR) video datasets to enhance visual clarity and reduce unwanted motion blur during high-speed action.
79 * 👤 **Distant Face Restoration**: Reduces facial distortion, blurriness, and structural collapse in medium and long shots.
80 * 🎨 **Clean & Natural Aesthetic**: Reduces excessive skin shine and unnatural glossy textures for more natural lighting and realistic materials.
81 * ⚔️ **Enhanced Dynamic Motion**: Improves motion fluidity and physical impact, including complex sequences such as **sword fighting, martial arts, and melee combat**. V2 expands support to a wider range of action scenarios.
82 * 🌌 **VFX & Fantasy Effects**: Optimized for fantasy spellcasting, particle auras, and magical combat visual effects.
83 * 🎭 **Expressive Facial Dynamics**: Captures subtle facial expressions and emotional nuances more vividly.
84 * 📹 **Cinematography & Camera Control**: Improves responsiveness to camera instructions, including pans, tilts, zooms, and tracking shots.
85 * 📝 **Improved Prompt Control in V2**: Stronger adherence to prompt instructions enables more precise control over scene content and presentation.
86 * 🎵 **Corrected Audio Reference Conditioning in V2**: Resolves the audio reference issue and supports using audio references to help adjust the character's color tone.
87 * 🖼️ **Stronger Style Rendering in V2**: Delivers improved results in **photorealistic, East Asian realism, 3D CG, and cinematic CG** styles when the intended aesthetic is specified in the prompt.
88 * 🛡️ **Preserved Base Capabilities**: Designed to retain MiniMax-H3's core multimodal generation capabilities, style adaptability, and broad generalization.
89
90 ---
91
92 ## 🎬 Showcase
93
94 <video src="https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/resolve/main/video/1.mp4" controls autoplay loop muted width="20%"></video>
95 <video src="https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/resolve/main/video/3.mp4" controls autoplay loop muted width="20%"></video>
96 <video src="https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/resolve/main/video/AIGCTYD2_00001_p84-audio_ganvc_1788634594%20(1).mp4" controls autoplay loop muted width="20%"></video>
97 <video src="https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/resolve/main/video/AIGCTYD2_00010_p87-audio_alipa_1788652671.mp4" controls autoplay loop muted width="20%"></video>
98
99 ---
100
101 ## 💡 Usage Guide
102
103 ### Multimodal Pipeline Support
104
105 The model supports the following workflows within **ComfyUI**:
106
107 * **Text-to-Video (T2V)**
108 * **Image-to-Video (I2V)**
109 * **Reference-to-Video (Ref2V)**
110 * **Video-to-Video (V2V)**
111
112 ### 📝 Prompt Recommendations for V2
113
114 For best results:
115
116 * Keep prompts within approximately **5,000 characters**.
117 * Use clear descriptions of the subject, action, setting, lighting, and camera movement.
118 * Explicitly include the intended visual style, such as `photorealistic`, `East Asian realism`, `3D CG`, or `cinematic CG`.
119 * Avoid repetitive descriptions, unnecessary instructions, and conflicting visual styles.
120 * For action scenes, describe the movement and sequence of events clearly.
121
122 ### 🎨 Example Style Prompts
123
124 **Photorealistic**
125
126 > Photorealistic style, natural lighting, realistic skin texture, detailed environmental materials, fluid character movement, cinematic camera tracking.
127
128 **East Asian Realism**
129
130 > East Asian realism, a traditional courtyard, realistic fabric textures, soft natural lighting, detailed architecture, fluid martial arts movements, cinematic composition.
131
132 **3D CG**
133
134 > 3D CG style, detailed character materials, expressive facial animation, fluid action, depth and dimensionality, polished 3D rendering.
135
136 **Cinematic CG**
137
138 > Cinematic CG style, dramatic cinematic lighting, detailed environments, dynamic action choreography, atmospheric effects, film-quality rendering.
139
140 ### 🎵 Audio Reference
141
142 V2 fixes the audio reference conditioning issue. You can use audio references to help adjust the character's color tone as part of your generation workflow.
143
144 ### 🚀 Recommended Acceleration LoRA
145
146 For high-speed generation, the recommended acceleration LoRA is:
147
148 * **`minimax_h3_ref2v_turbo_4step_v0.1`** — enables 4-step fast inference.
149
150 ### 🌐 Online Interactive Demo
151
152 Test the model directly in your browser without a local GPU setup:
153
154 👉 [**Try it on RunningHub Workflows**](https://www.runninghub.ai/post/2096339589492432897/?inviteCode=rh-v1559)
155
156 ---
157
158 ## 🙏 Acknowledgements
159
160 Special thanks to the **MiniMax** open-source team for creating and releasing the powerful **MiniMax-H3** multimodal video model and providing a solid foundation for the open-source community! 🤝
161
162 ---
163
164 ## 🤝 Community & Commercial Inquiries
165
166 Connect with us for tutorials, community discussions, workflow sharing, or commercial collaborations:
167
168 * **YouTube Channel**: [AIGC-Singularity](https://youtube.com/@AIGC-Singularity)
169 * **Bilibili Channel**: AIGC-Singularity Space
170 * **QQ Group 1**: `1058747239` (Request to join)
171 * **QQ Group 2**: `1072010342` (Request to join)
172 * **Business Inquiries (WeChat)**: `aigctyd`
173 * **Email**: `a592991299@gmail.com`