🍬SEED-Story: Multimodal generation of illustrated stories.

🍬SEED-Story: Multimodal generation of illustrated stories.
🚩 Hashtags: #Multimedia #Software #3D

💡 SEED-Story is a tool based on the SEED-X project (https://github.com/AILab-CVC/SEED-X) for creating a story from images and texts in a predefined style. Such a story can contain up to 25 episodes.
The methodology used in SEED-Story is based on a combination of pre-trained models that help to create realistic images and text, special adaptation to ensure consistency of characters and styles in the images and a multimodal attention mechanism to preserve important details of text and images.
SEED-Story was trained on a special detailed StoryStream dataset, which is based on several animated series.
👉 SEED-Story will require 40GB VRAM (with self-training capability included) m a set of additional models:
🛠SDXL Base 1.0 (https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
🛠QwenVL-Chat (https://huggingface.co/Qwen/Qwen-VL-Chat)

⚠️ GradioUI to SEED-Story is in development, and according to the developers will be unveiled soon (no timeline is given).

🛠 Model on HF (https://huggingface.co/TencentARC/SEED-Story)
🛠 Demo Video (https://www.youtube.com/watch?v=_t87U1tLiyQ)
🛠 Arxiv (https://arxiv.org/pdf/2407.08683)
🛠 Github (https://github.com/TencentARC/SEED-Story) [Stars: 382 | Issues: 9 | Forks: 24]

🔥Go to our website, https://brainshacking.com.
‼️ Subscribe to the main channel, https://t.me/BrainsHacking to participate in prize drawings and be the first to know about important updates.