Our research primarily focuses on Multimodal Large Language Model (MLLM), Embodied AI, and MLLM-based Agent, with an emphasis on perception, reasoning, and decision-making in interactive environments:

• Multimodal Large Language Models (MLLMs) aim to build and train models capable of understanding, reasoning, and generating content across multiple modalities, including text, images, audio, and video. Our research in this area explores the enhancement of multitask learning capabilities, the advancement of high-resolution perception, and the design of unified architectures for multimodal understanding and generation. Please refer to our GitHub Orgnization about JiuTian MLLM ("九天"多模态大模型) for more details. [bilibili]
• Embodied AI studies agents capable of perceiving, reasoning, and acting within physical environments. We aim to build systems that integrate multimodal perception, instructon comprehension, and continuous action planning to perform complex 3D tasks such as navigation, manipulation, and interactive behaviors. [bilibili]

  Task instruction: Open the drawer, put the toy inside, and then close it.

• MLLM-based Agent focuses on sequential decision-making across various complex environments such as MineCraft, mobile device. Our works encompasses a wide range of directions, including framework-based agent, native agent, RL-based reasoning enhancement. [bilibili]

  Task instruction: Search today's weather in Shenzhen on Chrome, then write the temperature into "today.md" using Markor.