H Company Releases NeoMME: A Family of 260M and 800M Single-Tower Multimodal Encoders That Drop the Vision Tower and Causal Decoder
2026-09-07 05:06Models🔥 42.2 heat score
1sources
1days unfolding
42.2heat score
5mentions
SummaryAI generated
On September 6, 2026, H Company released a new multi-modal encoder series called NeoMME. This series includes two single-tower model architectures with parameter sizes of 260 million and 800 million respectively, and its core design eliminates traditional visual towers and causal decoder components. NeoMME uses a single Transformer layer to process multi-language texts and raw image patches simultaneously, and it is pre-trained based on discrete mask diffusion technology. In the retrieval fine-tuning task, the NeoMME-Retriever version achieved 0.523 nDCG@10 (260M version) and 0.556 nDCG@10 (800M version) scores in the ViDoRe v3 benchmark. The model supports a context length of 16,384 tokens, is open-sourced under the Apache 2.0 license, and offers zero-day support in the Hugging Face Transformers library. In terms of performance, the 260M version…
Company H released NeoMME, a family of single-tower multi-modal encoders consisting of 260M and 800M parameters, with the visual tower and causal decoder removed. This model processes multi-language texts and raw image patches through the same Transformer layer, uses discrete mask diffusion pre-training, and the retrieval fine-tuning version of NeoMME-Retriever achieved 0.523 nDCG@10 (for the 260M version) and 0.556 (for the 800M version) on ViDoRe v3. NeoMME supports 16,384 token contexts, is open-source under Apache 2.0, and receives zero-day support from Hugging Face Transformers. On ViDoRe v3, the performance of the 260M model is close to that of the ColQwen2.5-v0.2 with 37.5B parameters; in terms of storage, hierarchical tokenization and pooling can reduce the size of each page index from 1.5 MB to 39.0 kB.