SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languages

Distill compact multilingual vision-text embeddings from large multimodal teachers for real-world deployment.

Mentors:
Peerat Limkonchotiwat Ekapol Chuangsuwanich Pume Tuchinda
Mentees (4):
Ashvanth SFaiz Assabil FirdausIlma Aliya FiddienPuja Ahmad Habibi

Project Proposal

We propose a training framework to distill a small vision-text embedding model from a large multimodal teacher. Existing KD approaches often assume a base-sized teacher and focus on monolingual settings, leaving large teachers and multilingual scenarios underexplored.

This project will design a KD framework for large-scale teacher models and multilingual vision-text models. The resulting model should be compact and efficient for real-world scenarios and edge devices.

Relevant publications:

  • MIEB: Massive Image Embedding Benchmark

Publications

  1. SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languages
    Puja Ahmad Habibi, Faiz Assabil Firdaus, Ashvanth S, Ekapol Chuangsuwanich, Pume Tuchinda, Peerat Limkonchotiwat
    ACCV • 2026
    Abstract

    Multilingual text-vision embedding models are essential for cross-lingual image-text retrieval, but Southeast Asian languages remain poorly supported due to the region’s linguistic diversity and limited data and computing resources. In this paper, we introduce SEA-CLIP-Tiny, a compact multilingual text-vision embedding model for Southeast Asia with fewer than 50M parameters. Our model adapts a CLIP-KD-style framework to Southeast Asian multilingual settings through regional data curation and multilingual teacher guidance. Experiments across seven Southeast Asian languages show that SEA-CLIP-Tiny achieves the strongest average retrieval performance among the evaluated student models, reaching 12.9%, 31.5%, and 42.2% at R@1, R@5, and R@10, respectively. Compared with MobileCLIP2, it improves average R@10 by 12.1 points while using 38.4% fewer parameters and lower measured CPU latency. These results highlight the importance of region-aware training for efficient multilingual text-vision models in Southeast Asia.

    PDF Code Resources