Vision language model optimization

CMU-Q Point of Contact: Mohammad Salameh

AI Summary :

This research aims to address computational constraints and cultural misalignment by compressing Vision-Language and Diffusion Models for edge devices and infusing them with Arab-centric knowledge.

Vision-Language Models (VLMs) and Diffusion Models (e.g., Stable Diffusion) have revolutionized multimodal AI(image, vido, text, audio, …). However, two critical barriers prevent their effective deployment in the Arab region: 1. Computational Constraints: These models are massive, requiring high-end GPUs that are often unavailable for local businesses, researchers, or mobile users in diverse Arab markets. 2. Cultural Misalignment: Existing models are largely "Western-centric," often failing to recognize Arab architectural styles, climate, or traditional attire Research Objectives ● Compression: Optimize models specifically for edge devices though reducing the number of parameters and inference latency of VLMs and Diffusion models using approaches such as Structured Pruning and Neural Architecture Search (NAS), Knowledge Distillation. ● Cultural Alignment: Infuse models with "Arab-centric" knowledge Important: If you anticipate any changes to your general research scope in the future, please notify the Research Office in advance so that the revisions can be reviewed and confirmed for compliance with institutional research policies.

Project:

START-UP-SALAMEH

Start year:

2026

Sponsor:

Qatar Foundation

Status:

Active