PhD Student - Domain-Specific Multimodal LLMs and AI Governance

About the role

Rakuten Asia, in partnership with the Economic Development Board (EDB) through its Industrial Postgraduate Programme (IPP), is recruiting PhD students to work on a research project developing domain-specific multimodal large language models (MLLMs) and AI governance systems. The project focuses on verticals such as e-commerce, travel marketplaces, customer service and fintech, with joint supervision of the PhD student.

Key responsibilities

• Build MLLMs that integrate domain knowledge for rapid business deployment.
• Advance AI governance through intelligent, explainable review systems.
• Optimize model architectures for multilingual, multi-task and multimodal performance in real-world e-commerce scenarios.
• Collect and curate domain-specific datasets (text, images, audio, video) and annotate them for tasks such as product classification, content moderation and IPR detection.
• Pretrain and fine-tune LLMs/MLLMs using PyTorch or TensorFlow, with prompt engineering, knowledge distillation and graph neural networks.
• Develop explainable review systems using XAI methods such as attention visualization and counterfactual reasoning.
• Apply training acceleration methods (mixed precision, distributed training) and deploy optimized models.

Required profile

• Strong academic record in machine learning, deep learning and multimodal AI.
• Expertise in computer vision and multimodal tasks (image search, classification, segmentation, detection, OCR, GNNs) or in NLP (pretraining, NLU, multilingual learning, NLG, semi-supervised learning).
• Audio signal processing expertise (speech recognition, speaker identification, audio event detection, feature extraction) is also listed.
• Experience applying LLMs to real-world problems is a plus; competition results and publications at top conferences (ACL, EMNLP, ICML, CVPR, ICCV, AAAI, ICASSP) are highly valued.

Required skills

• Hands-on experience training and deploying PyTorch models.
• Knowledge of training acceleration methods such as mixed precision and distributed training is a plus.

What we offer

• Full sponsorship for postgraduate studies.
• Hiring by Rakuten Asia upon successful completion of the programme.