Publications

2024

1. NAACL

   KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learning

   Cong-Duy Nguyen, Thong Nguyen, Xiaobao Wu, and 1 more author

   Proceedings of NAACL, 2024

   Abs arXiv Code

2. AAAI

   READ-PVLA: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling

   Thong Nguyen, Xiaobao Wu, Xinshuai Dong, and 5 more authors

   Proceedings of AAAI, 2024

   Abs arXiv

   Fully fine-tuning pretrained large-scale transformer models has become a popular paradigm for video-language modeling tasks, such as temporal language grounding and video-language summarization. With a growing number of tasks and limited training data, such full fine-tuning approach leads to costly model storage and unstable training. To overcome these shortcomings, we introduce lightweight adapters to the pre-trained model and only update them at fine-tuning time. However, existing adapters fail to capture intrinsic temporal relations among video frames or textual words. Moreover, they neglect the preservation of critical task-related information that flows from the raw video-language input into the adapter’s low-dimensional space. To address these issues, we first propose a novel REcurrent ADapter (READ) that employs recurrent computation to enable temporal modeling capability. Second, we propose Partial Video-Language Alignment (PVLA) objective via the use of partial optimal transport to maintain task-related information flowing into our READ modules. We validate our READ-PVLA framework through extensive experiments where READ-PVLA significantly outperforms all existing fine-tuning strategies on multiple low-resource temporal language grounding and video-language summarization benchmarks.

3. AAAI

   On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling

   Xiaobao Wu, Fengjun Pan, Thong Nguyen, and 4 more authors

   Proceedings of AAAI, 2024

   Abs arXiv

4. ESWA

   Supervised learning models for social bot detection: Literature review and benchmark

   Hoang-Dung Nguyen, Duc Q. Nguyen, Cong-Duy Nguyen, and 8 more authors

   Expert Systems with Applications, 2024

   Abs

5. ICLR

   Topic Modeling as Multi-Objective Contrastive Optimization

   Thong Nguyen, Xiaobao Wu, Xinshuai Dong, and 3 more authors

   Proceedings of ICLR, 2024

   Abs

2023

1. EMNLP

   Improving multimodal sentiment analysis: Supervised angular margin-based contrastive learning for enhanced fusion representation

   Cong-Duy Nguyen, Thong Nguyen, Anh Vu, and 1 more author

   Proceedings of EMNLP (Findings), 2023

   Abs arXiv

2. ACMMM

   Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment

   Cong-Duy Nguyen, The-Anh Vu-Le, Thong Nguyen, and 2 more authors

   Proceedings of ACM MM, 2023

   Abs arXiv

3. EMNLP

   DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Grounding

   Thong Nguyen, Xiaobao Wu, Xinshuai Dong, and 3 more authors

   Proceedings of EMNLP (Findings), 2023

   Abs arXiv

   Temporal Language Grounding seeks to localize video moments that semantically correspond to a natural language query. Recent advances employ the attention mechanism to learn the relations between video moments and the text query. However, naive attention might not be able to appropriately capture such relations, resulting in ineffective distributions where target video moments are difficult to separate from the remaining ones. To resolve the issue, we propose an energy-based model framework to explicitly learn moment-query distributions. Moreover, we propose DemaFormer, a novel Transformer-based architecture that utilizes exponential moving average with a learnable damping factor to effectively encode moment-query inputs. Comprehensive experiments on four public temporal language grounding datasets showcase the superiority of our methods over the state-of-the-art baselines.

4. ACL

   Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction

   Thong Nguyen, Xiaobao Wu, Xinshuai Dong, and 4 more authors

   Proceedings of ACL (Findings), 2023

   Abs arXiv Code

   Multimodal Review Helpfulness Prediction (MRHP) aims to rank product reviews based on predicted helpfulness scores and has been widely applied in e-commerce via presenting customers with useful reviews. Previous studies commonly employ fully-connected neural networks (FCNNs) as the final score predictor and pairwise loss as the training objective. However, FCNNs have been shown to perform inefficient splitting for review features, making the model difficult to clearly differentiate helpful from unhelpful reviews. Furthermore, pairwise objective, which works on review pairs, may not completely capture the MRHP goal to produce the ranking for the entire review list, and possibly induces low generalization during testing. To address these issues, we propose a listwise attention network that clearly captures the MRHP ranking context and a listwise optimization objective that enhances model generalization. We further propose gradient-boosted decision tree as the score predictor to efficaciously partition product reviews’ representations. Extensive experiments demonstrate that our method achieves state-of-the-art results and polished generalization performance on two large-scale MRHP benchmark datasets.

2022

1. EMNLP

   Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictions

   Thong Nguyen, Xiaobao Wu, Tuan Luu, and 3 more authors

   Proceedings of EMNLP, 2022

   Abs arXiv Code

   Modern Review Helpfulness Prediction systems are dependent upon multiple modalities, typically texts and images. Unfortunately, those contemporary approaches pay scarce attention to polish representations of cross-modal relations and tend to suffer from inferior optimization. This might cause harm to model’s predictions in numerous cases. To overcome the aforementioned issues, we propose Multimodal Contrastive Learning for Multimodal Review Helpfulness Prediction (MRHP) problem, concentrating on mutual information between input modalities to explicitly elaborate cross-modal relations. In addition, we introduce Adaptive Weighting scheme for our contrastive learning approach in order to increase flexibility in optimization. Lastly, we propose Multimodal Interaction module to address the unalignment nature of multimodal data, thereby assisting the model in producing more reasonable multimodal representations. Experimental results show that our method outperforms prior baselines and achieves state-of-the-art results on two publicly available benchmark datasets for MRHP problem.

2. InterSpeech

   A Vietnamese-English Neural Machine Translation System

   Tuan-Duy H. Nguyen, Duy Phung, Cong-Duy Nguyen, and 6 more authors

   Proceedings of InterSpeech Show & Tell Demonstrations, 2022

   Abs Code