Learning User Preferences from Multi-Modal Data

Given the huge number of choices that we often face when interacting with various online systems, such as e-commerce, search engines, and social media, there is a need to personalize user experience through recommending those items that user is most likely to be interested in. The essence to doing this well is to learn models for user preferences from large amounts of behavioral data that arise from users' interactions with those systems. The challenge is that these behavioral data come in diverse modalities, such as the numerical ratings users assign, the text reviews users write, the visual images that users post, as well as the network of other users that one follows. In this talk, we explore data mining and machine learning techniques for modeling user preferences from such multi-modal representations, towards building end-to-end recommendation systems.