Transcription
Fundamentals of DeepRecommender SystemsWenqi FanThe Hong Kong Polytechnic Universityhttps://wenqifan03.github.io, [email protected] website: https://deeprs-tutorial.github.ioData Science and Engineering Lab1
A General Architecture of Deep Recommender SystemPrediction layerHidden layer(e.g., MLP, CNN, RNN, etc. )Embedding layer0011Field 1User00Field mItemContext010Field MInteraction2
NeuMFNeuMF unifies the strengths of MF and MLP in modeling user-item interactions.p MF uses an inner product as the interaction functionp MLP is more sufficient to capture the complex structure of user interaction dataNeural Collaborative Filtering, WWW, 20173
Wide&DeepStandard MLPEmbeddingConcatenationp The wide linear models can memorize seen feature interactions using cross-product featuretransformations.p The deep models can generalize to previously unseen feature interactions through low- dimensionalembeddings.Wide & Deep Learning for Recommender Systems, 1st DLRS, 20164
Neural FMNeural Factorization Machines (NFMs) “deepens”FM by placing hidden layers above second-orderfeature interaction modeling.categorical variablesNeural Factorization Machines for Sparse Predictive Analytics, SIGIR, 20175
Neural FMNeural Factorization Machines (NFMs) “deepens”FM by placing hidden layers above second-orderfeature interaction modeling.“Deep layers” learn higher-order featureinteractions only, being much easier to train.Bilinear Interaction Pooling:''𝑓!" 𝑉# 𝑥 𝐯 𝑥( v(categorical variables %& (% )&Neural Factorization Machines for Sparse Predictive Analytics, SIGIR, 20176
DeepFMDeepFM ensembles FM and DNN and to low- and high-order feature interactionssimultaneously from the input raw features.FM component (low-order)Deep component (high-order)Prediction Model:DeepFM: A Factorization-Machine based Neural Network for CTR Prediction, IJCAI, 20177
DeepFMDeepFM ensembles FM and DNN and to low- and high-order feature interactionssimultaneously from the input raw features.FM component (low-order)Deep component (high-order)Share theembedding layerPrediction Model:DeepFM: A Factorization-Machine based Neural Network for CTR Prediction, IJCAI, 20178
DSCFCollaborative Filtering with users’ social relations(Social Recommendation)Dr. Anthony FauciNewsUS could see millions of coronaviruscases and 100,000 or more deathsDeep Social Collaborative Filtering, RecSys, 20199
DSCFCollaborative Filtering with users’ social relations(Social Recommendation)Users might be affected by direct/distant neighbors.Ø Information diffusionØ Users with high reputationsr'User EmbeddingRating Embedding Item EmbeddingConcatenationOutput Layer Concatenationβ1β2β3Attention Networkβ4.α1SequenceLearning Layerα2α4Attention Networkα3Dr. Anthony FaucihNews1h1h2h2US could see millions of coronaviruscases and 100,000 or more deathsh.hL -1L -1hLhL.Embedding Layer u[1]u[2]Random Walk LayerDeep Social Collaborative Filtering, RecSys, 2019. .u[l]10
DSCFCollaborative Filtering with users’ social relations(Social Recommendation)Users might be affected by direct/distant neighbors.Ø Information diffusionØ Users with high reputationsr'User EmbeddingRating Embedding Item EmbeddingConcatenationOutput Layer Concatenationβ1β2β3Attention Networkβ4.Dr. Anthony FauciBi-LSTM withattentionmechanismsα1SequenceLearning Layerα2α4Attention Networkα3hNews1h1h2h2US could see millions of coronaviruscases and 100,000 or more deaths.h.hL -1L -1hLhL.Social Sequencesvia Random WalktechniquesEmbedding Layer u[1]u[2]Random Walk LayerDeep Social Collaborative Filtering, RecSys, 2019 .u[l]11
DASOCollaborative Filtering with users’ social relations(Social Recommendation)Item DomainSocial Domain3p User behave and interact differently in the item/social domains.51Deep Adversarial Social Recommendation, IJCAI, 201912
DASOCollaborative Filtering with users’ social relations(Social Recommendation)Item DomainSocial Domain3p User behave and interact differently in the item/social domains.5Learning separated user representations in two domains.Deep Adversarial Social Recommendation, IJCAI, 2019113
DASOCollaborative Filtering with users’ social relations(Social Recommendation)Item DomainSocial Domain3p User behave and interact differently in the item/social domains.5Learning separated user representations in two domains.1Bidirectional Knowledge Transfer with Cycle ReconstructionDeep Adversarial Social Recommendation, IJCAI, 201914
Optimization for Ranking Tasksp Negative Sampling’s Main Issue: It often generates low-quality negative samples that do not help you learn goodrepresentation.Deep Adversarial Social Recommendation, IJCAI, 201915
Optimization for Ranking Tasksp Negative Sampling’s Main Issue: It often generates low-quality negative samples that do not help you learn goodrepresentation [Cai and Wang, 2018; Wang et al., ed“easy”/unrelatedinformativeDressDeep Adversarial Social Recommendation, IJCAI, 2019Men’s Wallets16
Optimization for Ranking Tasksp Negative Sampling’s Main Issue: It often generates low-quality negative samples that do not help you learn goodrepresentation [Cai and Wang, 2018; Wang et al., ed“easy”/unrelatedinformativeDressDeep Adversarial Social Recommendation, IJCAI, 2019Dynamically generate“difficult” negative samplesOptimization withAdversarial Learning(GAN)Men’s Wallets17
DASOCyclic UserModelingItem DomainAdversarial LearningSocial Domain Representations forGeneratorItem Domain Representations forGeneratorUser Representations on ItemDomain after Mapping (S- I)UIItem Domain Representations sSocial DomainAdversarial LearningRewardVUhI- ShS- IUser Representations on SocialDomain after Mapping (I- S)UISSocial Domain Representations DiscriminatorRewardLoss/RewardGeneratedSamplesp(v u)p(uk u)GeneratorGeneratorDeep Adversarial Social Recommendation, IJCAI, r18
DASOCyclic UserModelingItem DomainAdversarial LearningSocial Domain Representations forGeneratorItem Domain Representations forGeneratorUser Representations on ItemDomain after Mapping (S- I)UIItem Domain Representations sSocial DomainAdversarial LearningRewardVUhI- ShS- IUser Representations on SocialDomain after Mapping (I- S)UISSocial Domain Representations DiscriminatorRewardLoss/RewardGeneratedSamplesp(v u)p(uk u)GeneratorGeneratorDeep Adversarial Social Recommendation, IJCAI, r19
Item Domain Discriminator Modelp DiscriminatorGoal: distinguish real user-item pairs (i.e., real samples) and thegenerated “fake” samples (relevant)(Sigmoid)Score function:Deep Adversarial Social Recommendation, IJCAI, 201920
Item Domain Generator Modelp Generator ModelGoal:1. Approximate the underlying real conditional distribution pIreal(v ui)2. Generate (select/sample) the most relevant items for any given user ui.the transferred user representationfrom social domainOptimization withPolicy Gradient21
Sequential (Session-based) Recommendationuser’s sequential behavior0.80.60.1Next ItemSession-based Recommendations with Recurrent Neural Networks, ICLR, 2016.BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, CIKM, 2019.22
Sequential (Session-based) Recommendationuser’s sequential behavior0.80.60.1Next ItemGRU based sequential recommendation method(GRU4Rec)Session-based Recommendations with Recurrent Neural Networks, ICLR, 2016.BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, CIKM, 2019.23
Sequential (Session-based) Recommendationuser’s sequential behavior0.80.60.1Next ItemTransformerLayerGRU based sequential recommendation method(GRU4Rec)BERT4RecSession-based Recommendations with Recurrent Neural Networks, ICLR, 2016.BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, CIKM, 2019.24
Shortcomings of Existing Deep Recommender SystemsRecommendation Policies§ Offline optimization§ Short-term reward
Shortcomings of Existing Deep Recommender SystemsUsers Items𝑢!𝑣!𝑢!𝑣"𝑢!𝑣#𝑢"𝑣! Recommendation Policies§ Offline optimization§ Short-term rewardGraph-structured Data§ Information Isolated IslandIssue: ignore implicit/explicitrelationships among instances26
Shortcomings of Existing Deep Recommender SystemsUsers Items𝑢!𝑣!𝑢!𝑣"𝑢!𝑣#𝑢"𝑣! 00Field 1Recommendation Policies§ Offline optimization§ Short-term rewardGraph-structured Data§ Information Isolated IslandIssue: ignore implicit/explicitrelationships among instances1100Field m010Field MManually Deisgned Architectures§ Expert knowledge§ Time and engineering efforts27
Wide & Deep Learning for Recommender Systems, 1st DLRS, 2016, Standard MLP Embedding Concatenation , pThe wide linear models can memorize seen feature interactions using cross-product feature transformations. pThe deep models can generalize to previously unseen feature interactions through low-dimensional embeddings. 5, Neural FM,
