做淘客网站怎么建要购买数据库吗,wordpress广告联盟,wordpress文章显示在页面,wordpress导航添加登陆按钮Making Recommendations 文章目录协作型过滤搜集偏好寻找相近的用户欧几里得距离评价皮尔逊相关度评价应该选用哪一种相似性度量方法为评分者打分推荐物品匹配相似商品构建一个基于某数据平台的链接推荐系统数据平台API构造数据集推荐近邻与链接基于物品的过滤构造物品比较数据…Making Recommendations 文章目录协作型过滤搜集偏好寻找相近的用户欧几里得距离评价皮尔逊相关度评价应该选用哪一种相似性度量方法为评分者打分推荐物品匹配相似商品构建一个基于某数据平台的链接推荐系统数据平台API构造数据集推荐近邻与链接基于物品的过滤构造物品比较数据集获得推荐使用MovieLens数据集基于用户进行过滤还是基于物品进行过滤小结源码recommendations.py源码如何根据群体偏好来为人们提供推荐。
协作型过滤
Collaborative Filtering
一个协作型过滤算法通常的做法是对一大群人进行搜索并从中找出与我们品味相近的一群人。
算法会对这些人所偏爱的其他内容考察并将它们组合起来构造出一个经过排名的推荐列表。有许多不同的方法可以帮助我们确定哪些人与自己品味相近并将他们的选择组合成列表。
搜集偏好
Collecting Preferences
这里使用Python嵌套字典保存影迷对电影的评价。
数据结构
评分{人a:{电影A:(1~5评分),电影B:(1~5评分),...},人b:......
}# A dictionary of movie critics and their ratings of a small
# set of movies
critics{Lisa Rose: {Lady in the Water: 2.5, Snakes on a Plane: 3.5,Just My Luck: 3.0, Superman Returns: 3.5, You, Me and Dupree: 2.5, The Night Listener: 3.0},
Gene Seymour: {Lady in the Water: 3.0, Snakes on a Plane: 3.5, Just My Luck: 1.5, Superman Returns: 5.0, The Night Listener: 3.0, You, Me and Dupree: 3.5},
Michael Phillips: {Lady in the Water: 2.5, Snakes on a Plane: 3.0,Superman Returns: 3.5, The Night Listener: 4.0},
Claudia Puig: {Snakes on a Plane: 3.5, Just My Luck: 3.0,The Night Listener: 4.5, Superman Returns: 4.0, You, Me and Dupree: 2.5},
Mick LaSalle: {Lady in the Water: 3.0, Snakes on a Plane: 4.0, Just My Luck: 2.0, Superman Returns: 3.0, The Night Listener: 3.0,You, Me and Dupree: 2.0},
Jack Matthews: {Lady in the Water: 3.0, Snakes on a Plane: 4.0,The Night Listener: 3.0, Superman Returns: 5.0, You, Me and Dupree: 3.5},
Toby: {Snakes on a Plane:4.5,You, Me and Dupree:1.0,Superman Returns:4.0}}recommendations.py源码 import osos.getcwd() os.chdir(.) from recommendations import criticscritics[Lisa Rose][Lady in the Water]
2.5critics[Toby][Snakes on a Plane]4.5critics[Toby]
{Snakes on a Plane: 4.5, Superman Returns: 4.0, You, Me and Dupree: 1.0}寻找相近的用户
Finding Similar Users
我们需要方法来确定人们品味的相似程度。
为此我们可将每个人与所有其他人进行对比并计算他们的相似度评价值。
有两种方法
欧几里得距离皮尔逊相关度。
欧几里得距离评价
Euclidean Distance Score 若两人在“偏好空间”中的距离越近他们的兴趣偏好就越相似。
多数量的评分项同样适用这距离公式。
计算两人间距离 from math import sqrtsqrt(pow(4.5-4,2)pow(1-2,2))
1.118033988749895通常要对这数进行处理来对偏好越相近的情况给出越大的值。
因此可计算得到距离值加1这可避免遇到被零整除的错误并取其倒数 1/(1sqrt(pow(4.5-4,2)pow(1-2,2)))
0.4721359549995794这样就返回0与1之间的值值越大偏好更相似。
构造出用来计算相似度的函数。
from math import sqrt# Returns a distance-based similarity score for person1 and person2
def sim_distance(prefs,person1,person2):# Get the list of shared_itemssi{}for item in prefs[person1]: if item in prefs[person2]: si[item]1# if they have no ratings in common, return 0if len(si)0: return 0# Add up the squares of all the differencessum_of_squaressum([pow(prefs[person1][item]-prefs[person2][item],2) for item in prefs[person1] if item in prefs[person2]])return 1/(1sum_of_squares)使用示例 import recommendationsrecommendations.sim_distance(recommendations.critics,Lisa Rose, Gene Seymour)
0.14814814814814814皮尔逊相关度评价
Pearson Correlation Score 皮尔逊相关系数是判断两组数据与某一直线拟合程度的一种度量。
对应的公式比欧式距离公式要复杂但是它在数据不是很规范normalized的时候譬如影评人对影片的评价总是相对于平均水平偏离很大时会倾向于给出更好的结果。 上图可看出一条直线。因其绘制原则是尽可能地靠近图上的所有坐标点故而被称作最佳拟合线best-fit line。
若两位评论者对所有影片的评分情况都相同那么这条直线将成为对角线并且会与图上所有的坐标点都相交从而得到一个结果为1的理想相关度评价。
比上图有更佳拟合度的图。 皮尔逊方法能修正“夸大分值grade inflation”。若某人总是倾向于给出比另一个人更高的分值而二者的分值之差又始终保持一致则他们依然可能会存在很好的相关性。
换成欧式距离公式评价方法会因为一个人的评价始终比另一个人的更为“严格”从而导致评价始终相对偏低得出两者不相近的结论即使他们品味很相似也是如此。
构造出用来计算相似度的函数。
# Returns the Pearson correlation coefficient for p1 and p2
def sim_pearson(prefs,p1,p2):# Get the list of mutually rated itemssi{}for item in prefs[p1]: if item in prefs[p2]: si[item]1# if they are no ratings in common, return 0if len(si)0: return 0# Sum calculationsnlen(si)# Sums of all the preferences#1sum1sum([prefs[p1][it] for it in si])#2sum2sum([prefs[p2][it] for it in si])# Sums of the squares#3sum1Sqsum([pow(prefs[p1][it],2) for it in si])#4sum2Sqsum([pow(prefs[p2][it],2) for it in si]) # Sum of the products#5pSumsum([prefs[p1][it]*prefs[p2][it] for it in si])# Calculate r (Pearson score)numpSum-(sum1*sum2/n)densqrt((sum1Sq-pow(sum1,2)/n)*(sum2Sq-pow(sum2,2)/n))if den0: return 0rnum/denreturn r该函数将返回一个介于-1与1之间的数值。值为1则表明两个人对每一样物品均有则完全一致的评价。 recommendations.sim_pearson(recommendations.critics,Lisa Rose, Gene Seymour)
0.39605901719066977应该选用哪一种相似性度量方法
哪一种方法最优完全取决于具体的应用。
为评分者打分
给出前几名相似度高的评论者。
# Returns the best matches for person from the prefs dictionary.
# Number of results and similarity function are optional params.
def topMatches(prefs,person,n5,similaritysim_pearson):scores[(similarity(prefs,person,other),other) for other in prefs if other!person]scores.sort()scores.reverse()return scores[0:n]使用示例 recommendations.topMatches(recommendations.critics,Toby,n3)
[(0.9912407071619299, Lisa Rose), (0.9244734516419049, Mick LaSalle), (0.8934051474415647, Claudia Puig)]推荐物品
先前程序是找趣味相近的人。
我们希望一个影片的推荐。 当然可以找出趣味相近的人喜欢的影片中找出一部自己还未看过的影片。
这做法有点随意permissive
这是因为
有可能该评论者未对某些影片作出评论而这些影片恰恰是你中意的。有可能该评论者对某影片情有独钟其他评论者却对此影片嗤之以鼻。
因此需要通过一个经过加权的评价值来为影片打分评论者的评分结果因此而形成了先后排名的。 # Gets recommendations for a person by using a weighted average
# of every other users rankings
def getRecommendations(prefs,person,similaritysim_pearson):totals{}simSums{}for other in prefs:# dont compare me to myselfif otherperson: continuesimsimilarity(prefs,person,other)# ignore scores of zero or lowerif sim0: continuefor item in prefs[other]:# only score movies I havent seen yetif item not in prefs[person] or prefs[person][item]0:# Similarity * Scoretotals.setdefault(item,0)totals[item]prefs[other][item]*sim# Sum of similaritiessimSums.setdefault(item,0)simSums[item]sim# Create the normalized listrankings[(total/simSums[item],item) for item,total in totals.items()]# Return the sorted listrankings.sort()rankings.reverse()return rankings接下来便可以得到适合的影片推荐 import recommendationsrecommendations.getRecommendations(recommendations.critics,Toby)
[(3.3477895267131013, The Night Listener), (2.8325499182641614, Lady in the Water), (2.5309807037655645, Just My Luck)]recommendations.getRecommendations(recommendations.critics,Toby,similarityrecommendations.sim_distance)
[(3.5002478401415877, The Night Listener), (2.7561242939959363, Lady in the Water), (2.461988486074374, Just My Luck)]到此我们已经建立起了一个完整的推荐系统它适用于任何类型的商品或网路链接。
我们所要做的全部事情就是建立一个涉及人员、物品和评价值的字典然后就可以借此来为任何人提供建议。
匹配相似商品
Matching Products
我们想了解哪些商品是彼此相近。 在这种情况下我们可以通过查看某一特定物品被哪些人喜欢以及哪些其他物品被这些人喜欢来决定相似程度。
事实上和先前来决定人与人之间相似度的方法是一样的——只需将人员与物品对换即可。
先前的数据结构
评分{人a:{电影A:(1~5评分),电影B:(1~5评分),...},人b:......
}换成
评分{电影A:{人a:(1~5评分),人b:(1~5评分),...},电影B:......
}定义一个函数来进行对换
def transformPrefs(prefs):result{}for person in prefs:for item in prefs[person]:result.setdefault(item,{})# Flip item and personresult[item][person]prefs[person][item]return result然后调用topMatche函数得到一组与《Superman Returns》最为相近的影片 movies recommendations.transformPrefs(recommendations.critics)recommendations.topMatches(movies,Superman Returns)
[(0.6579516949597695, You, Me and Dupree), (0.4879500364742689, Lady in the Water), (0.11180339887498941, Snakes on a Plane), (-0.1798471947990544, The Night Listener), (-0.42289003161103106, Just My Luck)]Just My Luck’与Superman Returns呈负相关关系 我们还可以为影片推荐评论者例如我们正在考虑邀请谁和自己一起参加某部影片的重映。 recommendations.getRecommendations(movies, Just My Luck)
[(4.0, Michael Phillips), (3.0, Jack Matthews)]更多案例
为了向不同的个体推荐商品在线零售商可能会收集人们的购买历史然后找到购买商品的潜在客户。在专门推荐链接的网站上这样做可以确保新出的链接能够被那些最有可能对它产生兴趣的网站用户找到。
构建一个基于某数据平台的链接推荐系统
这数据平台是del.icio.us在线书签网站。该网址登陆不了。
目标利用平台的数据查找相近的用户并向他们推荐以前未曾看过的链接。
数据平台API
pydelicious.py以及deliciousrec.py未知原因不能使用可能网站问题
下面两节置空。
构造数据集
略
推荐近邻与链接
略
基于物品的过滤
Item-Based Filtering
考虑到性能上的问题。
试想对于Amazon有这海量客户将一个客户 和 所有其他客户进行比较然后再对 每位客户评分过的商品进行比较工作量何其巨大。
同样一商品销售量百万的网站或许客户在偏好方面彼此间很少有重叠这可能令客户的相似性判断变得十分困难。 先前才用到的技术成为基于用户的协作型过滤user-based collaborative filtering。
接下来介绍基于物品的协作型过滤item-based collaborative filtering。
基于物品的协作型过滤的优点
在拥有大量数据集的情况下基于基于物品的协作型过滤能够得出更好的结论而且它允许将大量计算任务预先执行空间换时间从而需要给予推荐的用户能够更快地得到他们所要的结果。 基于物品的协作型过滤的总体思路是
为每件物品预先计算好最为相近的其它物品。然后当为某位客户提供推荐就可以查看它曾经评分过得的物品从中选出排位靠前者再构造出一个加权列表其中包含看了与这些选中物品最为相近的其他物品。 这里最为显著的区别在于尽管第一步要求我们检查所有的数据但是物品间的比较不会像用户间的比较那么频繁。
这就意味着无需不停计算与每样物品最为相近的其他物品可将这样的运算任务安排在网络流量不是很大的时候进行或者在独立于主应用之外的另一台计算机上独立进行空间换时间。
构造物品比较数据集
def calculateSimilarItems(prefs,n10):# Create a dictionary of items showing which other items they# are most similar to.result{}# Invert the preference matrix to be item-centricitemPrefstransformPrefs(prefs)c0for item in itemPrefs:# Status updates for large datasetsc1if c%1000: print %d / %d % (c,len(itemPrefs))# Find the most similar items to this one#物-人 数据集 得出 最相似的 物scorestopMatches(itemPrefs,item,nn,similaritysim_distance)result[item]scoresreturn result运用示例 import recommendationsitemsim recommendations.calculateSimilarItems(recommendations.critics)itemsim
{Lady in the Water: [(0.4, You, Me and Dupree), (0.2857142857142857, The Night Listener), (0.2222222222222222, Snakes on a Plane), (0.2222222222222222, Just My Luck), (0.09090909090909091, Superman Returns)],
Snakes on a Plane: [(0.2222222222222222, Lady in the Water), (0.18181818181818182, The Night Listener), (0.16666666666666666, Superman Returns), (0.10526315789473684, Just My Luck), (0.05128205128205128, You, Me and Dupree)],
Just My Luck: [(0.2222222222222222, Lady in the Water), (0.18181818181818182, You, Me and Dupree), (0.15384615384615385, The Night Listener), (0.10526315789473684, Snakes on a Plane), (0.06451612903225806, Superman Returns)],
Superman Returns: [(0.16666666666666666, Snakes on a Plane), (0.10256410256410256, The Night Listener), (0.09090909090909091, Lady in the Water), (0.06451612903225806, Just My Luck), (0.05333333333333334, You, Me and Dupree)],
You, Me and Dupree: [(0.4, Lady in the Water), (0.18181818181818182, Just My Luck), (0.14814814814814814, The Night Listener), (0.05333333333333334, Superman Returns), (0.05128205128205128, Snakes on a Plane)],
The Night Listener: [(0.2857142857142857, Lady in the Water), (0.18181818181818182, Snakes on a Plane), (0.15384615384615385, Just My Luck), (0.14814814814814814, You, Me and Dupree), (0.10256410256410256, Superman Returns)]}获得推荐
现在已经可以在不遍历整个数据集的情况下利用反映物品相似度的字典上一节给出的来给出推荐。
可以取到用户评价过的所有物品找出其相似物品并根据相似度对其进行加权。
可以容易地根据物品字典来得到相似度。
利用基于物品的方法寻找推荐过程 先前[#推荐物品]章节用到的表 本节用到表与[#推荐物品]表不同之处是没有涉及所有评论者
def getRecommendedItems(prefs,itemMatch,user):userRatingsprefs[user]scores{}totalSim{}# Loop over items rated by this userfor (item,rating) in userRatings.items( ):# Loop over items similar to this onefor (similarity,item2) in itemMatch[item]:# Ignore if this user has already rated this itemif item2 in userRatings: continue# Weighted sum of rating times similarityscores.setdefault(item2,0)scores[item2]similarity*rating# Sum of all the similaritiestotalSim.setdefault(item2,0)totalSim[item2]similarity# Divide each total score by total weighting to get an averagerankings[(score/totalSim[item],item) for item,score in scores.items( )]# Return the rankings from highest to lowestrankings.sort( )rankings.reverse( )return rankings运行示例 recommendations.getRecommendedItems(recommendations.critics,itemsim,Toby)
[(3.182634730538922, The Night Listener), (2.5983318700614575, Just My Luck), (2.4730878186968837, Lady in the Water)]使用MovieLens数据集
涉及电影评价的真实数据集
网站提供很多有关电影数据集最后选择大小最小的文件ml-100k.zip
该网站还有图书笑话等数据等待你的发掘。
注个人将数据文件放置配套的源码包中。 只需关注的文件之一u.item
文件的前5行
一组有关影片ID和片名的列表
1|Toy Story (1995)|01-Jan-1995||http://us.imdb.com/M/title-exact?Toy%20Story%20(1995)|0|0|0|1|1|1|0|0|0|0|0|0|0|0|0|0|0|0|0
2|GoldenEye (1995)|01-Jan-1995||http://us.imdb.com/M/title-exact?GoldenEye%20(1995)|0|1|1|0|0|0|0|0|0|0|0|0|0|0|0|0|1|0|0
3|Four Rooms (1995)|01-Jan-1995||http://us.imdb.com/M/title-exact?Four%20Rooms%20(1995)|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1|0|0
4|Get Shorty (1995)|01-Jan-1995||http://us.imdb.com/M/title-exact?Get%20Shorty%20(1995)|0|1|0|0|0|1|0|0|1|0|0|0|0|0|0|0|0|0|0
5|Copycat (1995)|01-Jan-1995||http://us.imdb.com/M/title-exact?Copycat%20(1995)|0|0|0|0|0|0|1|0|1|0|0|0|0|0|0|0|1|0|0只需关注的文件之二u.data
文件的前5行
用户id 影片id 用户对影片的评分 用户的评价时间196 242 3 881250949
186 302 3 891717742
22 377 1 878887116
244 51 2 880606923
166 346 1 886397596加载文件的函数
def loadMovieLens(path/data/movielens):# Get movie titlesmovies{}for line in open(path/u.item):(id,title)line.split(|)[0:2]movies[id]title# Load dataprefs{}for line in open(path/u.data):(user,movieid,rating,ts)line.split(\t)prefs.setdefault(user,{})prefs[user][movies[movieid]]float(rating)return prefs运用示例 prefs[20]
{Return of the Jedi (1983): 4.0, Jungle2Jungle (1997): 4.0, Back to the Future (1985): 3.0, Jurassic Park (1993): 4.0, Sting, The (1973): 3.0, Sabrina (1995): 4.0, Island of Dr. Moreau, The (1996): 1.0, Mission: Impossible (1996): 3.0,
Twister (1996): 4.0, Toy Story (1995): 3.0, Willy Wonka and the Chocolate Factory (1971): 3.0, Sound of Music, The (1965): 3.0, Home Alone (1990): 2.0, Scream (1996): 1.0, Braveheart (1995): 5.0, Indiana Jones and the Last Crusade (1989): 4.0,
Young Frankenstein (1974): 2.0, Raiders of the Lost Ark (1981): 4.0, Dantes Peak (1997): 4.0, Mr. Hollands Opus (1995): 4.0, Die Hard (1988): 2.0, Speed (1994): 4.0, Michael (1996): 1.0, Christmas Carol, A (1938): 4.0, Lost World: Jurassic Park, The (1997): 4.0,
Ghost and the Darkness, The (1996): 5.0,African Queen, The (1951): 3.0, Space Jam (1996): 2.0, Ransom (1996): 4.0, Silence of the Lambs, The (1991): 3.0, Searching for Bobby Fischer (1993): 5.0, Preachers Wife, The (1996): 4.0, Blues Brothers, The (1980): 3.0, Happy Gilmore (1996): 1.0, Volcano (1997): 4.0,
Aliens (1986): 2.0, Independence Day (ID4) (1996): 3.0, E.T. the Extra-Terrestrial (1982): 2.0, Seven (Se7en) (1995): 2.0, Forrest Gump (1994): 1.0, Aladdin (1992): 3.0, Miracle on 34th Street (1994): 3.0, Empire Strikes Back, The (1980): 3.0, Eraser (1996): 3.0,
Its a Wonderful Life (1946): 5.0, Star Wars (1977): 3.0, Beauty and the Beast (1991): 4.0, One Flew Over the Cuckoos Nest (1975): 1.0}基于用户的推荐 recommendations.getRecommendations(prefs,20)[0:10]
[(5.0, World of Apu, The (Apur Sansar) (1959)), (5.0, Whole Wide World, The (1996)), (5.0, Thieves (Voleurs, Les) (1996)), (5.0, Strawberry and Chocolate (Fresa y chocolate) (1993)),
(5.0, Star Kid (1997)), (5.0, Someone Elses America (1995)), (5.0, Sliding Doors (1998)), (5.0, Santa with Muscles (1996)), (5.0, Saint of Fort Washington, The (1993)), (5.0, Quiet Room, The (1996))]基于物品的推荐 itemsim recommendations.calculateSimilarItems(prefs,n50)
100 / 1664
200 / 1664
300 / 1664
400 / 1664
500 / 1664
600 / 1664
700 / 1664
800 / 1664
900 / 1664
1000 / 1664
1100 / 1664
1200 / 1664
1300 / 1664
1400 / 1664
1500 / 1664
1600 / 1664 recommendations.getRecommendedItems(prefs,itemsim,87)[0:30]
[(5.0, Whats Eating Gilbert Grape (1993)), (5.0, Vertigo (1958)), (5.0, Usual Suspects, The (1995)), (5.0, Toy Story (1995)), (5.0, Titanic (1997)), (5.0, Sword in the Stone, The (1963)), (5.0, Stand by Me (1986)), (5.0, Sling Blade (1996)), (5.0, Silence of the Lambs, The (1991)),
(5.0, Shining, The (1980)), (5.0, Shine (1996)), (5.0, Sense and Sensibility (1995)), (5.0, Scream (1996)), (5.0, Rumble in the Bronx (1995)), (5.0, Rock, The (1996)), (5.0, Robin Hood: Prince of Thieves (1991)), (5.0, Reservoir Dogs (1992)), (5.0, Police Story 4: Project S (Chao ji ji hua) (1993)), (5.0, House of the Spirits, The (1993)),
(5.0, Fresh (1994)), (5.0, Denise Calls Up (1995)), (5.0, Day the Sun Turned Cold, The (Tianguo niezi) (1994)), (5.0, Before the Rain (Pred dozhdot) (1994)), (5.0, Assignment, The (1997)), (5.0, 1-900 (1994)), (4.875, Eds Next Move (1996)), (4.833333333333333, Anna (1996)),
(4.8, Dark City (1998)), (4.75, Flower of My Secret, The (Flor de mi secreto, La) (1995)), (4.75, Broken English (1996))]基于用户进行过滤还是基于物品进行过滤
在针对大数据集成生成推荐列表时基于物品进行过滤的方式明显要比基于用户的过滤更快不过他的确有维护物品相似度表的额外开销。
对于稀疏数据集如大多数书签都是为小众所收藏基于物品的过滤方法通常要优于基于用户的过滤方法而对于密集数据集电影评价而言两者效果几乎一致。
基于用户的过滤方法更加易于实现而且无需额外步骤因此它通常更适用于规模较小的变化非常频繁的内存数据集。
在应用方面告诉用户还有哪些人与自己有着相近偏好是有一定价值的如有交友业务相关的。但对于一个购物网站而言并不想这么做。
小结 相似度评价值两方法 欧几里得距离公式皮尔逊相关度公式 协作型过滤 基于用户的协作型过滤user-based collaborative filtering基于物品的协作型过滤item-based collaborative filtering 基本原数据结构
人-物
评分{人a:{电影A:(1~5评分),电影B:(1~5评分),...},人b:......
}物-人
评分{人a:{电影A:(1~5评分),电影B:(1~5评分),...},人b:......
}本文用到的recommendations.py源码中重要函数或变量
函数或变量作用备注讲解所在章节critics人-物数据集prefs参数sim_distance(prefs,person1,person2)欧几里得距离公式计算相似度recommendations.sim_distance(recommendations.critics,‘Lisa Rose’, ‘Gene Seymour’)[#欧几里得距离评价]sim_pearson(prefs,p1,p2)皮尔逊相关度公式计算相似度recommendations.sim_pearson(recommendations.critics,‘Lisa Rose’, ‘Gene Seymour’)[#皮尔逊相关度评价]topMatches(prefs,person,n5,similaritysim_pearson)根据给出 人-物 / 物-人数据集相关度计算算法欧氏皮氏 得出前n最相似个人 / 物recommendations.topMatches(recommendations.critics,‘Toby’,n3)[#为评分者打分][#匹配相似商品]getRecommendations(prefs,person,similaritysim_pearson)根据人-物数据集 相似度计算后再加权平均得出推荐物recommendations.getRecommendations(recommendations.critics,‘Toby’)recommendations.getRecommendations(movies, ‘Just My Luck’)[#推荐物品][#匹配相似商品]transformPrefs(prefs)人-物数据集转换成物-人数据集recommendations.transformPrefs(recommendations.critics)[#匹配相似商品]calculateSimilarItems(prefs,n10)物-人 数据集 得出 各物的相似度记作物-物recommendations.calculateSimilarItems(recommendations.critics)[#构造物品比较数据集]getRecommendedItems(prefs,itemMatch,user)一个人-物物-物数据集加权平均 得出推荐物recommendations.getRecommendedItems(recommendations.critics,itemsim,‘Toby’)[#获得推荐]
源码
recommendations.py源码
# A dictionary of movie critics and their ratings of a small
# set of movies
critics{Lisa Rose: {Lady in the Water: 2.5, Snakes on a Plane: 3.5,Just My Luck: 3.0, Superman Returns: 3.5, You, Me and Dupree: 2.5, The Night Listener: 3.0},
Gene Seymour: {Lady in the Water: 3.0, Snakes on a Plane: 3.5, Just My Luck: 1.5, Superman Returns: 5.0, The Night Listener: 3.0, You, Me and Dupree: 3.5},
Michael Phillips: {Lady in the Water: 2.5, Snakes on a Plane: 3.0,Superman Returns: 3.5, The Night Listener: 4.0},
Claudia Puig: {Snakes on a Plane: 3.5, Just My Luck: 3.0,The Night Listener: 4.5, Superman Returns: 4.0, You, Me and Dupree: 2.5},
Mick LaSalle: {Lady in the Water: 3.0, Snakes on a Plane: 4.0, Just My Luck: 2.0, Superman Returns: 3.0, The Night Listener: 3.0,You, Me and Dupree: 2.0},
Jack Matthews: {Lady in the Water: 3.0, Snakes on a Plane: 4.0,The Night Listener: 3.0, Superman Returns: 5.0, You, Me and Dupree: 3.5},
Toby: {Snakes on a Plane:4.5,You, Me and Dupree:1.0,Superman Returns:4.0}}from math import sqrt# Returns a distance-based similarity score for person1 and person2
def sim_distance(prefs,person1,person2):# Get the list of shared_itemssi{}for item in prefs[person1]: if item in prefs[person2]: si[item]1# if they have no ratings in common, return 0if len(si)0: return 0# Add up the squares of all the differencessum_of_squaressum([pow(prefs[person1][item]-prefs[person2][item],2) for item in prefs[person1] if item in prefs[person2]])return 1/(1sum_of_squares)# Returns the Pearson correlation coefficient for p1 and p2
def sim_pearson(prefs,p1,p2):# Get the list of mutually rated itemssi{}for item in prefs[p1]: if item in prefs[p2]: si[item]1# if they are no ratings in common, return 0if len(si)0: return 0# Sum calculationsnlen(si)# Sums of all the preferencessum1sum([prefs[p1][it] for it in si])sum2sum([prefs[p2][it] for it in si])# Sums of the squaressum1Sqsum([pow(prefs[p1][it],2) for it in si])sum2Sqsum([pow(prefs[p2][it],2) for it in si]) # Sum of the productspSumsum([prefs[p1][it]*prefs[p2][it] for it in si])# Calculate r (Pearson score)numpSum-(sum1*sum2/n)densqrt((sum1Sq-pow(sum1,2)/n)*(sum2Sq-pow(sum2,2)/n))if den0: return 0rnum/denreturn r# Returns the best matches for person from the prefs dictionary.
# Number of results and similarity function are optional params.
def topMatches(prefs,person,n5,similaritysim_pearson):scores[(similarity(prefs,person,other),other) for other in prefs if other!person]scores.sort()scores.reverse()return scores[0:n]# Gets recommendations for a person by using a weighted average
# of every other users rankings
def getRecommendations(prefs,person,similaritysim_pearson):totals{}simSums{}for other in prefs:# dont compare me to myselfif otherperson: continuesimsimilarity(prefs,person,other)# ignore scores of zero or lowerif sim0: continuefor item in prefs[other]:# only score movies I havent seen yetif item not in prefs[person] or prefs[person][item]0:# Similarity * Scoretotals.setdefault(item,0)totals[item]prefs[other][item]*sim# Sum of similaritiessimSums.setdefault(item,0)simSums[item]sim# Create the normalized listrankings[(total/simSums[item],item) for item,total in totals.items()]# Return the sorted listrankings.sort()rankings.reverse()return rankingsdef transformPrefs(prefs):result{}for person in prefs:for item in prefs[person]:result.setdefault(item,{})# Flip item and personresult[item][person]prefs[person][item]return resultdef calculateSimilarItems(prefs,n10):# Create a dictionary of items showing which other items they# are most similar to.result{}# Invert the preference matrix to be item-centricitemPrefstransformPrefs(prefs)c0for item in itemPrefs:# Status updates for large datasetsc1if c%1000: print %d / %d % (c,len(itemPrefs))# Find the most similar items to this onescorestopMatches(itemPrefs,item,nn,similaritysim_distance)result[item]scoresreturn resultdef getRecommendedItems(prefs,itemMatch,user):userRatingsprefs[user]scores{}totalSim{}# Loop over items rated by this userfor (item,rating) in userRatings.items( ):# Loop over items similar to this onefor (similarity,item2) in itemMatch[item]:# Ignore if this user has already rated this itemif item2 in userRatings: continue# Weighted sum of rating times similarityscores.setdefault(item2,0)scores[item2]similarity*rating# Sum of all the similaritiestotalSim.setdefault(item2,0)totalSim[item2]similarity# Divide each total score by total weighting to get an averagerankings[(score/totalSim[item],item) for item,score in scores.items( )]# Return the rankings from highest to lowestrankings.sort( )rankings.reverse( )return rankingsdef loadMovieLens(path/data/movielens):# Get movie titlesmovies{}for line in open(path/u.item):(id,title)line.split(|)[0:2]movies[id]title# Load dataprefs{}for line in open(path/u.data):(user,movieid,rating,ts)line.split(\t)prefs.setdefault(user,{})prefs[user][movies[movieid]]float(rating)return prefs
