Topic Model Based Recommendation Systems

基于主题模型的推荐系统

Topic Model Based Recommendation Systems

基于主题模型的推荐系统

A very quick and (hopefully) easy to follow introduction into the intuition (and very low level Maths) involved in Topic Model Based Recommendation Systems.

关于基于主题模型的推荐系统背后直觉（以及极低层次数学）的一个快速且（希望）易于理解的入门介绍。

Check out my GitHub for a working simple recommendation system based on Topic Modelling.

查看我的GitHub，了解一个基于主题建模的简单可运行的推荐系统。

In todays world, sometimes it feels like we are plagued with never ending decisions. Whether it be the Friday night movie or the next song to keep people dancing at an NYE party.

在当今世界，有时我们感觉被永无止境的选择所困扰——无论是周五晚上的电影，还是跨年派对上让人们继续跳舞的下一首歌曲。

So how do recommendation systems actually work? In this article I’m going to explain one approach based on Topic Modelling using a Latent Dirichlet Allocation (LDA).

那么推荐系统究竟是如何工作的呢？在本文中，我将解释一种基于主题建模的方法，具体使用潜在狄利克雷分配（LDA）。

Topic Modelling

主题建模

Before we talk about how to model a topic, we need to first understand what a topic actually is.

在讨论如何为主题建模之前，我们首先需要理解主题到底是什么。

This is not an intuitive idea to think about so we will describe it in terms of collections of words.

这并不是一个直观的概念，因此我们将用单词集合来描述它。

If we have a collection of documents randomly selected from a database, we can imagine that some of the words contained in these documents may be semantically similar, or be related to the same area.

如果我们从数据库中随机选取一批文档，可以想象其中某些单词在语义上是相似的，或者属于同一领域。

For example, if these documents were a collection of film reviews. We can imagine that we might be able to form groups of positive and negative reviews based on the words contained within them. Alternatively, we may wish to form collections of documents relating to Sci-Fi, Comedy, Romance etc.

例如，假设这些文档是影评集合。我们可以想象，根据其中包含的词语，也许能够形成正面和负面评论的组别。或者，我们可能希望形成与科幻、喜剧、爱情等相关的文档集合。

So, as we are starting to reorganise this collection of documents into many smaller collections. At the same time, we are starting to see that there are many layers of possibilities. Where each possibility is a selection of topics.

因此，当我们开始将这一文档集合重新组织成许多较小的集合时，同时也会看到许多层可能性，其中每一层可能性都对应着一组主题选择。

You may then ask the question, how can we get a computer to organise these documents into topics and how do we know what topics it will pick? To answer this, we are going to think at a slightly deeper level…

这时你可能会问：我们如何让计算机将这些文档组织成主题？我们又如何知道它会选择哪些主题？为了回答这个问题，我们需要从稍微深一点的层次来思考……

The Less General Idea

不那么宏观的思路

In this article, I am going to describe one option for how to get a computer to perform topic modelling. However there are many other algorithms, approaches and methodologies out in the wild.

在本文中，我将介绍如何让计算机执行主题建模的一种选项。当然，业界还有许多其他算法、方法和方法论。

Going back to our collection of documents and sticking with the approach of looking at the words contained within them, we can build up a vocabulary containing all of the unique words in the database.

回到我们的文档集合，继续采用查看其中单词的方法，我们可以构建一个包含数据库中所有唯一单词的词汇表。

Say we have 1000 different unique words across 4 documents and we want to characterise each document by which words are contained within them (and by how many of each word).

假设我们在4个文档中有1000个不同的唯一单词，我们希望根据每个文档包含的单词（以及每个单词出现的次数）来表征每个文档。

So now we can imagine that for each document, we have a vector with dimension 1000 (one dimension for each unique word). And at each position in each vector there is the count of how many times the word that this position corresponds to, appears in the document.

因此，我们可以想象，对于每个文档，我们有一个维度为1000的向量（每个唯一单词占一个维度）。在每个向量的每个位置上，记录着该位置对应单词在文档中出现的次数。

For example, the first position in the vector corresponds to the first word in the vocabulary, which we will say is “Robot”. The first document is a film review about Terminator 23 (or whatever number we are on now…) and so the word “Robot” is mentioned 19 times. Therefore in the first position of the vector corresponding to the first document, we have (19,…).

例如，向量中的第一个位置对应词汇表中的第一个单词，我们假设它是“Robot”。第一个文档是关于《终结者23》（或者我们现在拍到第几部了……）的影评，因此“Robot”这个词出现了19次。于是，在第一个文档对应的向量的第一个位置上，我们得到 (19, …)。

The second position corresponds to the word “Sport” and is mentioned zero times in the Terminator review which gives us (19, 0, …) and so on…

第二个位置对应单词“Sport”，在《终结者》影评中出现0次，于是得到 (19, 0, …)，以此类推。

The second document happens to be a review for a Tennis documentary and so for this document we have the vector (0, 10, …), since “Robot” is mentioned 0 times.

第二个文档恰好是一部网球纪录片的影评，因此对于这个文档，我们有向量 (0, 10, …)，因为“Robot”出现了0次。

Too many topics?

主题太多？

Yes, way too many. I agree, as will your computer.

是的，太多了。我同意，你的计算机也会同意。

At the moment we have essentially defined a “topic” for each word in the vocabulary. Which clearly is not ideal and will not provide us with many clearly separated topics to play with later on.

目前，我们实质上为词汇表中的每个单词都定义了一个“主题”。这显然不理想，也不会为我们提供许多清晰分离的主题供后续使用。

The next step then is to find a middle ground where each of our reviews belong to a broader topic which is defined by a number of words in the vocabulary.

下一步是找到一个折中方案，使每篇影评都属于一个由词汇表中若干单词共同定义的更宽泛的主题。

Latent Dirichlet Allocation

潜在狄利克雷分配（LDA）

We are now looking to reduce the number of connections coming from each document by introducing a hidden layer of topics between the individual words in the vocabulary.

我们现在希望通过在词汇表中的各个单词之间引入一个隐藏的主题层，来减少来自每个文档的连接数量。

This is exactly what we are going to use Latent Dirichlet Allocation (LDA) for.

这正是我们将使用潜在狄利克雷分配（LDA）的目的。

LDA requires us to define a required number of topics we want, this is what we call a hyperparameter (a parameter which is defined before the algorithm is run).

LDA要求我们定义所需的主题数量，这就是我们所说的超参数（在算法运行之前定义的参数）。

In a real world we can imagine scanning over many numbers of topics to find the best outcome (aptly called a hyperparameter scan).

在实际应用中，我们可以想象扫描多个主题数量以找到最佳结果（这恰当地称为超参数扫描）。

Let’s say we are looking for 10 topics. This means we wish to add a hidden layer between the previous connections we had, which were linking documents to words (remembering our large vectors (19, 0, …) and (0, 10, …)).

假设我们要寻找10个主题。这意味着我们希望在之前的连接（即文档到单词的连接，还记得我们的大向量 (19, 0, …) 和 (0, 10, …)）之间添加一个隐藏层。

Mathematically and computationally this is very desirable for us, since we can replace these huge vectors describing each document with new vectors of size 10 (or however many topics you have chosen).

这在数学和计算上对我们非常有利，因为我们可以用大小为10的新向量（或任何你选择的主题数）来代替描述每个文档的巨大向量。

To describe what the LDA is attempting to achieve, it is easiest to look at a matrix formulation below…

为了描述LDA试图实现的目标，最简单的方式是看下面的矩阵公式：

So in our original perfect description of the documents, we had the matrix S. This matrix is the most complete picture we can have of all the documents, there is no information lost since each word is its own topic and if we ignore the orderings of the words, we are able to perfectly recreate each document.

在我们最初对文档的完美描述中，我们有矩阵 S。这个矩阵是我们所能拥有的所有文档的最完整图像，没有信息丢失，因为每个单词都是它自己的主题，如果我们忽略单词顺序，就能够完美地重建每个文档。

However, some of the information is too fine grained. For example, we don’t really need a separate topic for “Robot” and “Android”. These can just be combined into a coarser topic of “Sci-Fi” or whatever you want to name it.

然而，有些信息过于细粒度。例如，我们并不真正需要为“Robot”和“Android”分别设立主题，它们可以合并为一个更粗粒度的主题，比如“科幻”，或者任何你想起的名字。

In this case, we can see that we have sacrificed some information. So if we were to recreate the document, there is no guarantee that we would get back the word “Robot”, since we only have information that a similar word from the same topic was mentioned.

在这种情况下，我们确实牺牲了一些信息。因此，如果我们要重建文档，并不能保证会得到“Robot”这个词，因为我们只知道来自同一主题的相似词被提及过。

This is what matrices M and N are doing in this case. The Latent dimension K is our hidden layer of topics (i.e. 10 topics). And given this dimension K, the LDA is learning the matrices M and N in an attempt to best recreate the matrix S.

这正是矩阵 M 和 N 在此所做的事情。潜在维度 K 是我们的隐藏主题层（例如10个主题）。给定维度 K，LDA 学习矩阵 M 和 N，以尽可能好地重建矩阵 S。

It is not essential you understand the maths here to know what is going on, the takeaway points are:

你并不需要完全理解其中的数学细节，关键在于以下几点：

LDA creates coarser grained topics based on the documents given to it.

LDA根据给出的文档创建更粗粒度的主题。

As the LDA model does this, we lose specific information about the individual documents.

当LDA模型这样做时，我们丢失了关于单个文档的特定信息。

If you think about how you would sort film reviews into 10 topics, this will hopefully start to make sense. Imagine if I asked you to summarise one of the topics you had created, you wouldn’t be able to recite every word of each document, but you’d probably be able to give a few of the most common words describing the overall topic. Hence, you have lost information.

如果你想一想如何将影评分成10个主题，这应该开始变得合理。想象一下，如果我让你总结你创建的主题之一，你可能无法背诵每篇文档的每个单词，但你可能能给出几个描述整体主题的最常见单词。因此，你丢失了信息。

Why do we want to lose information?

我们为什么要丢失信息？

Losing information may sound like a bad thing, but it actually helps Machine Learning models find patterns in data that they would have missed otherwise.

丢失信息听起来可能是坏事，但它实际上帮助机器学习模型找到数据中原本可能遗漏的模式。

The trick is to not lose the useful information, which in some cases can be learnt in an algorithm or it must be controlled via a hyperparameter (as for LDA with the number of topics).

关键在于不要丢失有用信息，有些情况下这可以在算法中学习，或者必须通过超参数来控制（如LDA中的主题数量）。

Let’s look at an example… say we have a noisy set of data points loosely describing a quadratic function (plot A).

让我们看一个例子……假设我们有一组嘈杂的数据点，大致描述一个二次函数（图A）。

In plot B, here we have tried to keep all the information we have to describe the data points, but does this look right? Probably not, our model is too specific and has not really caught the general trend.

在图B中，我们试图保留所有信息来描述数据点，但这样看起来对吗？可能不对，我们的模型过于具体，并没有真正捕捉到总体趋势。在图B中，我们试图保留所有信息来描述数据点，但这样看起来对吗？可能不对，我们的模型过于具体，并没有真正捕捉到总体趋势。

In plot C, we have lost too much information. The model is too simple for the data.

在图C中，我们丢失了太多信息，模型对数据来说过于简单。

In plot D, we have a good description of the data. We have found a good hyperparameter to be able to lose the less useful information and keep the important parts.

在图D中，我们对数据进行了良好的描述。我们找到了一个好的超参数，能够丢弃不太有用的信息，保留重要部分。

This may all sound familiar because it is exactly the description of underfitting and overfitting data, just in the context of NLP and topic modelling.

这一切可能听起来很熟悉，因为它正是对欠拟合和过拟合数据的描述，只不过是在自然语言处理和主题建模的语境下。

Back to Topics…

回到主题……

Hopefully that brief interlude was useful. If not, sorry about that but we’re back on track now!

希望那段简短的插曲有所帮助。如果没有，抱歉，但我们现在回到正轨！

So we have our LDA model which has sorted all these documents into a distribution of topics. Note that these are not hard clustered topics, they are distributions. So if 3 of the 10 topics we had were Sci-Fi, Documentary and Technology, we could have a film review for a RomCom between astronauts that would have a distribution of (0.2, 0.4, 0.3,…).

因此，我们有了LDA模型，它将所有文档按主题分布进行了分类。注意，这些不是硬聚类主题，而是分布。所以，如果我们的10个主题中有科幻、纪录片和科技，那么一篇关于宇航员之间的浪漫喜剧的影评可能会有 (0.2, 0.4, 0.3, …) 的分布。

On the diagram above, this corresponds to the top layer of connections between the documents and the topics.

在上图中，这对应着文档与主题之间的顶层连接。

These distributions are actually called Embeddings. Since we have embedded information about the document into a usable mathematical format.

这些分布实际上被称为嵌入（Embeddings），因为我们已经将文档中的信息嵌入到一种可用的数学格式中。

Take our embedding from earlier, a = (0.2, 0.4, 0.3,…). Now if we have two more documents with embeddings b = (0.1, 0.5, 0.2,…) and c = (0.8, 0, 0.1,…). Is b or c more similar to embedding a?

以上面嵌入 a = (0.2, 0.4, 0.3, …) 为例。如果我们还有另外两篇文档的嵌入 b = (0.1, 0.5, 0.2, …) 和 c = (0.8, 0, 0.1, …)，那么 b 和 c 哪个与 a 更相似？

There are a few ways to answer this question but we will choose the simple (and very effective) cosine similarity. Which you may remember from Maths courses at school or college. This basically calculates the distance between the two embeddings and if they are closer together, they are more similar.

有几种方法可以回答这个问题，但我们选择简单且非常有效的余弦相似度。你可能在学校或大学的数学课程中记得它。它基本上计算两个嵌入之间的距离，如果它们更接近，则更相似。

In this case a and b are more similar than a and c. So if we were to ask the computer which one out of b and c would it recommend given a. We would expect that it may recommend to us the document b.

在这种情况下，a 与 b 的相似度高于 a 与 c。因此，如果给定 a，我们问计算机推荐 b 还是 c，我们会期望它可能推荐文档 b。

This is how we can use topic modelling to create recommendations.

这就是我们如何利用主题建模来创建推荐的方法。

Round up

总结

So we have looked at what topics are, then at what the LDA algorithm gives us and then finally how we can use these mathematical objects to produce recommendations.

我们已了解了什么是主题，然后了解了LDA算法给我们提供了什么，最后了解了如何利用这些数学对象来生成推荐。

The key point for recommendations in topic modelling based on similarity is that we are assessing how similar the encoded information of each document is. As we have discussed, these recommendations may be completely terrible depending on how our topics are found and distributed (plot B or plot C from the crude explanation earlier). Or they might be fantastic and hit the sweet spot (plot D).

在基于相似性的主题建模推荐中，关键点在于我们评估每篇文档的编码信息有多相似。正如我们所讨论的，这些推荐可能完全糟糕，取决于主题的发现和分布方式（参见前面粗略解释中的图B或图C），也可能非常出色并达到最佳效果（图D）。

The other point to note is that although we can control the number of topics, we have less control on what these topics are. This is determined from the data, which we are able to manipulate by removing unimportant words for example. But if we don’t have a representative sample of Sci-Fi film reviews in our database then the likelihood is that this topic will not exist.

另一件要注意的事是，虽然我们可以控制主题数量，但我们对这些主题是什么的控制较少。这由数据决定，我们可以通过例如移除不重要的词来操纵数据。但如果我们数据库中没有科幻影评的代表性样本，那么这个主题很可能不存在。

Another reason why its all about the DATA…

这正是数据至关重要的另一个原因……

This is a really short and low level insight into how these types of algorithms can be used to give recommendations. There is loads of much more extensive descriptions out there so I encourage you to read around.

这只是一个非常简短且低层次的对这类算法如何用于提供推荐的介绍。外面有大量更详尽的描述，因此我鼓励你广泛阅读。

If you are interested, I have a working recommendation system code on my GitHub.

如果你感兴趣，我的GitHub上有一个可运行的推荐系统代码。

References

参考文献 参考文献

[1] — D. Blei, et. al., Latent Dirichlet Allocation (2003)

[1] — D. Blei, et. al., Latent Dirichlet Allocation (2003)