Examples

Basic Examples

Here is a random integer matrix:

In[1]:=

Out[3]=

Here are the NNMF factors:

In[4]:=

Out[5]=

Here is the matrix product of the obtained factors:

In[6]:=

Out[6]=

Note that element-wise the relative errors between the original matrix and reconstructed matrix are small:

In[7]:=

Out[7]=

Scope

Here is a random matrix with its first two columns having much larger magnitudes than the rest:

In[8]:=

Out[9]=

Here we compute NNMF factors:

In[10]:=

Here is the relative error of the approximation by the obtained matrix factorization:

In[11]:=

Out[11]=

Here is the relative error for the first three columns:

In[12]:=

Out[12]=

Here are comparison plots:

In[13]:=

Out[13]=

Options

“Normalization”

NNMF factors can be normalized in two ways: (i) the Euclidean norms of the columns of the left factor are all equal to 1, and (ii) the Euclidean norms of the rows of the right factor are all equal to 1.

Here is a table that shows NNMF factors for different normalization specifications:

In[14]:=

Out[14]=

“RegularizationParameter”

The implemented NNMF algorithm uses the Gradient Descent algorithm, the regularization parameter controls the “learning rate” and can have dramatic influence on the number of iterations steps and approximation precision.

Here we compute NNMF with different regularization multiplier parameters:

In[15]:=

Plot the results:

In[16]:=

Out[16]=

Here are the corresponding norms:

In[17]:=

Out[17]=

Applications

Topics extraction

One of the main motivations for developing NNMF algorithms is the easier interpretation of extracted topics in the framework of Latent Semantic Analysis. The following code illustrates the extraction of topics over a dataset of movie reviews.

Take a movie reviews dataset:

In[18]:=

Out[16]=

Change the labels “positive” and “negative” to have the prefix “tag”:

In[19]:=

Concatenate to each review the corresponding label and make an association:

In[20]:=

Out[21]=

Select movie reviews that are non-trivial:

In[22]:=

Out[7]=

Split each review into words and delete stop words:

In[23]:=

Convert the movie reviews association into a list of id-word pairs (“long form”):

In[24]:=

Out[13]=

Replace each word with its Porter stem:

In[25]:=

Out[25]=

Find the frequency of appearance of each unique word stems and pick word that appear in more than 30 reviews:

In[26]:=

Out[28]=

Filter the id-word pairs list to contain only word that are “popular enough”:

In[29]:=

Compute a id-word contingency matrix:

In[30]:=

Here is sample of the contingency matrix as dataset:

In[31]:=

Out[32]=

Visualize the contingency matrix:

In[33]:=

Out[34]=

Take the contingency sparse matrix:

In[35]:=

Here is a plot that shows the Pareto Principle adherence of the selected popular words:

In[36]:=

Out[36]=

Judging from the Pareto plot above we should apply the Inverse Document Frequency (IDF) formula:

In[37]:=

Normalize each row with Euclidean norm:

In[38]:=

In order to get computations faster we take a sample sub-matrix:

In[39]:=

Out[40]=

Apply NNMF in order to extract 24 topics using 12 maximum iteration steps and normalizing the right factor:

In[41]:=

Out[42]=

Show the extracted topics using the right factor (H):

In[43]:=

Out[43]=

Statistical thesaurus

Here we show statistical thesaurus entries for random words, selected words, and the labels (“tag:positive”,”tag:negative”):

In[44]:=

Out[20]=

Properties and Relations

NNMF should be compared with Singular Value Decomposition (SVD) and Independent Component Analysis (ICA).

Generate 3D points:

In[45]:=

Compute matrix factorizations using NNMF, SVD, and ICA and make a comparison plots grid (rotate the graphics boxes for better perception):

In[46]:=

Out[31]=

Possible Issues

Slow computations

Because of the Gradient Descent implementation through invocation of LinearSolve at every step for most larger data the computations are much slower than say the corresponding thing SingularValueDecomposition computations.

In most practical cases, though, NNMF does not have to be run for more than, say, 20 steps.

Local minima issues

NNMF is prone to “get stuck” in local minima. Hence, it is a good idea to run it several times in verify results interpretation. Different runs might involve different values for the regularization parameters.

Random initialization

NNMF uses random initialization of left factor. Because of that when multiple NNMF executions are done over the same data, in general the extracted bases would be different.

If an adequate number of topics and other execution parameters are used the bases differences should be small; the basis vectors are expected to be shuffled.