Mahout : PearsonCorrelationSimilarity : Compute item similarity

It is an implementation of Pearson correlation. For users X and Y, the following values are calculated:

    sumX2: sum of the square of all X's preference values

    sumY2: sum of the square of all Y's preference values

    sumXY: sum of the product of X and Y's preference value for all items for which both X and Y express a preference

The correlation is then:

sumXY / sqrt(sumX2 * sumY2)

PearsonCorrelationSimilarity returns NaN, if similarity is unknown. Let’s say I had following input data.

customer.csv

1,4,3
1,7,2
1,8,2
1,10,1
2,3,2
2,4,3
2,6,3
2,7,1
2,9,1
3,0,3
3,3,2
3,4,1
3,8,3
3,9,1
4,2,5
4,3,4
4,7,3
4,9,2
5,4,5
5,6,4
5,7,1
5,8,3

1,4,3 means customer 1 like item 4 and rated it 3

Following application computes PearsonCorrelationSimilarity between two items.

import java.io.File;
import java.io.IOException;

import org.apache.mahout.cf.taste.common.TasteException;
import org.apache.mahout.cf.taste.impl.model.file.FileDataModel;
import org.apache.mahout.cf.taste.impl.similarity.PearsonCorrelationSimilarity;
import org.apache.mahout.cf.taste.model.DataModel;

public class PearsonCorrelationSimilarityEx {
 public static String dataFile = "/Users/harikrishna_gurram/customer.csv";

 public static void main(String args[]) throws IOException, TasteException {

  DataModel model = new FileDataModel(new File(dataFile));

  PearsonCorrelationSimilarity similarity = new PearsonCorrelationSimilarity(
    model);

  long itemIds[] = { 3, 4, 6, 7, 8, 9, 10 };

  double distance[] = similarity.itemSimilarities(4, itemIds);

  for (int i = 0; i < itemIds.length; i++) {
   System.out.println("distance between item 4 and " + itemIds[i]
     + " is " + distance[i]);
  }

 }
}

Output

distance between item 4 and 3 is NaN
distance between item 4 and 4 is 0.9999999999999998
distance between item 4 and 6 is 0.9999999999999998
distance between item 4 and 7 is -0.49999999999999895
distance between item 4 and 8 is 0.0
distance between item 4 and 9 is NaN
distance between item 4 and 10 is NaN

Prevoius                                                 Next                                                 Home