Top-N Accuracy Metrics

The lenskit.metrics.topn module contains metrics for evaluating top-N recommendation lists.

Classification Metrics

These metrics treat the recommendation list as a classification of relevant items.

lenskit.metrics.topn.precision(recs, relevant)

Compute the precision of a set of recommendations.

Parameters:
  • recs (array-like) – a sequence of recommended items
  • relevant (set-like) – the set of relevant items
Returns:

the fraction of recommended items that are relevant
Return type:

double
lenskit.metrics.topn.recall(recs, relevant)

Compute the recall of a set of recommendations.

Parameters:
  • recs (array-like) – a sequence of recommended items
  • relevant (set-like) – the set of relevant items
Returns:

the fraction of relevant items that were recommended.
Return type:

double

Ranked List Metrics

These metrics treat the recommendation list as a ranked list of items that may or may not be relevant.

lenskit.metrics.topn.recip_rank(recs, relevant)

Compute the reciprocal rank of the first relevant item in a recommendation list. This is used to compute MRR.

Parameters:
  • recs (array-like) – a sequence of recommended items
  • relevant (set-like) – the set of relevant items
Returns:

the reciprocal rank of the first relevant item.
Return type:

double

Utility Metrics

The nDCG function estimates a utility score for a ranked list of recommendations.

lenskit.metrics.topn.ndcg(scores, items=None, discount=<ufunc 'log2'>)

Compute the Normalized Discounted Cumulative Gain of a series of scores. These should be relevance scores; they can be \({0,1}\) for binary relevance data.

Discounted cumultative gain is computed as:

\[\begin{split}\begin{align*} \mathrm{DCG}(L,u) & = \sum_{i=1}^{|L|} \frac{r_{ui}}{d(i)} & \\ \mathrm{nDCG}(L, u) & = \frac{\mathrm{DCG}(L,u)}{\mathrm{DCG}(L_{\mathrm{ideal}}, u)} \end{align*}\end{split}\]
Parameters:
  • scores (pd.Series or array-like) – relevance scores for items. If items is None, these should be in order of recommendation; if items is not None, then this must be a pandas.Series indexed by item ID.
  • items (array-like) – the list of item IDs, if the item list and score list is to be provided separately.
  • discount (ufunc) – the rank discount function. Each item’s score will be divided the discount of its rank, if the discount is greater than 1.
Returns:

the nDCG of the scored items.
Return type:

double