What does Completing Preferences from Sparse Purchases conclude, in numbers, about “Alternating Least Squares: Breaking a Hard Problem into a Chain of Small Systems”?
Answer: There are 343 observed cells. With a ridge parameter of 0.001 the training error is 0.3886 but the test error is as high as 4.2879: with almost no penalty, customers and products with few records are fitted into the noise, and predictions drift far off on the unobserved cells. This row's training error is even higher than the 0.3361 obtained with 0.100, a sign that without a penalty alternating least squares does not converge cleanly either. The test error is lowest, 0.7649, at 1.000; beyond that the penalty overwhelms the data, training and test errors rise together, and at 30.000 they stand at 1.7089 and 1.7581, a model that has learned almost nothing.
Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (23 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.001, 0.100, 0.3361, 0.3886, 0.7649, 1.000
| observed 343 cells, noise sd 0.5, rank 3 | ||
| lambda | train RMSE | test RMSE |
| 0.001 | 0.3886 | 4.2879 |
| 0.100 | 0.3361 | 1.2052 |
| 1.000 | 0.3986 | 0.7649 |
| 3.000 | 0.6428 | 1.0035 |
| 10.000 | 1.3645 | 1.5411 |
| 30.000 | 1.7089 | 1.7581 |
Show the full program
# Chapter 2. p22: the ridge penalty with noisy scores.
# Model: 40 customers x 30 products, true rank 3 (Gaussian factors),
# each observed score carries Gaussian noise with sd 0.5; 30 percent
# of cells observed (illustrative). ALS at rank 3 with ridge lambda;
# train RMSE is on observed noisy scores, test RMSE compares the
# predictions on unobserved cells with the noise-free truth.
import math
import random
def dot(a, b):
return math.fsum(x * y for x, y in zip(a, b))
def solve(A, b):
# Gaussian elimination with partial pivoting for a small system
n = len(b)
M = [list(A[i]) + [b[i]] for i in range(n)]
for k in range(n):
p = max(range(k, n), key=lambda i: abs(M[i][k]))
M[k], M[p] = M[p], M[k]
for i in range(k + 1, n):
f = M[i][k] / M[k][k]
M[i] = [M[i][j] - f * M[k][j] for j in range(n + 1)]
x = [0.0] * n
for i in range(n - 1, -1, -1):
s = math.fsum(M[i][j] * x[j] for j in range(i + 1, n))
x[i] = (M[i][n] - s) / M[i][i]
return x
def update(trip, F, r, lam, prior=None):
# weighted ridge step for one row: minimise
# sum w (y - f.x)^2 + lam |x - prior|^2 over x
pr = prior if prior is not None else [0.0] * r
A = [[math.fsum(w * F[j][a] * F[j][b] for j, _, w in trip)
+ (lam if a == b else 0.0) for b in range(r)]
for a in range(r)]
y = [math.fsum(w * F[j][a] * v for j, v, w in trip) + lam * pr[a]
for a in range(r)]
return solve(A, y)
def als(nu, ni, obs, r, lam, iters, seed, V=None, trace=None):
# alternating least squares on (u, i, value, weight) tuples
rng = random.Random(seed)
if V is None:
V = [[rng.gauss(0.0, 1.0) for _ in range(r)] for _ in range(ni)]
bu = [[] for _ in range(nu)]
bi = [[] for _ in range(ni)]
for u, i, x, w in obs:
bu[u].append((i, x, w))
bi[i].append((u, x, w))
U = None
for t in range(iters):
U = [update(bu[u], V, r, lam) for u in range(nu)]
V = [update(bi[i], U, r, lam) for i in range(ni)]
if trace is not None:
trace.append((t + 1, U, V))
return U, V
def factors(n, r, rng):
return [[rng.gauss(0.0, 1.0) for _ in range(r)] for _ in range(n)]
def product(A, B):
return [[dot(a, b) for b in B] for a in A]
def mask(nu, ni, p, rng):
return [(u, i) for u in range(nu) for i in range(ni)
if rng.random() < p]
def relerr(M, U, V, cells):
num = math.fsum((dot(U[u], V[i]) - M[u][i]) ** 2 for u, i in cells)
den = math.fsum(M[u][i] ** 2 for u, i in cells)
return math.sqrt(num / den)
def rmse(M, U, V, cells):
num = math.fsum((dot(U[u], V[i]) - M[u][i]) ** 2 for u, i in cells)
return math.sqrt(num / len(cells))
rng = random.Random(20261113)
nu, ni, r = 40, 30, 3
M = product(factors(nu, r, rng), factors(ni, r, rng))
seen = mask(nu, ni, 0.3, rng)
ss = set(seen)
rest = [(u, i) for u in range(nu) for i in range(ni) if (u, i) not in ss]
noisy = {c: M[c[0]][c[1]] + rng.gauss(0.0, 0.5) for c in seen}
obs = [(u, i, noisy[(u, i)], 1.0) for u, i in seen]
print("observed {} cells, noise sd 0.5, rank 3".format(len(seen)))
print(" lambda train RMSE test RMSE")
for lam in (0.001, 0.1, 1.0, 3.0, 10.0, 30.0):
U, V = als(nu, ni, obs, r, lam, 60, 3)
tr = math.sqrt(math.fsum((x - dot(U[u], V[i])) ** 2
for u, i, x, _ in obs) / len(obs))
print(" {:7.3f} {:.4f} {:.4f}".format(
lam, tr, rmse(M, U, V, rest)))Single source and citation
Fan, Y. (2026). Low-Rank Matrix Completion and Alternating Least Squares: Inferring Unstated Preferences from Sparse Purchase Records (Version 1.0.0), Chapter 2: Alternating Least Squares: Breaking a Hard Problem into a Chain of Small Systems. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-328 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026328,
author = {Fan, Yafu},
title = {Low-Rank Matrix Completion and Alternating Least Squares: Inferring Unstated Preferences from Sparse Purchase Records},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Sparse Yet Completable: How Many Entries a Low-Rank Matrix Needs
- Chapter 3 · Power Iteration and the Leading Singular Vector: A Good Start for Alternating Least Squares
- Chapter 4 · Incoherence: How Best-Sellers, Small Niches and Long Tails Defeat Completion
- Chapter 5 · Missing Not at Random: How Far Seeing Only Purchases Leads Inference Astray
- Chapter 6 · Cold Start: Catching New Customers and New Products with Means and Category Priors
- Chapter 7 · Eastern and Western Sources: "Things of a Kind Gather Together" and Spearman's General Factor
- Chapter 8 · The Buyer's Recommendation Sheet and a Plan to Refute the Book: A Loop on the Shop's Own Data