A Seção 3.5.2 em *Os elementos do aprendizado estatístico* é útil porque coloca a regressão PLS no contexto correto (de outros métodos de regularização), mas é realmente muito breve e deixa algumas declarações importantes como exercícios. Além disso, considera apenas um caso de uma variável dependente univariada .y$\mathbf{y}$

A literatura sobre PLS é vasta, mas pode ser bastante confusa porque existem muitos "sabores" diferentes de PLS: versões univariadas com um único DV (PLS1) e versões multivariadas com vários DVs Y (PLS2), versões simétricas tratando X e Y versões iguais e assimétricas ("regressão PLS") tratando X como variáveis independentes e Y como variáveis dependentes, versões que permitem uma solução global via SVD e versões que exigem deflações iterativas para produzir cada próximo par de direções PLS, etc. etc.y$\mathbf{y}$Y$\mathbf{Y}$X$\mathbf{X}$Y$\mathbf{Y}$X$\mathbf{X}$Y$\mathbf{Y}$

Tudo isso foi desenvolvido no campo da quimiometria e permanece um pouco desconectado da literatura estatística ou de aprendizado de máquina "convencional".

The overview paper that I find most useful (and that contains many further references) is:

For a more theoretical discussion I can further recommend:

### A short primer on PLS regression with univariate y$y$ (aka PLS1, aka SIMPLS)

β$\beta $y=Xβ+ϵ$y=X\beta +\u03f5$β=(X⊤X)−1X⊤y$\beta =({\mathbf{X}}^{\mathrm{\top}}\mathbf{X}{)}^{-1}{\mathbf{X}}^{\mathrm{\top}}\mathbf{y}$β$\beta $Xβ$\mathbf{X}\beta $ with y$\mathbf{y}$. If there is a lot of predictors, then it is always possible to find some linear combination that happens to have a high correlation with y$\mathbf{y}$. This will be a spurious correlation, and such β$\beta $ will usually point in a direction explaining very little variance in X$\mathbf{X}$. Directions explaining very little variance are often very "noisy" directions. If so, then even though on training data OLS solution performs great, on testing data it will perform much worse.

In order to prevent overfitting, one uses regularization methods that essentially force β$\beta $ to point into directions of high variance in X$\mathbf{X}$ (this is also called "shrinkage" of β$\beta $; see Why does shrinkage work?). One such method is principal component regression (PCR) that simply discards all low-variance directions. Another (better) method is ridge regression that smoothly penalizes low-variance directions. Yet another method is PLS1.

PLS1 replaces the OLS goal of finding β$\beta $ that maximizes correlation corr(Xβ,y)$\mathrm{corr}(\mathbf{X}\beta ,\mathbf{y})$ with an alternative goal of finding β$\beta $ with length ∥β∥=1$\Vert \beta \Vert =1$ maximizing covariance

cov(Xβ,y)∼corr(Xβ,y)⋅var(Xβ)−−−−−−−√,$$\mathrm{cov}(\mathbf{X}\beta ,\mathbf{y})\sim \mathrm{corr}(\mathbf{X}\beta ,\mathbf{y})\cdot \sqrt{\mathrm{var}(\mathbf{X}\beta )},$$

which again effectively penalizes directions of low variance.

Finding such β$\beta $ (let's call it β1${\beta}_{1}$) yields the first PLS component z1=Xβ1${\mathbf{z}}_{1}=\mathbf{X}{\beta}_{1}$. One can further look for the second (and then third, etc.) PLS component that has the highest possible covariance with y$\mathbf{y}$ under the constraint of being uncorrelated with all the previous components. This has to be solved iteratively, as there is no closed-form solution for all components (the direction of the first component β1${\beta}_{1}$ is simply given by X⊤y${\mathbf{X}}^{\mathrm{\top}}\mathbf{y}$ normalized to unit length). When the desired number of components is extracted, PLS regression discards the original predictors and uses PLS components as new predictors; this yields some linear combination of them βz${\beta}_{z}$ that can be combined with all βi${\beta}_{i}$ to form the final βPLS${\beta}_{\mathrm{P}\mathrm{L}\mathrm{S}}$.

Note that:

- If all PLS1 components are used, then PLS will be equivalent to OLS. So the number of components serves as a regularization parameter: the lower the number, the stronger the regularization.
- If the predictors X$\mathbf{X}$ are uncorrelated and all have the same variance (i.e. X$\mathbf{X}$ has been
*whitened*), then there is only one PLS1 component and it is equivalent to OLS.
- Weight vectors βi${\beta}_{i}$ and βj${\beta}_{j}$ for i≠j$i\ne j$ are not going to be orthogonal, but will yield uncorrelated components zi=Xβi${\mathbf{z}}_{i}=\mathbf{X}{\beta}_{i}$ and zj=Xβj${\mathbf{z}}_{j}=\mathbf{X}{\beta}_{j}$.

**All that being said, I am not aware of ***any* practical advantages of PLS1 regression over ridge regression (while the latter does have lots of advantages: it is continuous and not discrete, has analytical solution, is much more standard, allows kernel extensions and analytical formulas for leave-one-out cross-validation errors, etc. etc.).

Quoting from Frank & Friedman:

RR, PCR, and PLS are seen in Section 3 to operate in a similar fashion. Their principal goal is to shrink the solution coefficient vector away from the OLS solution toward directions in the predictor-variable space of
larger sample spread. PCR and PLS are seen to shrink more heavily away
from the low spread directions than RR, which provides the optimal shrinkage (among linear estimators) for an equidirection prior. Thus
PCR and PLS make the assumption that the truth is likely to have particular preferential alignments with the high spread directions of the
predictor-variable (sample) distribution. A somewhat surprising result
is that PLS (in addition) places increased probability mass on the true
coefficient vector aligning with the K$K$th principal component direction,
where K$K$ is the number of PLS components used, in fact expanding the
OLS solution in that direction.

They also conduct an extensive simulation study and conclude (emphasis mine):

For the situations covered by this simulation study, one can conclude
that all of the biased methods (RR, PCR, PLS, and VSS) provide
substantial improvement over OLS. [...] **In all situations, RR dominated
all of the other methods studied.** PLS usually did almost as well as RR
and usually outperformed PCR, but not by very much.

**Update:** In the comments @cbeleites (who works in chemometrics) suggests two possible advantages of PLS over RR:

An analyst can have an *a priori* guess as to how many latent components should be present in the data; this will effectively allow to set a regularization strength without doing cross-validation (and there might not be enough data to do a reliable CV). Such an *a priori* choice of λ$\lambda $ might be more problematic in RR.

RR yields one single linear combination βRR${\beta}_{\mathrm{R}\mathrm{R}}$ as an optimal solution. In contrast PLS with e.g. five components yields five linear combinations βi${\beta}_{i}$ that are then combined to predict y$y$. Original variables that are strongly inter-correlated are likely to be combined into a single PLS component (because combining them together will increase the explained variance term). So it might be possible *to interpret* the individual PLS components as some real latent factors driving y$y$. The claim is that it is easier to interpret β1,β2,${\beta}_{1},{\beta}_{2},$ etc., as opposed to the joint βPLS${\beta}_{\mathrm{P}\mathrm{L}\mathrm{S}}$. Compare this with PCR where one can also see as an advantage that individual principal components can potentially be interpreted and assigned some qualitative meaning.