Talk:Ridge regression

Merging that way is not advised, they are overlapping methods but in different theoretical frameworks, and the Bayesian is more general. -Anon

Fold into Tikhanov regression?

Ridge regression is Tikhanov regression. And that areticle is much more detailed. I propose we merge the two articles, basically delete this one and point to the other David (talk) 01:42, 22 January 2022 (UTC)

Yeah, it definitely should be; this article is also worse than the one on Tikhonov regularization. For instance, this article claims that ridge regression is used to deal with multicollinearity, when in reality there is no need for the variables to be correlated. Ridge regression just regularizes coefficients towards 0 to improve out-of-sample performance. Closed Limelike Curves (talk) 00:49, 15 May 2022 (UTC)[reply]

I think this article is about something different. Tikhonov or Tikhonov-Philips regularization is really a method of solving an operator equation where the inverse is not bounded in sensible norms. This is the way ``regularization is understood in inverse problems. Of course when a finite dimensional approximation is used it coincides with ridge regression, but I think conceptually quite different. The current article will be very confusing to anyone who was trying to look up Tikhov regularization. Also a more modern approach, even in infinite dimensions, is to see regularization as a MAP estimate, in the Bayesian framework. Of course at this point the worlds of statistical regression and functional analysis start to merge a bit. I think overall the reader would be better served with an article that starts with functional analysis and classical examples of inverting compact operators and Tikhonov regularization, and Philips regularization. Then perhaps both articles should point to each other as a different point of view.Billlion (talk) 11:30, 20 July 2026 (UTC)

Explain the name

The article left me confused about the name of the regression. Was the name originally 'RIDGE' and was thus originally an abbreviation? Did it later change to 'Ridge', maybe due to the abbreviation being clumsy in hindsight? Or is this referring to something within the mathematical method?

bilderbikkel 11:18, 6 May 2022 (UTC)

Applications

The method (Tikhonov regularization) is applied in several fields of applied physics like,

• Geophysics (seismic studies for example)
• Atmospheric inverse problem, in general it is an important method in remote sensing
• Medicine in studies using functional magnetic resonance imaging.
• Deep learning regularization techniques are also using this method (for genomic data for instance)

I would suggest to add this kind of information to the page. AyubuZimbale (talk) 07:31, 4 November 2024 (UTC)

"Most real-world phenomena have the effect of low-pass filters[clarification needed] in the forward direction where A {\displaystyle A} maps x {\displaystyle \mathbf {x} } to b {\displaystyle \mathbf {b} }"

There is a classification needed tag. Basically, this says that if we take an x with "reasonable" values, and add noise to with mean zero but possibly "unreasonable" (large) amplitude (i.e. SD of the noise), then the matrix A will act to make most of these additional noise deviations cancel one another.

More specifically, if we think of x as a time series, that is to say, a process in continuous time that has been sampled at discrete points in time, then any white noise superimposed on the underlying "actual" signal will cancel out, and this effect will be strongest for the "high-frequency" component.

The trouble with the passage is that "low-pass filter" was added here by some author (not me) who felt this analogy would aid intuition. If clarification is needed, then it just misses its mark and the reader had best skip it altogether. 145.53.11.225 (talk) 11:38, 2 July 2025 (UTC)

In other cases, high-pass operators (e.g., a difference operator or a weighted Fourier operator) may be used to enforce smoothness if the underlying vector is believed to be mostly continuous

- this passage continues the above idea and, if anything, more clarification is needed here, because this talk of being "mostly continuous" only makes sense if we think of x as being a discrete sampling of some underlying smooth signal, such as an acoustic signal. In that context, this remark makes perfect sense - but it probably looks quite odd if you came here from a statistical linear regression problem.

(Indeed, all this mathematically speaking very loose talk seems to indicate that a signal processing engineer has been at this page. But no matter.) 145.53.11.225 (talk) 11:45, 2 July 2025 (UTC)

Generalized Tikhonov regularization

the alternate solution at the end stipulates "Q is not a null matrix"; however, both solutions are equal and equivalent to the standard least squares solution in this case. I propose to remove that condition. Chris2crawford (talk) 12:29, 23 August 2025 (UTC)

Error in the introduction

The overview states that the regularization is equivalent to the 'constraint' beta^T beta = c. This has two problems: 1) as shown in the section "Tikhonov regularization", the value of c is actually 0 in this case (lambda = alpha^2). Not even the generalized problem 'constrains' beta to the surface of a sphere, but to the point beta_0, ie. minimizing |beta-beta_0| 2) it is not a constaint in the technical sense of the word but an additional term in the objective function pulling beta towards \vec 0. Chris2crawford (talk) 12:59, 23 August 2025 (UTC)

Disjoint transition between Ridge and Tikhonov sections

There is an abrupt change in notation from y=X beta to b = A x, which unfortunately permutes two of the three variable letters! Thus, although the two articles are the same concept, the current merge confounds instead of unifies this connection.

While the math in the Tikhonov section is more mathematically correct than the Ridge section, its introduction is sloppy and incorrect. The probelm: "However, if no x satisfies the equation" was already solved by the previous sentence: "The standard approach is ordinary least squares..", which itself is not solving the problem "Ax=b" but minimizing |Ax-b|^2. So the problem of overdetermination is already solved by least squares, and only underdetermination is being solved by Tikhonov regularization, but both are conflated in the introduction. Chris2crawford (talk) 13:24, 23 August 2025 (UTC)

Bad choice of symbols

Tikhonov regularization: $\mathbf {b}$ and ${\mathcal {b}}$ are too similar. Instead of denoting the augmented versions of $(A,\mathbf {b} )$ with $({\mathcal {A}},{\mathcal {b}})$ I suggest $({\tilde {A}},{\tilde {\mathbf {b} }})$. Also I would note that ${\mathcal {b}}$, written \mathcal{b} (as in the article), is rendered the same as $b$, written b. HomoVafer (talk) 09:07, 2 September 2025 (UTC)

Suggested Merging of History subsection into the introduction

The History subsection contains information already mentioned in the introduction. Perhaps the History section should be removed and any useful information there incorporated into the introduction, which also talks about the history of the methodology. KlimaxMethod (talk) 08:49, 8 January 2026 (UTC)