1. Covariance Geometry & Mahalanobis Distance

Euclidean distance fails in multivariate spaces because it treats all orthogonal axes with equal weight and zero correlation. The Mahalanobis Distance ($D_M$) scales deviations by the inverse covariance matrix $\boldsymbol{\Sigma}^{-1}$:

$$D_M(\mathbf{x}) = \sqrt{ (\mathbf{x} - \boldsymbol{\mu})^T \boldsymbol{\Sigma}^{-1} (\mathbf{x} - \boldsymbol{\mu}) }$$

Under multivariate normality $\mathbf{x} \sim \mathcal{N}_d(\boldsymbol{\mu}, \boldsymbol{\Sigma})$, the squared Mahalanobis distance follows a Chi-Squared distribution:

$$D_M^2(\mathbf{x}) \sim \chi_d^2$$

2. Multicollinearity & Variance Inflation Factor (VIF)

When regressing predictor $X_j$ against all other predictors $X_{-j}$, let the coefficient of determination be $R_j^2$. The Variance Inflation Factor (VIF) quantifies variance expansion due to collinearity:

$$\text{VIF}_j = \frac{1}{1 - R_j^2}$$$$\text{Var}(\hat{\beta}_j) = \sigma^2 \cdot \frac{\text{VIF}_j}{\sum_{i=1}^n (X_{ij} - \bar{X}_j)^2}$$

A threshold of $\text{VIF}_j > 10$ ($R_j^2 > 0.90$) signals pathological multicollinearity requiring regularization or feature removal.

3. Python Verification: Exact Mahalanobis Distance Outlier Scoring

import numpy as np
from scipy.stats import chi2

class MultivariateOutlierDetector:
    """Rigorous Mahalanobis distance outlier scorer with Chi-Squared p-value thresholding."""
    def __init__(self, data: np.ndarray):
        self.mu = np.mean(data, axis=0)
        self.cov = np.cov(data, rowvar=False)
        self.inv_cov = np.linalg.pinv(self.cov)
        self.dim = data.shape[1]

    def compute_distances(self, X: np.ndarray) -> np.ndarray:
        diff = X - self.mu
        # Compute quadratic form (x - mu)^T * Sigma^-1 * (x - mu)
        left = np.dot(diff, self.inv_cov)
        sq_dist = np.sum(left * diff, axis=1)
        return np.sqrt(sq_dist)

    def detect_outliers(self, X: np.ndarray, alpha: float = 0.01) -> np.ndarray:
        distances = self.compute_distances(X)
        threshold_sq = chi2.ppf(1.0 - alpha, df=self.dim)
        return distances ** 2 > threshold_sq

if __name__ == "__main__":
    np.random.seed(42)
    # 2D correlated bivariate normal sample
    mean = [0, 0]
    cov = [[2.0, 1.6], [1.6, 2.0]]
    X_normal = np.random.multivariate_normal(mean, cov, size=500)
    
    detector = MultivariateOutlierDetector(X_normal)
    # Test point at (-3, 3) -> Euclidean distance is moderate, but against correlation it's an extreme outlier!
    test_pts = np.array([[0.0, 0.0], [2.0, 2.0], [-3.0, 3.0]])
    dists = detector.compute_distances(test_pts)
    outliers = detector.detect_outliers(test_pts, alpha=0.001)
    
    for pt, d, out in zip(test_pts, dists, outliers):
        print(f"Point {pt} -> Mahalanobis Distance: {d:.2f} | Outlier Flag (alpha=0.001): {out}")