Sufficient Dimension Reduction
Broadly viewed, dimension reduction has always been a central statistical concept. In the second half of the nineteenth century 'reduction of observations' was widely viewed as a core goal of statistical methodology, and in 1922 Fisher formulated the concept of a sufficient statistic as means of reduction without loss of information.
In recent years "sufficient dimension reduction, SDR" has been used to denote a body of new ideas and methods for dimension reduction. Like Fisher's classical notion of a sufficient statistic, SDR strives for reduction without loss of information. But unlike sufficient statistics, sufficient reductions may contain unknown parameters and thus need to be estimated. In the context of regression, a reduction R(X) of the p-dimensional predictor X is sufficient if the conditional distribution of the response Y given X is the same as the distribution of Y given R(X). See Cook (2007, Sections 8.2 and 8.3) and Adragni and Cook (2009) for a more formal definition and overviews of SDR in regression.
The phrase "sufficient dimension reduction" with its modern meaning was introduced in the late 1990's (Cook 1998a; Cook and Yin, 1999, JASA, p. 1187-1200) in the context of regression graphics, where the overarching goal is to find a "sufficient summary plot" of Y versus R(X) that retains all of the relevant regression information. Since that time the phrase and the associated ideas have been used with increasing frequency in the statistics literature, with ever more ambitious goals. A similar phrase "sufficient dimensionality reduction" was introduced in the machine learning literature by Globerson and Tishby (2003), but the underlying ideas are quite different from those associated with SDR and the methodology is not dimension reduction in the usual sense (Burges, C.J.C. (2009). Dimension reduction: A guided tour. Foundations and Trends in Machine Learning 4, 275--365.)
There are both model-based and model-free SDR methods. In the model-based approach the form of the reduction R(X) can in principle be determined from the model itself. Reductions are typically constrained to be linear in the model-free approach and then the goal is to estimate the "central subspace" (Cook 1994, 1996, 1998b), defined as the intersection of all subspaces S with the property that Y is independent of X given the projection of X onto S.
Last updated: March 21, 2011.