Chapter 19 Dynamic Bandwidth Interpretation

Chapter 18 introduced recursive mean-split estimation as a nonparametric method built from local, benchmark-relative partitions. Its main theoretical conclusion was stronger than a claim about partitions chosen independently of the responses: arithmetic cell means can be consistent even when the observations, including their responses, help select the partition, provided the selected predictor-space neighborhoods localize, their local sample sizes grow, and the admissible partition class has controlled complexity. Chapter 18 also distinguished that cell-mean result from the additional interpolation and neighbor-weighting operations used by NNS.reg.

The next question is geometric: what controls the scale over which the estimator averages observations?

In conventional kernel regression, the answer is a bandwidth. In NNS, the answer begins with the recursively constructed predictor cell. Each split at a local center changes both the region represented by a regression point and the number of observations contributing to its response coordinate. The resulting local scale is dynamic: it depends on the realized predictor distribution, the requested partition order, the stopping controls, and, when order is selected from directional dependence, potentially on the responses as well.

This chapter develops two related bandwidth concepts:

  1. Partition bandwidth: the geometric size of the predictor region represented by a local mean.
  2. Prediction bandwidth: the predictor neighborhood that actually contributes to a fitted value after interpolation or regression-point weighting.

The two are identical for a simple cell-mean estimator, but need not coincide for the complete NNS.reg predictor. Keeping them separate gives an exact connection to consistency theory, the original NNS motivation, and the finite-sample bias-variance tradeoff.

The central lesson is not that NNS dispenses with smoothing. Rather, NNS makes smoothing implicit in its data-driven geometry. Increasing resolution without limit can preserve observation noise; keeping resolution bounded can preserve approximation bias. A consistency-compatible NNS sequence refines the geometry while retaining increasingly reliable local estimates.


19.1 Bandwidth in Classical Nonparametric Estimation

Suppose independent observations obey

\[ Y_i=f(X_i)+\varepsilon_i, \qquad E[\varepsilon_i\mid X_i]=0, \]

where the regression target is

\[ f(x)=E[Y\mid X=x]. \]

The Nadaraya-Watson kernel estimator is

\[ \hat f_h(x)= \frac{\sum_{i=1}^n K\!\left((x-X_i)/h\right)Y_i} {\sum_{i=1}^n K\!\left((x-X_i)/h\right)}, \]

when the denominator is positive. The kernel determines relative weighting and the bandwidth \(h>0\) determines how far the weights reach. A small bandwidth emphasizes nearby observations and gives a flexible, potentially noisy estimate; a large bandwidth averages more observations, usually reducing sampling variation at the cost of coarser approximation.

Histogram and partition estimators use a similar principle, but select observations by membership in cells. Local polynomial methods additionally fit a low-order polynomial to the weighted neighborhood. Splines can control smoothness through basis resolution and/or a smoothing penalty. Not every classical nonparametric method uses one global numeric bandwidth, and adaptive or variable-bandwidth kernels are also possible.

The shared problem is the balance between localization error and sampling error. Under standard smoothness and design assumptions, a small neighborhood reduces the former, while sufficient observations in that neighborhood reduce the latter. A bandwidth must therefore be interpreted together with the number and arrangement of observations it retains.

For a fixed-dimensional predictor, the familiar heuristic is

\[ h_n\longrightarrow0, \qquad nh_n^d\longrightarrow\infty, \]

where \(d\) is dimension and the second condition assumes a locally regular predictor density and a neighborhood volume of order \(h_n^d\). It is a heuristic, not a theorem for every partition rule or distribution. The NNS analogue replaces the prespecified scale \(h_n\) with the observed scale of an adaptive cell and replaces \(nh_n^d\) with its realized occupancy.


19.2 From Partial-Moment Splits to Local Neighborhoods

19.2.1 The mean benchmark

Let \(R\) be a region occupied by observations with indices \(I_R\). With arithmetic-mean reduction, define

\[ \bar X_R=\frac{1}{|I_R|}\sum_{i\in I_R}X_i, \qquad \bar Y_R=\frac{1}{|I_R|}\sum_{i\in I_R}Y_i. \]

For one predictor, the benchmark \(\bar X_R\) has the first-order partial-moment balance

\[ E_n[(X-\bar X_R)_+\mid R] = E_n[(\bar X_R-X)_+\mid R]. \]

Thus a mean split is not an arbitrary geometric cut. It is located at the local point where upper and lower first-order deviations balance. Repeating the split within occupied daughter regions defines a nested, benchmark-relative partition. This is the directional interpretation developed throughout the NNS framework and illustrated in Chapter 18.

In NNS.part(..., type = "XONLY", noise.reduction = "mean"), the region is subdivided using the predictor center alone. In joint NNS.part mode, both predictor and response centers define four partial-moment quadrants; the resulting joint cells describe observed \((X,Y)\) geometry rather than intervals identifiable from a new \(x\) without its unknown response.

The current univariate numeric-order NNS.reg implementation constructs predictor-only regression points. The regression-point response coordinates are still estimated from observed responses, and order = NULL chooses a bounded order using dependence information. Consequently, even when the split axis is only \(X\), selection of the depth can be response-adaptive. Chapter 18 explains why this is not, by itself, a contradiction of consistency.

19.2.2 The cell-mean building block

Let \(A_n(x)\) denote a predictor-space cell containing a query \(x\). Its occupancy and local arithmetic response mean are

\[ N_n(x)=\sum_{i=1}^n\mathbf1\{X_i\in A_n(x)\}, \qquad \hat f_n^{\mathrm{cell}}(x) =\frac{1}{N_n(x)}\sum_{i:X_i\in A_n(x)}Y_i, \]

for a nonempty cell. The estimator has the exact weight representation

\[ \hat f_n^{\mathrm{cell}}(x) =\sum_{i=1}^n w_{ni}^{\mathrm{cell}}(x)Y_i, \qquad w_{ni}^{\mathrm{cell}}(x) =\frac{\mathbf1\{X_i\in A_n(x)\}}{N_n(x)}. \]

These weights are uniform inside the cell and zero outside. Unlike a symmetric kernel window, the cell need not be centered at \(x\) or have the same width on either side. Its shape is created by the recursive means, and its occupancy reflects the sample’s local predictor mass.

This formula describes the cell-mean estimator, not every value returned by NNS.reg. The package can connect regression-point means by straight segments, smooth them using a spline when requested, and combine multivariate RPM rows through distance-based weighting. We will describe those additional operations separately.


19.3 The Stochastic Partition Bandwidth

19.3.1 Geometric cell diameter

For a bounded interval or rectangular predictor cell, a natural bandwidth analogue is

\[ h_n^{\mathrm{part}}(x)=\operatorname{diam}(A_n(x)), \]

with diameter measured in the relevant predictor norm. In one dimension, the diameter is simply the width of the interval. For an axis-aligned rectangle in \(d\) dimensions, the Euclidean diameter is

\[ h_n^{\mathrm{part}}(x) =\left(\sum_{j=1}^d h_{nj}(x)^2\right)^{1/2}, \]

where \(h_{nj}(x)\) is its width along coordinate \(j\).

The realized widths are:

  • stochastic, because the cuts use sample-dependent centers;
  • location-specific, because different cells can have different widths;
  • endogenous to the estimator, because order, occupancy, reducer and any selected tuning rule determine where the cuts occur.

A fixed order does not generally impose a common cell width. Two cells at the same depth may have different sizes and very different occupancies. This makes recursive mean splitting unlike a uniform, prespecified grid, though both are partition estimators.

19.3.2 The support and boundary qualification

There is an important distinction between a theoretical cell and its observed data span. If the first split occurs at \(c\), the raw cells \(( -\infty,c]\) and \((c,\infty)\) have infinite diameter when regarded as subsets of the full real line. That is not a meaningful local bandwidth for a bounded-support regression problem.

On a compact design domain \(S\), use the support-restricted cell \(A_n(x)\cap S\) when defining geometric diameter. Alternatively, at a fixed query define the observed predictor reach

\[ r_n^{\mathrm{cell}}(x) =\max_{i:X_i\in A_n(x)}\|X_i-x\|, \]

for nonempty cells. This quantity measures the maximum observed distance from the query to observations used by the cell mean; it can be defined even if the formal outer cell is unbounded. A shrinking theoretical diameter implies shrinking observed reach, but the converse need not hold without assumptions on the predictor support and sample coverage.

The observed within-cell span, for example \(\max_{i\in A}X_i-\min_{i\in A}X_i\), is a useful diagnostic, but is not necessarily the mathematical diameter of the interval assigned by the partition. The distinction is especially relevant near the boundary of the design domain.

19.3.3 Adaptive to what?

At a fixed order, predictor-only arithmetic-mean splits respond to the distribution of \(X\). If the response changes while the predictor sample remains the same, those particular split boundaries do not change. They are therefore density- or geometry-adaptive, not automatically local-curvature-adaptive.

Response information enters through other mechanisms: joint \((X,Y)\) partitions, response-dependent choice of order, and final regression-point combination. In NNS.reg, a dependence-selected order is a dimension-specific resolution choice, not a separate local optimization of curvature at every \(x\). Keeping these sources of adaptivity distinct avoids attributing a property to the geometric split that actually belongs to tuning or prediction.

The bandwidth interpretation should accordingly be read as: the realized region width is data-driven, not that its width is automatically optimal for the unknown regression function.


19.4 Bandwidth, Occupancy, and Conditional-Mean Consistency

Chapter 18 established an elementary pointwise result for response-adaptive, axis-aligned predictor cells. Here the same result becomes a statement about stochastic bandwidth.

Assume independent identically distributed observations \((X_i,Y_i)\), a fixed predictor dimension \(d\), bounded responses \(|Y_i|\le M\), and a version of \(f(x)=E[Y\mid X=x]\) continuous at the query \(x\). Let \(A_n(x)\) be an axis-aligned rectangular predictor cell selected by a measurable rule that may use the full sample, including the responses. The sufficient conditions are

\[ \boxed{ r_n^{\mathrm{cell}}(x)\xrightarrow{P}0, \qquad \frac{N_n(x)}{\log n}\xrightarrow{P}\infty. } \]

One may replace observed reach by a shrinking support-restricted diameter. Under these conditions, the arithmetic cell mean satisfies

\[ \boxed{ E\bigl[(\hat f_n^{\mathrm{cell}}(x)-f(x))^2\bigr] \longrightarrow0. } \]

The point is not that choosing cells with \(Y\) is harmless in every circumstance. Rather, selection over all rectangular cells has sufficiently controlled complexity that one can bound the random response-average error simultaneously over those candidate cells. The selected cell then inherits that uniform bound, even if the selection used the responses.

19.4.1 Why response selection does not defeat this result

Write \(\varepsilon_i=Y_i-f(X_i)\). Since \(|Y_i|\le M\), we have \(|\varepsilon_i|\le 2M\) and \(E[\varepsilon_i\mid X_i]=0\). For any nonempty predictor cell,

\[ \hat f_n^{\mathrm{cell}}(x)-f(x) =\underbrace{\frac1{N_n(x)}\sum_{i:X_i\in A_n(x)}\varepsilon_i}_{\text{sample noise}} +\underbrace{\frac1{N_n(x)}\sum_{i:X_i\in A_n(x)}\{f(X_i)-f(x)\}}_{\text{localization error}}. \]

Let \(\omega_{f,x}(r)=\sup_{\|z-x\|\le r}|f(z)-f(x)|\). The second term is bounded in magnitude by \(\omega_{f,x}(r_n^{\mathrm{cell}}(x))\), which tends to zero by continuity and shrinking reach.

For the first term, conditional on the predictors, the number of distinct sample subsets selected by axis-aligned rectangles in fixed dimension is at most \((n+1)^{2d}\). Hoeffding’s inequality and a union bound therefore yield, for any \(m\ge1\) and \(t>0\),

\[ \Pr\!\left( \sup_{\substack{A\text{ rectangular}\\N_n(A)\ge m}} \left|\frac1{N_n(A)}\sum_{i:X_i\in A}\varepsilon_i\right|>t \right) \le 2(n+1)^{2d}\exp\!\left(-\frac{m t^2}{8M^2}\right). \]

This bound concerns the supremum over candidate rectangular subsets, before the selected partition is known. If the chosen cell’s occupancy grows faster than \(\log n\), the right side tends to zero. Since both the cell average and \(f(x)\) are bounded, convergence in probability also gives mean-square convergence.

This argument establishes the cell-mean building block. It does not require the split positions to be preselected, independent of \(Y\), or symmetric about \(x\). It does require fixed dimension, appropriate localization and occupancy, and the specified rectangular candidate class. The broader integrated theory of Nobel (1996) uses partition complexity and shrinking cells without requiring the same pointwise \(N_n/\log n\) condition in every selected cell.

19.4.2 A resolution sequence compatible with consistency

For a regular one-dimensional design, suppose recursive splits remain sufficiently balanced that cells at depth \(O_n\) have widths of order \(2^{-O_n}\) and occupancies of order \(n2^{-O_n}\). Then a sufficient illustrative scaling regime is

\[ O_n\longrightarrow\infty, \qquad \frac{n2^{-O_n}}{\log n}\longrightarrow\infty. \]

For example, a depth sequence of order \(O_n\asymp \tfrac12\log_2 n\) gives widths of order \(n^{-1/2}\) and local counts of order \(n^{1/2}\), subject to those balancing assumptions. That example establishes compatibility between refinement and noise averaging. It does not prove that the actual iterative-mean recursion always yields balanced daughters; ties, atoms, uneven predictor distributions, depth caps, and stopping rules must be examined.

More abstractly, if \(K_n\) cells of comparable probability are produced, the heuristic requirements are

\[ K_n\longrightarrow\infty, \qquad \frac{K_n\log n}{n}\longrightarrow0. \]

Cell count alone is not enough: local geometry and actual occupancy are the quantities used in the theorem.


19.5 Two Bandwidths in the Complete NNS Predictor

A cell mean is piecewise constant. Univariate NNS.reg ordinarily returns a piecewise-linear interpolation between selected regression points. This creates a second, prediction-level neighborhood.

19.5.1 Linear interpolation of local response means

Suppose two regression-point centers bracket an interior query \(x\):

\[ c_-\le x\le c_+, \qquad r_- = \frac1{N_-}\sum_{i:X_i\in B_-}Y_i, \qquad r_+ = \frac1{N_+}\sum_{i:X_i\in B_+}Y_i, \]

where \(B_-\) and \(B_+\) are the predictor intervals represented by those centers. The linear interpolation weights are

\[ \lambda_- = \frac{c_+-x}{c_+-c_-}, \qquad \lambda_+ = \frac{x-c_-}{c_+-c_-}, \qquad \lambda_-+\lambda_+=1. \]

At an interior query, both weights lie in \([0,1]\), and the interpolated estimate is

\[ \hat f_n^{\mathrm{lin}}(x) =\lambda_-r_-+\lambda_+r_+. \]

Thus linear interpolation does not create a new response measurement. It combines two already estimated response centers. Expressed directly in observation weights,

\[ \hat f_n^{\mathrm{lin}}(x)=\sum_iw_{ni}^{\mathrm{lin}}(x)Y_i, \quad w_{ni}^{\mathrm{lin}}(x) =\frac{\lambda_-}{N_-}\mathbf1\{X_i\in B_-\} +\frac{\lambda_+}{N_+}\mathbf1\{X_i\in B_+\}. \]

If the two cells are disjoint, these weights are nonnegative and sum to one. The resulting predictor is still a local average, but its effective neighborhood is the union of the contributing cells rather than solely the one containing \(x\).

Define its prediction reach by

\[ r_n^{\mathrm{pred}}(x)= \max_{i:w_{ni}^{\mathrm{lin}}(x)>0}\|X_i-x\|. \]

A smaller diameter for one cell need not improve prediction if interpolation gives appreciable weight to a response center whose underlying observations remain far from \(x\). Conversely, if both contributing cells shrink around \(x\), both response means concentrate around \(f(x)\), and convex interpolation preserves that convergence.

Under the assumptions of Chapter 18, the following is sufficient: with probability approaching one, the two bracketing regression points represent intervals whose observed reaches from \(x\) vanish and whose occupancies divided by \(\log n\) diverge. Uniform rectangular concentration makes each response mean consistent; the convex combination is then also consistent. This is the connection between partition bandwidth and prediction bandwidth needed for the complete univariate mean-reducer predictor.

19.5.2 Effective sample size of interpolation

When the cell means use independent noise conditional on an \(X\)-chosen partition, the variance of an interpolated estimate with disjoint constituent cells is controlled by

\[ \operatorname{Var}(\hat f_n^{\mathrm{lin}}(x)\mid X) \le\sigma^2\left(\frac{\lambda_-^2}{N_-} +\frac{\lambda_+^2}{N_+}\right) \]

if the conditional noise variances are bounded by \(\sigma^2\). This suggests the diagnostic effective sample size

\[ n_{\mathrm{eff}}(x) =\left(\frac{\lambda_-^2}{N_-} +\frac{\lambda_+^2}{N_+}\right)^{-1}. \]

For response-selected cells, the displayed conditional variance calculation must not be applied as though the selected groups were independent of their own noise. The weight formula remains an algebraic description, but uniform concentration over the eligible cell family is the rigorous tool used above. Effective sample size is still informative as a measure of dispersion of prediction weights; it is not a substitute for the selection-aware proof.

19.5.3 Endpoints, extrapolation, and alternative smoothing

The current univariate NNS.reg code adds regression points at the minimum and maximum observed predictor values. Their response coordinates use the selected reducer on observations sharing the endpoint predictor. For distinct continuous predictors these endpoint groups are usually singletons. They are useful interpolation anchors, but provide no averaging-based noise suppression on their own.

At a fixed interior query, sufficiently local, well-supported bracketing points may eventually make the endpoint anchors irrelevant. At a boundary query, and certainly outside the training predictor range, a theorem about interior convex interpolation does not automatically apply. The implementation extends the first or last line segment to extrapolate; outside the bracketing interval, linear extrapolation coefficients need not be nonnegative and can magnify noise. An optional smoothing spline (smooth = TRUE) also has its own weight and regularity properties, and requires separate analysis.

Consequently, the statement “NNS has a dynamic bandwidth” is accurate only after specifying which stage of the estimator is meant. The partition creates the local representative scale; the final prediction rule determines the actual influence of those representatives at a query.


19.6 Why Neither Extreme Guarantees Statistical Consistency

The 2018 Vinod-Viole paper, Nonparametric Regression Using Clusters (published online in 2017), stresses a genuine geometric feature: as order increases, NNS can represent increasingly fine structure and, in a sufficiently refined limit, connect all observed points. Its discussion also cautions that excessive order can be undesirable in noisy data. The following examples explain both observations mathematically.

19.6.1 Bounded order: an approximation floor

The current automatic NNS.reg order is dependence-selected but bounded above by 10. For a numeric X-only order \(O\), the partitioner supplies at most \(2^{O-1}\) regression-point centers, and the univariate regression-point builder adds the two observed predictor endpoints. At \(O\le10\), the fitted curve therefore has at most 513 linear pieces.

Consider noiseless data from \(f(x)=x^2\) with a predictor distributed uniformly on \([0,1]\). Regardless of where those pieces are placed, the best integrated squared approximation error of an affine function on an interval of length \(\ell\) is \(\ell^5/180\). By convexity, the best possible piecewise-affine approximation with at most \(m\) pieces has integrated squared error at least

\[ \frac{1}{180m^4}. \]

With \(m\le513\), that gives

\[ \int_0^1\{\hat f_n(x)-x^2\}^2\,dx \ge\frac{1}{180(513)^4}>0. \]

The bound is extremely small, but strictly positive independent of sample size. An adaptive placement of finitely many line segments cannot guarantee asymptotically zero error for every curved regression function. The issue is not that the NNS architecture lacks increasing-resolution capability; it is that the bounded default order does not implement an order sequence growing without bound.

19.6.2 Maximum order: a noise floor

At the opposite extreme, consider evenly spaced predictors \(X_i=i/(n+1)\), even \(n\), and independent responses \(Y_i=\varepsilon_i\) with mean zero and variance \(\sigma^2>0\). The true regression function is \(f(x)=0\). With mean reduction and order = "max", each distinct predictor is its own regression point.

At \(x=1/2\), midway between two observations, the linear interpolant is

\[ \hat f_n(1/2)=\frac{\varepsilon_{n/2}+\varepsilon_{n/2+1}}2, \]

and hence

\[ E[(\hat f_n(1/2)-f(1/2))^2]=\frac{\sigma^2}{2}, \]

for every even \(n\). The fitted values interpolate the training responses perfectly, yet the error at a new interior point does not vanish. In bandwidth language, the prediction neighborhood shrinks, but its effective sample size stays at two.

19.6.3 The positive noiseless interpolation limit

The maximum-order construction does have a genuine positive approximation theorem. Suppose \(f\) is continuous on a compact interval and we observe its noiseless values on increasingly dense predictor meshes. When the maximal gap between successive predictors tends to zero, the linear interpolant through those observations converges uniformly to \(f\).

Indeed, if \(\delta_n\) is the largest observed gap and \(\omega_f\) is the modulus of continuity of \(f\), then on the range covered by the observed endpoints,

\[ \sup_x|\hat f_n^{\mathrm{max}}(x)-f(x)| \le\omega_f(\delta_n)\longrightarrow0. \]

This recovers the structural intuition of the 2018 paper: increasingly fine line segments approximate a continuous noiseless function. It must not be confused with statistical recovery of \(f(x)=E[Y\mid X=x]\) in the presence of observation noise. For noisy data, consistency requires a sequence that shrinks neighborhoods without reducing the effective number of observations to a fixed small value.


19.7 Arithmetic Means, Gravity, and the Target of Estimation

The preceding consistency statements concern the conditional arithmetic mean

\[ f(x)=E[Y\mid X=x]. \]

Under noise.reduction = "mean", each regression point’s response coordinate is an arithmetic average of the observations assigned to its group. This makes the connection to ordinary conditional-mean regression exact at the local-representative level. It is the appropriate setting for the theorems in Chapter 18 and the bandwidth analysis above.

The current package default is different: noise.reduction = "off" uses NNS.gravity, a blend of mean, median, quartile midpoint and a histogram-based modal component. Gravity can be a valuable alternative central tendency, but generally targets a different functional of a conditional response distribution.

To see why reducer choice matters independently of bandwidth, suppose \(Y\) is Bernoulli with \(P(Y=1\mid X=x)=0.2\), constant in \(x\). Its conditional mean is 0.2. In sufficiently large local samples, the median, modal component, and quartile midpoint approach 0 while the mean approaches 0.2. Gravity therefore approaches 0.05 in this example, rather than 0.2. Even perfect localization and growing occupancy need not make the default gravity response center a consistent estimator of the conditional arithmetic mean.

This observation does not make gravity an invalid estimator. It means that any consistency assertion must name the estimand. Median, mode, gravity, and arithmetic mean are not interchangeable for asymmetric conditional distributions. Similarly, type = "CLASS" is a classification workflow and should not be treated as the arithmetic mean regression path merely because its responses may be coded numerically. When demonstrating numeric regression for a two-valued response, use type = "REG" explicitly.


19.8 Multivariate Dynamic Bandwidth and the RPM

For predictors \(X_i\in\mathbb R^d\), NNS computes per-regressor partition representatives and forms joint group memberships. The multivariate implementation uses the resulting predictor-coordinate representative locations to form interval IDs, then combines the IDs across dimensions. Thus the actual RPM group cells, rather than an assumed copy of every standalone NNS.part terminal quadrant, are the appropriate cells to analyze when assessing the complete multivariate regression procedure.

19.8.1 Coordinatewise and joint scale

Write \(A_n^{(j)}(x_j)\) for the interval associated with predictor coordinate \(j\) and a given RPM group. The intersected group cell is a rectangle of the form

\[ A_n^{\mathrm{joint}}(x) =\prod_{j=1}^d A_n^{(j)}(x_j), \]

with coordinatewise widths \(h_{nj}(x_j)\). When all coordinate widths tend to zero, the Euclidean diameter of this rectangle also tends to zero. But local joint occupancy is not determined by the marginal widths alone. Correlated predictors, thin supports and sparse intersections can produce few or no observations in many product cells.

If the per-regressor label counts are \(K_1,\ldots,K_d\), the number of occupied RPM groups satisfies

\[ K_{\mathrm{RPM}}\le \min\left(n,\prod_{j=1}^d K_j\right). \]

The product, rather than a sum, matters. Even modest refinement in every coordinate can create many joint groups. The method simplifies the construction of the marginal partitions, but does not remove the need for sufficient observations in the occupied intersections.

19.8.2 A third scale: neighbor reach

The RPM stores representative predictor coordinates and response centers. Multivariate prediction then selects and weights nearby rows according to the configured distance and n.best rules. As a result, a query may receive influence from multiple group cells.

For a nonnegative, normalized observation-weight representation \(\hat f_n(x)=\sum_iw_{ni}(x)Y_i\), a useful locality diagnostic is the nonlocal weight

\[ W_n(x,\delta)=\sum_{i=1}^n w_{ni}(x)\mathbf1\{\|X_i-x\|>\delta\}. \]

The condition \(W_n(x,\delta)\to0\) for every fixed \(\delta>0\) expresses local concentration of prediction weight. Unlike the largest nonzero-weight distance, it can still vanish when distant observations receive weights tending to zero. Effective sample size can be recorded as

\[ n_{\mathrm{eff}}(x)=\frac1{\sum_iw_{ni}(x)^2}, \]

when the observation weights are nonnegative and sum to one. Neither formula by itself handles every implementation detail; for example, the k=1 exact-nearest-distance-tie path uses gravity to combine tied RPM responses, and optional dimension reduction changes the metric space in which locality is measured. Those paths require their own target- and geometry-aware checks.

A sufficient route to multivariate mean consistency is for RPM rows carrying nonnegligible prediction weight to represent uniformly local rectangular groups with sufficiently increasing occupancies, while the final aggregation remains stable. Chapter 18 states this as a conditional sufficient principle. A generic neighbor-count default, fixed dimension by itself, or high training fit does not establish it automatically.

19.8.3 Dimensionality and rates

In a regular \(d\)-dimensional design, halving the width of every coordinate reduces the volume of a rectangular neighborhood by a factor near \(2^d\). Thus the number of observations available to estimate a local response mean can fall quickly as dimension increases. This is the ordinary dimensionality cost of local estimation. NNS can adapt to occupied geometry, avoid spending representatives on empty intersections, and compress groups when multiple observations share them. These advantages do not prove a dimension-free convergence rate or universal avoidance of the curse of dimensionality.

The practical diagnostics are the per-coordinate widths, actual joint-group counts, occupied RPM rows, and neighbor-weight distribution at the prediction point.


19.9 Comparison with Other Partition and Smoothing Methods

19.9.1 Kernel regression

A kernel method supplies weights through distances to the query and a selected bandwidth; a cell-mean NNS partition supplies uniform weights over a data-generated cell. Both can be local smoothers, and both can be tuned too coarsely or too finely. Some kernels have compact support; others assign positive weight at every distance. Adaptive bandwidth and cross-validation are available to kernel methods too, so the distinction is how the neighborhood is constructed, not that only NNS can adapt.

For NNS.reg, interpolation changes the weighting beyond a pure cell indicator. When comparing methods, the meaningful comparison is between the complete prediction weights and their effective local support, not just between a kernel’s global bandwidth and the width of one NNS cell.

19.9.2 Fixed-grid and partitioning-based series estimators

A fixed grid chooses boundaries without looking at the observed predictor distribution. Recursive mean splitting instead places cuts at local centers of occupied regions. Both may use piecewise-constant or piecewise-polynomial approximations, and both require complexity and localization conditions for consistency.

The partitioning-based least-squares series framework developed by Cattaneo, Farrell, and Feng (2020) helps interpret resolution, local basis functions and approximation bias. Its principal asymptotic statements do not automatically prove the full response-selected NNS interpolation procedure. Nobel’s (1996) histogram-regression theory is the more direct general precedent for data-dependent response-selected partitions; Chapter 18 supplies the elementary fixed-dimensional rectangle argument for NNS-style cell means.

19.9.3 CART and regression trees

CART typically selects boundaries to improve an impurity or prediction-error criterion. NNS mean splits instead use the local central benchmark, without a greedy search for the split that best reduces squared error. Both produce partitions with locally varying cell sizes, but their adaptivity is driven by different rules. For a fixed predictor sample and fixed order, mean-based X-only splits do not relocate in response to a change in \(Y\), whereas regression-tree splits ordinarily can.

This explains why two methods may both have stochastic bandwidths without making the same predictions or satisfying the same convergence assumptions.


19.10 A Simple Illustration of Partition and Prediction Bandwidth

Consider the six observations

\[ (X,Y)= (1,2),(2,3),(3,3),(6,8),(7,9),(8,9). \]

The initial predictor mean is

\[ \bar X=\frac{1+2+3+6+7+8}{6}=4.5. \]

An X-only mean split produces a left region with \(X\in\{1,2,3\}\) and a right region with \(X\in\{6,7,8\}\). Their regression-point centers are

\[ (c_L,r_L)=\left(2,\frac83\right), \qquad (c_R,r_R)=\left(7,\frac{26}{3}\right). \]

If the predictor domain is restricted to \([1,8]\), the geometric widths of the two cells formed at the first split are each 3.5. Their observed spans are each 2. These are different ways of describing the same realized partition, but only the support-restricted geometric width is the actual interval diameter.

As a piecewise-constant estimator, the left interval gives \(\hat f^{\mathrm{cell}}(3)=8/3\). If the two regression-point means are connected with a straight segment, the interpolated value at \(x=3\) is instead

\[ \hat f^{\mathrm{lin}}(3) =\frac45\frac83+\frac15\frac{26}{3} =\frac{58}{15}. \]

The right-hand group contributes one fifth of that prediction, even though its predictor observations lie to the right of the first partition boundary. This is an explicit example in which the prediction neighborhood extends beyond the query’s own cell.

The following R code reproduces the X-only partition and asks NNS.reg for arithmetic-mean regression estimates at specified points. The explicit type = "REG" requests numeric regression, while the two noise.reduction = "mean" settings align the examples with the conditional-mean theory.

library(NNS)

x <- c(1, 2, 3, 6, 7, 8)
y <- c(2, 3, 3, 8, 9, 9)

part <- NNS.part(
  x, y,
  type = "XONLY", order = 2,
  obs.req = 0, min.obs.stop = FALSE,
  noise.reduction = "mean"
)
part$regression.points

fit <- NNS.reg(
  x, y,
  type = "REG", order = 2,
  noise.reduction = "mean",
  point.est = c(3, 4.5),
  plot = FALSE, ncores = 1
)
fit$regression.points
fit$Point.est

With the current univariate numeric-order architecture, regression points at the observed predictor endpoints are also included. The code is therefore an illustration of the actual package prediction path, not a claim that its result must always equal the two-center illustrative calculation above at every depth or under every stopping rule.

A second exercise is to vary the sample size while recording the realized order, widths and occupancies. This supplies a direct empirical check of whether a chosen tuning schedule achieves the simultaneous refinement and averaging demanded by the theory; a rising number of regression points alone is not conclusive.


19.11 A Practical Consistency Diagnostic

For a specified regression configuration, distinguish three quantities:

Diagnostic Meaning Why it matters
Partition radius or diameter Localization of each underlying predictor cell Controls approximation bias when \(f\) is continuous
Cell occupancy or effective sample size Amount of independent response information being averaged Controls sampling noise, subject to selection-aware analysis
Prediction reach and nonlocal weight The actual observation influence after interpolation/RPM weighting Determines whether the complete predictor stays local

The relevant asymptotic checks are:

  1. Localization: at an interior query, contributing predictor cells or weighted observation neighborhoods become local.
  2. Increasing support: cells carrying substantive weight average enough observations; a local count growing faster than \(\log n\) is one explicit sufficient requirement for response-adaptively selected fixed-dimensional rectangles.
  3. Controlled selection: the admissible partition class has the requisite complexity, as axis-aligned rectangles do for fixed dimension.
  4. Prediction stability: interpolation weights or neighbor combinations do not continue to give material influence to unstable or distant representatives.
  5. Target alignment: arithmetic means estimate conditional means; gravity, median and mode require their own estimands and convergence arguments.

The same diagnostics can guide finite-sample validation, but asymptotic consistency and predictive accuracy at a given sample size remain different questions. Cross-validation may help choose resolution, yet it is not a substitute for a theorem showing that the chosen tuning sequence meets those conditions.


19.12 Structural Interpretation within NNS

The dynamic-bandwidth view connects the probability and estimation parts of the NNS framework without replacing their original motivation.

Earlier chapters defined directional deviations and partial moments relative to benchmarks. Chapter 18 converted local benchmark geometry into recursive partitions and regression points. This chapter shows that the resulting regions are also local smoothing neighborhoods, each with its own stochastic size and occupancy. Mean splitting places their boundaries at locally balanced first-order partial moments, and the chosen prediction rule converts the corresponding response centers into a regression function.

This is also the appropriate place to reconcile the historical NNS papers. Vinod and Viole (2018) established the significance of connecting local cluster means by line segments, including the exact limiting interpolation of noiseless observed functions. Viole (2021) recognized the iterative partitioning device as an adaptive bandwidth mechanism and linked it to the broader partition-estimation literature. The present chapter supplies the explicit statistical separation that complements those contributions: increasing approximation resolution is not equivalent to suppressing observation noise. Both effects must be controlled when the target is a conditional mean.

The contribution remains distinctive: directional-benchmark geometry generates data-adaptive representation scales without requiring the analyst to specify a kernel bandwidth as the primitive tuning parameter. But order, occupancy, choice of center and the ultimate prediction weights still matter. Bandwidth has become implicit, not irrelevant.


19.13 Summary

Recursive mean splitting admits a direct bandwidth interpretation, but the full interpretation has two levels.

  • The partition bandwidth is the size of the predictor cell represented by an NNS local center. It is stochastic, location-dependent, and controlled by the realized partition geometry.
  • The prediction bandwidth is the actual neighborhood contributing to a fitted value after interpolation or multivariate RPM weighting. It can span more than one partition cell.
  • In the arithmetic-mean setting, the cell-mean estimator is a local-constant smoother. Response-adaptive selection does not itself prevent consistency: for fixed-dimensional rectangular cells, shrinking local reach and occupancy growing faster than \(\log n\) suffice under the bounded-response and continuity assumptions of Chapter 18.
  • Consistent interpolation additionally requires that the response centers receiving appreciable weight are local and adequately supported. Endpoints, extrapolation, and optional smoothing must be treated separately.
  • Maximum-order interpolation is an exact sample-fit limit, and for noiseless continuous functions on dense meshes it gives uniform approximation. With noisy responses, singleton points can preserve nonvanishing sampling error.
  • Conversely, a bounded maximum order can preserve nonvanishing approximation bias for curved regression functions. The current dependence-selected NNS.reg default is bounded, even though the general NNS architecture allows larger numeric orders.
  • The default gravity reducer need not estimate the conditional arithmetic mean. The relevant target must be specified before making a consistency claim.
  • In several dimensions, the sizes of factorwise partitions do not determine occupancy of joint RPM groups. Final neighbor weights create another layer of effective prediction scale.

The coherent conclusion of Chapters 18 and 19 is that NNS with arithmetic-mean reduction is a consistency-capable data-adaptive partition architecture. Its consistency arises from a sequence that both localizes and continues to average enough observations, not from a perfect fit or a particular bounded default order alone.

Chapter 21 develops clustering as another consequence of recursive partition geometry, and Chapter 22 uses the regression-point matrix to construct complete multivariate predictors.


19.14 References

  • Stone, C. J. (1977). Consistent nonparametric regression. Annals of Statistics, 5(4), 595-620.

  • Nobel, A. (1996). Histogram regression estimation using data-dependent partitions. Annals of Statistics, 24(3), 1084-1105. https://doi.org/10.1214/aos/1032526958

  • Lugosi, G., & Nobel, A. (1996). Consistency of data-driven histogram methods for density estimation and classification. Annals of Statistics, 24(2), 687-706.

  • Cattaneo, M. D., Farrell, M. H., & Feng, Y. (2020). Large sample properties of partitioning-based series estimators. Annals of Statistics, 48(3), 1718-1741. https://doi.org/10.1214/19-AOS1865

  • Györfi, L., Kohler, M., Krzyżak, A., & Walk, H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer.

  • Wand, M. P., & Jones, M. C. (1995). Kernel Smoothing. Chapman and Hall.

  • Fan, J., & Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall.

  • Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth.

  • Vinod, H. D., & Viole, F. (2018). Nonparametric regression using clusters. Computational Economics, 52, 1317-1334 (published online June 19, 2017). https://doi.org/10.1007/s10614-017-9713-5

  • Viole, F. (2021, April 22). Partitional Estimation Using Partial Moments. Working paper.

  • Stone, H. (1961). Approximation of curves by line segments. Mathematics of Computation, 15(73), 40-47.