When building HearingAid Pro, we faced a classic mobile DSP challenge: AirPods capture audio at 44,100 Hz, but our speech enhancement pipeline expects 16,000 Hz. Naive downsampling—taking every third sample—introduces aliasing artifacts that destroy intelligibility. The solution: polyphase decimation, a filter-bank architecture that achieves mathematically perfect resampling with minimal CPU cost.

The Aliasing Problem in Sample-Rate Conversion

Decimation by factor M means keeping every Mth sample and discarding the rest. Without filtering, frequency components above the new Nyquist limit (8 kHz for 16 kHz output) fold back into the passband. For speech, this manifests as metallic ringing on sibilants and loss of formant clarity.

The textbook solution: apply a lowpass FIR filter at the original sample rate, then downsample. A 44.1→16 kHz conversion (decimation by 2.75625) typically requires a 128-tap FIR at 44.1 kHz—5,644,800 multiply-accumulates per second on a single channel. On an iPhone 12's efficiency cores (1.8 GHz), that's 0.31% CPU, but it scales linearly with channel count. For our stereo AirPods Pro pipeline with five-stage processing, the naive approach consumed 3.1% CPU just for resampling.

Polyphase Decomposition: Interleaving the Filter

The key insight: if we're discarding samples, why compute them? Polyphase decomposition splits the prototype filter into M parallel subfilters, each operating at the output rate. For decimation by M=3, a 12-tap filter becomes three 4-tap filters. Each output sample is computed by one subfilter, cycling through the bank.

The prototype lowpass filter h[n] (length L) decomposes into M subfilters:

p_k[m] = h[mM + k]  for k = 0, 1, ..., M-1

Each subfilter operates at the decimated rate, so computational cost drops by exactly factor M. For our 44.1→16 kHz case with M≈2.76, we achieve 64% reduction in multiply-accumulates. The output at time index n is:

y[n] = sum(p_{n mod M}[m] * x[nM - m])

Rational Decimation: Handling Non-Integer Factors

Real-world sample-rate pairs rarely divide evenly. 44,100 / 16,000 = 441/160 after reduction. We implement this as a two-stage process: upsample by 160 (interpolation), then downsample by 441 (decimation). Polyphase structures handle both.

For interpolation by L, we insert L-1 zeros between samples, then filter. The polyphase form inverts: L parallel filters run at the input rate, their outputs interleaved. Combined interpolation-decimation becomes a single polyphase matrix operation with L×M subfilters, but only L filters execute per output sample.

In practice, we use a Noble identity transformation: swap the order of upsampling and filtering when mathematically equivalent. This moves the expensive filtering step to the lower sample rate. For 44.1→16 kHz, our 160-phase bank operates at 44.1 kHz, but each phase is a short 8-tap filter (versus 128 taps for direct implementation).

Implementation: Fixed-Point Arithmetic and NEON

Floating-point multiply-accumulate on ARM Cortex-A is fast, but fixed-point Q15 format (16-bit signed, 15 fractional bits) enables SIMD parallelism. ARM NEON processes eight Q15 samples per instruction with vmlal_s16 (multiply-accumulate-long), doubling throughput.

Our filter coefficients are precomputed in Q15, stored in a circular buffer aligned to 128 bits. The inner loop for a single polyphase branch:

int32_t acc = 0;
for (int i = 0; i < taps; i += 8) {
    int16x8_t x = vld1q_s16(&input[i]);
    int16x8_t h = vld1q_s16(&coeff[phase][i]);
    acc = vmlal_s16(acc, x, h);
}
output[n] = acc >> 15;

Eight-way parallelism reduces the loop count by 8×. For our 8-tap subfilters, the loop executes once per output sample. On iPhone 12, measured latency is 0.11% CPU per stereo channel—down from 3.1% for the naive approach.

Windowing and Transition Bandwidth

The prototype filter's design governs passband ripple, stopband attenuation, and transition bandwidth. We use a Kaiser window with β=6.5, yielding 60 dB stopband rejection and a transition band from 7.2 to 8.8 kHz (1.6 kHz width). This keeps speech formants below 7 kHz pristine while suppressing aliasing above 8 kHz.

Tighter transition bands require longer filters. Doubling the transition bandwidth halves the filter length—and halves the CPU cost. For speech intelligibility, 1.6 kHz is sufficient; for music, we'd need 0.4 kHz (4× more taps). This tradeoff is application-specific.

Phase Response and Group Delay

Linear-phase FIR filters introduce constant group delay: L/2 samples at the input rate. For our 128-tap filter at 44.1 kHz, that's 1.45 ms. Polyphase decomposition preserves this—it's an algebraic rearrangement, not an approximation. The delay matters for real-time applications: in HearingAid Pro, we budget 5 ms end-to-end (capture to playback). Resampling consumes 1.45 ms, leaving 3.55 ms for enhancement algorithms.

Minimum-phase filters reduce delay to ~0.5 ms but introduce frequency-dependent phase distortion. For hearing aids, linear phase is critical—phase shifts alter binaural cues (ITD/ILD) that the brain uses for spatial hearing. We accept the delay cost.

Fractional Delay Filters

When upsampling, polyphase interpolation requires fractional-sample delays between phases. A Lagrange interpolator (polynomial fit) achieves this with 4-6 taps per phase. For our 160-phase bank, each phase filter has 8 taps, approximating a sinc function windowed by Kaiser. The result: 60 dB image rejection.

Memory Layout and Cache Efficiency

Circular buffers for audio input must align to the subfilter length. We allocate (taps + block_size) samples, using modulo indexing. On ARM, pointer arithmetic with bitmask ANDs is faster than modulo for power-of-two sizes, but our 8-tap filters don't align. Instead, we use explicit wraparound checks every 512 samples (cache-line boundary).

Coefficient storage: 160 phases × 8 taps × 2 bytes = 2.5 KB, fitting in L1 cache. Interleaving coefficients by phase (rather than by tap index) improves spatial locality—each phase's 8 taps load in a single cache line.

Benchmarks: Real-World Performance

On iPhone 12 (A14 Bionic), our polyphase resampler processes stereo 44.1→16 kHz at 0.22% CPU (both channels). For comparison: Apple's AVAudioConverter uses a similar polyphase algorithm and benchmarks at 0.19% CPU—within measurement noise. The open-source libsamplerate (SRC_SINC_BEST_QUALITY) runs at 0.31% CPU, likely due to longer filters (128 taps vs. our 64).

On Android (Snapdragon 888), our implementation runs at 0.28% CPU. The 27% slowdown versus iPhone reflects differences in NEON scheduler and memory subsystem, not algorithmic changes—the same C intrinsics compile for both.

When Polyphase Isn't Enough

For time-varying sample rates (e.g., Bluetooth audio with drift correction), polyphase banks are static. We layer a Farrow structure—a polynomial interpolator that accepts fractional phase increments—atop the polyphase decimator. This adds 0.08% CPU but handles ±50 ppm clock drift without artifacts.

For extreme decimation factors (e.g., 48 kHz → 8 kHz, factor 6), cascaded stages outperform single-stage polyphase. We decimate by 2, then 3, using shorter filters at each stage. Total cost: 0.15% CPU versus 0.21% for single-stage.

Practical Takeaways

Polyphase decimation is the standard for mobile audio resampling because it's mathematically exact and computationally minimal. Key lessons: (1) decompose the filter to operate at the output rate; (2) use fixed-point Q15 with NEON for 2× throughput; (3) design the prototype filter's transition band to match application needs—speech tolerates 1.6 kHz, music needs 0.4 kHz; (4) accept linear-phase delay for binaural applications; (5) for rational factors, combine interpolation and decimation in a single polyphase matrix.

In production, the difference between naive downsampling and polyphase is the difference between a 2-star app with "sounds tinny" reviews and a 4.7-star app. The math is 40 years old, but it still ships in every audio app you use.