spectrochempy.Filter#

class Filter(log_level='WARNING', *, cutoff=None, cval=0.0, delta=None, deriv=0, lamb=1.0, mask_policy='default', method='savgol', mode='interp', order=2, period=None, quality_factor=30.0, sigma=None, sigma_units='points', size=5, wavelet='sym4', wavelet_level=None, wavelet_threshold=None)[source]#

Filters/smoothers processor.

The filters can be applied to 1D datasets consisting in a single row with n_features or to a 2D dataset with shape (n_observations, n_features). Butterworth, notch, Gaussian and wavelet filters also support higher-dimensional datasets. Physical settings use the active coordinate on axes with multiple coordinates.

Various filters/smoothers can be applied to the data. The currently available filters are:

  • Moving average (avg)

  • Convolution filters (han, hamming, bartlett, blackman)

  • Savitzky-Golay filter (savgol)

  • Whittaker-Eilers filter (whittaker)

  • Zero-phase Butterworth low-pass filter (butterworth)

  • Zero-phase narrow band-stop filter (notch)

  • Gaussian convolution (gaussian)

  • Discrete wavelet denoising (wavelet)

Parameters:
  • log_level (any of ["INFO", "DEBUG", "WARNING", "ERROR"], optional, default: "WARNING") – The log level at startup. It can be changed later on using the set_log_level method or by changing the log_level attribute.

  • cutoff (float, optional) – Required for Butterworth. Frequency in cycles per unit of the selected coordinate. The combined forward-backward response is −3 dB at cutoff.

  • cval (float, optional, default: 0.0) – The value to fill past the edges of the input if mode is 'constant'.

  • delta (float, optional, default: None) – The signed sample spacing passed to scipy.signal.savgol_filter. This is only used if deriv > 0.

    When None (the default), the signed spacing is automatically derived from the coordinate of the processed axis if the retained samples are uniformly spaced, otherwise from the original coordinate if it is uniform. On a non-uniform or missing coordinate a warning is emitted and the index-based delta=1.0 is used as a fallback.

    When set to a numeric value, that value is passed directly to SciPy with its sign. No unit-based correction (_reversed) is applied. For a descending coordinate, supply a negative delta if the derivative should follow the physical axis.

  • deriv (int, optional, default: 0) – The order of the derivative to compute.

  • lamb (float, optional, default: 10.0) – The smoothing parameter for the Whittaker-Eilers filter.

  • mask_policy ({“default”, “split”}, optional, default: “default”) – With “split”, filter each contiguous unmasked segment independently along the selected dimension, even when spectra have different masks. Preserve the mask and apply the filter’s boundary handling at each segment’s ends. Masked values are converted to the output dtype and may be rounded if that dtype cannot represent them exactly. If any unmasked segment is too short for the configured filter, raise ValueError for the whole operation. Entirely masked spectra are skipped. The default uses the method’s original mask handling.

  • method (any of ["avg", "han", "hamming", "bartlett", "blackman", "median", "savgol", "whittaker", "butterworth", "notch", "gaussian", "wavelet"], optional, default: "savgol") – The filter method to be applied. By default, the Savitzky-Golay (savgol) filter is applied.

  • mode (any value of ['mirror', 'constant', 'nearest', 'wrap', 'interp'], optional, default: 'interp') – The type of extension to use for the padded signal to which the filter is applied.

    • When mode is ‘constant’, the padding value is given by cval.

    • When the ‘interp’ mode is selected (the default), no extension is used. Instead, a polynomial of degree order is fit to the last size values of the edges, and this polynomial is used to evaluate the last size // 2 output values.

    • When mode is ‘nearest’, the last size values are repeated.

    • When mode is ‘mirror’, the padding is created by reflecting the signal about the end of the signal.

    • When mode is ‘wrap’, the signal is wrapped around on itself to create the padding.

    See scipy.signal.savgol_filter for more details on ‘mirror’, ‘constant’, ‘wrap’, and ‘nearest’.

  • order (int, optional, default: 2) – For Savitzky-Golay, the order of the polynomial used to fit the data. For Butterworth, the positive filter order per pass.

  • period (float, optional) – Required for the notch filter. Period of the fringe in units of the selected coordinate.

  • quality_factor (float, optional, default: 30.0) – Notch center frequency divided by the single-pass −3 dB bandwidth. Must exceed 0.5 and give a bandwidth below the Nyquist frequency.

  • sigma (float, optional) – Required for Gaussian convolution. Standard deviation of the Gaussian.

  • sigma_units ({“points”, “coordinate”}, optional, default: “points”) – Interpret Gaussian sigma in sample points or in units of the selected coordinate. FWHM equals sqrt(8 log(2)) times sigma.

  • size (int, optional, default: 5) – The size of the filter window. size must be a positive odd integer.

  • wavelet (str, optional, default: “sym4”) – Discrete wavelet family and order, using PyWavelets names.

  • wavelet_level (int or None, optional) – Positive decomposition level. The default is the smaller of three and the maximum level for the selected wavelet and axis length.

  • wavelet_threshold (float or None, optional) – Nonnegative soft threshold in signal units. By default it is estimated separately for each spectrum from the median absolute deviation of the finest detail coefficients (MAD divided by 0.67449), multiplied by sqrt(2 log(n)), where n is the selected axis length.

Notes

Split filtering requires finite values at unmasked samples and uses the original coordinate for spacing checks and derivative scaling. Minimum segment lengths are the window size for Savitzky–Golay in interp mode, one more than the difference order for Whittaker, and the usual padding or decomposition minimum for IIR and wavelet filters, respectively. Split filtering requires real data for window filters, Savitzky–Golay, Whittaker and wavelets.

By default, Butterworth, notch, Gaussian and wavelet filters require finite, unmasked data. Butterworth, notch and coordinate-based Gaussian filtering require a uniformly spaced coordinate. Descending coordinates are supported. Butterworth and notch use odd extension with SciPy’s default padding and require more than the padding length in samples along the selected axis. The padding is three times the Butterworth order plus three, or nine points for the notch. IIR settings whose response cannot be resolved in floating point are rejected. Gaussian convolution uses the configured mode and cval. Inputs outside SciPy’s supported floating-point dtypes are converted to double precision. Overflow during conversion is rejected. The default interp selects SciPy’s reflect mode for this method.

Wavelet denoising uses symmetric extension and sample indices along the selected axis. It soft-thresholds detail coefficients and preserves the approximation coefficients. It accepts real data and computes in double precision. It requires the optional dependency installed with pip install "spectrochempy[wavelets]".

See also

smooth

Function to smooth data using various window filters.

savgol

Savitzky-Golay filter.

savgol_filter

Alias of savgol

whittaker

Whittaker-Eilers filter.

Initialize the BaseConfigurable class.

Parameters:
  • log_level (int, optional) – The log level at startup. Default is logging.WARNING.

  • **kwargs (dict) – Additional keyword arguments for configuration.

Attributes Summary

config

traitlets.config.Config object.

cutoff

Butterworth cutoff in cycles per coordinate unit.

cval

Value to fill past the edges of the input if mode is ‘constant’.

delta

The signed sample spacing passed to scipy.signal.savgol_filter.

deriv

The order of the derivative to compute in the case of the Savitzky-Golay (savgol) filter.

lamb

Smoothing/Regularization parameter.

log

Return log output.

mask_policy

Filter contiguous unmasked segments independently when set to split.

method

The filter method to be applied.

mode

The type of extension to use for the padded signal to which the filter is applied.

name

Object name

order

The order of the polynomial used to fit the datain the case of the Savitzky-Golay (savgol) filter.

period

Notch period in coordinate units.

quality_factor

Notch center frequency divided by single-pass bandwidth.

sigma

Gaussian standard deviation.

sigma_units

Interpret Gaussian sigma in points or in coordinate units.

size

The size of the filter window.size must be a positive odd integer.

wavelet

Discrete wavelet name.

wavelet_level

Discrete wavelet decomposition level.

wavelet_threshold

Soft threshold in signal units.

Methods Summary

params([default])

Return current or default configuration values.

reset()

Reset configuration parameters to their default values.

to_dict()

Return config value in a dict form.

transform(dataset[, dim])

Apply the configured filter along a dimension of the dataset.

Attributes Documentation

config#

traitlets.config.Config object.

cutoff#

Butterworth cutoff in cycles per coordinate unit.

cval#

Value to fill past the edges of the input if mode is ‘constant’.

delta#

The signed sample spacing passed to scipy.signal.savgol_filter. This is only used if deriv > 0.

When None (the default), the signed spacing is automatically derived from the coordinate of the processed axis if the retained samples are uniformly spaced, otherwise from the original coordinate if it is uniform. On a non-uniform or missing coordinate a warning is emitted and the index-based delta=1.0 is used as a fallback.

When set to a numeric value, that value is passed directly to SciPy with its sign. No unit-based correction (_reversed) is applied. For a descending coordinate, supply a negative delta if the derivative should follow the physical axis.

deriv#

The order of the derivative to compute in the case of the Savitzky-Golay (savgol) filter. This must be a non-negative integer. The default is 0, which means to filter the data without differentiating.

lamb#

Smoothing/Regularization parameter. The larger lamb, the smoother the data.

log#

Return log output.

mask_policy#

Filter contiguous unmasked segments independently when set to split.

method#

The filter method to be applied. By default, the Savitzky-Golay (savgol) filter is applied.

mode#

The type of extension to use for the padded signal to which the filter is applied.

  • When mode is ‘constant’, the padding value is given by cval.

  • When the ‘interp’ mode is selected (the default), no extension is used. Instead, a polynomial of degree order is fit to the last size values of the edges, and this polynomial is used to evaluate the last size // 2 output values.

  • When mode is ‘nearest’, the last size values are repeated.

  • When mode is ‘mirror’, the padding is created by reflecting the signal about the end of the signal.

  • When mode is ‘wrap’, the signal is wrapped around on itself to create the padding.

See scipy.signal.savgol_filter for more details on ‘mirror’, ‘constant’, ‘wrap’, and ‘nearest’.

name#

Object name

order#

The order of the polynomial used to fit the datain the case of the Savitzky-Golay (savgol) filter. order must be less than size. In the case of the Whittaker-Eilers filter, order is the difference order of the penalized least squares. For Butterworth it is the positive filter order per pass.

period#

Notch period in coordinate units.

quality_factor#

Notch center frequency divided by single-pass bandwidth.

sigma#

Gaussian standard deviation.

sigma_units#

Interpret Gaussian sigma in points or in coordinate units.

size#

The size of the filter window.size must be a positive odd integer.

wavelet#

Discrete wavelet name.

wavelet_level#

Discrete wavelet decomposition level.

wavelet_threshold#

Soft threshold in signal units.

Methods Documentation

params(default=False)[source]#

Return current or default configuration values.

Parameters:

default (bool, optional, default: False) – If default is True, the default parameters are returned, else the current values.

Returns:

dict – Current or default configuration values.

reset()[source]#

Reset configuration parameters to their default values.

to_dict()[source]#

Return config value in a dict form.

Returns:

dict – A regular dictionary.

transform(dataset, dim=-1)[source]#

Apply the configured filter along a dimension of the dataset.

Parameters:
  • dataset (NDDataset) – The dataset to filter.

  • dim (int or str, optional, default: -1) – A dimension name or an axis index.

Returns:

NDDataset – The filtered dataset.

Examples using spectrochempy.Filter

Compare spectral smoothers on noisy peaks

Compare spectral smoothers on noisy peaks

Savitzky-Golay and Whittaker-Eilers smoothing of a Raman spectrum

Savitzky-Golay and Whittaker-Eilers smoothing of a Raman spectrum

Compare smoothing window sizes on a Raman spectrum

Compare smoothing window sizes on a Raman spectrum