Initial commit
This commit is contained in:
648
skills/neurokit2/references/signal_processing.md
Normal file
648
skills/neurokit2/references/signal_processing.md
Normal file
@@ -0,0 +1,648 @@
|
||||
# General Signal Processing
|
||||
|
||||
## Overview
|
||||
|
||||
NeuroKit2 provides comprehensive signal processing utilities applicable to any time series data. These functions support filtering, transformation, peak detection, decomposition, and analysis operations that work across all signal types.
|
||||
|
||||
## Preprocessing Functions
|
||||
|
||||
### signal_filter()
|
||||
|
||||
Apply frequency-domain filtering to remove noise or isolate frequency bands.
|
||||
|
||||
```python
|
||||
filtered = nk.signal_filter(signal, sampling_rate=1000, lowcut=None, highcut=None,
|
||||
method='butterworth', order=5)
|
||||
```
|
||||
|
||||
**Filter types (via lowcut/highcut combinations):**
|
||||
|
||||
**Lowpass** (highcut only):
|
||||
```python
|
||||
lowpass = nk.signal_filter(signal, sampling_rate=1000, highcut=50)
|
||||
```
|
||||
- Removes frequencies above highcut
|
||||
- Smooths signal, removes high-frequency noise
|
||||
|
||||
**Highpass** (lowcut only):
|
||||
```python
|
||||
highpass = nk.signal_filter(signal, sampling_rate=1000, lowcut=0.5)
|
||||
```
|
||||
- Removes frequencies below lowcut
|
||||
- Removes baseline drift, DC offset
|
||||
|
||||
**Bandpass** (both lowcut and highcut):
|
||||
```python
|
||||
bandpass = nk.signal_filter(signal, sampling_rate=1000, lowcut=0.5, highcut=50)
|
||||
```
|
||||
- Retains frequencies between lowcut and highcut
|
||||
- Isolates specific frequency band
|
||||
|
||||
**Bandstop/Notch** (powerline removal):
|
||||
```python
|
||||
notch = nk.signal_filter(signal, sampling_rate=1000, method='powerline', powerline=50)
|
||||
```
|
||||
- Removes 50 or 60 Hz powerline noise
|
||||
- Narrow notch filter
|
||||
|
||||
**Methods:**
|
||||
- `'butterworth'` (default): Smooth frequency response, flat passband
|
||||
- `'bessel'`: Linear phase, minimal ringing
|
||||
- `'chebyshev1'`: Steeper rolloff, ripple in passband
|
||||
- `'chebyshev2'`: Steeper rolloff, ripple in stopband
|
||||
- `'elliptic'`: Steepest rolloff, ripple in both bands
|
||||
- `'powerline'`: Notch filter for 50/60 Hz
|
||||
|
||||
**Order parameter:**
|
||||
- Higher order: Steeper transition, more ringing
|
||||
- Lower order: Gentler transition, less ringing
|
||||
- Typical: 2-5 for physiological signals
|
||||
|
||||
### signal_sanitize()
|
||||
|
||||
Remove invalid values (NaN, inf) and optionally interpolate.
|
||||
|
||||
```python
|
||||
clean_signal = nk.signal_sanitize(signal, interpolate=True)
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
- Handle missing data points
|
||||
- Remove artifacts marked as NaN
|
||||
- Prepare signal for algorithms requiring continuous data
|
||||
|
||||
### signal_resample()
|
||||
|
||||
Change sampling rate of signal (upsample or downsample).
|
||||
|
||||
```python
|
||||
resampled = nk.signal_resample(signal, sampling_rate=1000, desired_sampling_rate=500,
|
||||
method='interpolation')
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'interpolation'`: Cubic spline interpolation
|
||||
- `'FFT'`: Frequency-domain resampling
|
||||
- `'poly'`: Polyphase filtering (best for downsampling)
|
||||
|
||||
**Use cases:**
|
||||
- Match sampling rates across multi-modal recordings
|
||||
- Reduce data size (downsample)
|
||||
- Increase temporal resolution (upsample)
|
||||
|
||||
### signal_fillmissing()
|
||||
|
||||
Interpolate missing or invalid data points.
|
||||
|
||||
```python
|
||||
filled = nk.signal_fillmissing(signal, method='linear')
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'linear'`: Linear interpolation
|
||||
- `'nearest'`: Nearest neighbor
|
||||
- `'pad'`: Forward/backward fill
|
||||
- `'cubic'`: Cubic spline
|
||||
- `'polynomial'`: Polynomial fitting
|
||||
|
||||
## Transformation Functions
|
||||
|
||||
### signal_detrend()
|
||||
|
||||
Remove slow trends from signal.
|
||||
|
||||
```python
|
||||
detrended = nk.signal_detrend(signal, method='polynomial', order=1)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'polynomial'`: Fit and subtract polynomial (order 1 = linear)
|
||||
- `'loess'`: Locally weighted regression
|
||||
- `'tarvainen2002'`: Smoothness priors detrending
|
||||
|
||||
**Use cases:**
|
||||
- Remove baseline drift
|
||||
- Stabilize mean before analysis
|
||||
- Prepare for stationarity-assuming algorithms
|
||||
|
||||
### signal_decompose()
|
||||
|
||||
Decompose signal into constituent components.
|
||||
|
||||
```python
|
||||
components = nk.signal_decompose(signal, sampling_rate=1000, method='emd')
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
|
||||
**Empirical Mode Decomposition (EMD):**
|
||||
```python
|
||||
components = nk.signal_decompose(signal, sampling_rate=1000, method='emd')
|
||||
```
|
||||
- Data-adaptive decomposition into Intrinsic Mode Functions (IMFs)
|
||||
- Each IMF represents different frequency content (high to low)
|
||||
- No predefined basis functions
|
||||
|
||||
**Singular Spectrum Analysis (SSA):**
|
||||
```python
|
||||
components = nk.signal_decompose(signal, method='ssa')
|
||||
```
|
||||
- Decomposes into trend, oscillations, and noise
|
||||
- Based on eigenvalue decomposition of trajectory matrix
|
||||
|
||||
**Wavelet decomposition:**
|
||||
- Time-frequency representation
|
||||
- Localized in both time and frequency
|
||||
|
||||
**Returns:**
|
||||
- Dictionary with component signals
|
||||
- Trend, oscillatory components, residual
|
||||
|
||||
**Use cases:**
|
||||
- Isolate physiological rhythms
|
||||
- Separate signal from noise
|
||||
- Multi-scale analysis
|
||||
|
||||
### signal_recompose()
|
||||
|
||||
Reconstruct signal from decomposed components.
|
||||
|
||||
```python
|
||||
reconstructed = nk.signal_recompose(components, indices=[1, 2, 3])
|
||||
```
|
||||
|
||||
**Use case:**
|
||||
- Selective reconstruction after decomposition
|
||||
- Remove specific IMFs or components
|
||||
- Adaptive filtering
|
||||
|
||||
### signal_binarize()
|
||||
|
||||
Convert continuous signal to binary (0/1) based on threshold.
|
||||
|
||||
```python
|
||||
binary = nk.signal_binarize(signal, method='threshold', threshold=0.5)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'threshold'`: Simple threshold
|
||||
- `'median'`: Median-based
|
||||
- `'mean'`: Mean-based
|
||||
- `'quantile'`: Percentile-based
|
||||
|
||||
**Use case:**
|
||||
- Event detection from continuous signal
|
||||
- Trigger extraction
|
||||
- State classification
|
||||
|
||||
### signal_distort()
|
||||
|
||||
Add controlled noise or artifacts for testing.
|
||||
|
||||
```python
|
||||
distorted = nk.signal_distort(signal, sampling_rate=1000, noise_amplitude=0.1,
|
||||
noise_frequency=50, artifacts_amplitude=0.5)
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `noise_amplitude`: Gaussian noise level
|
||||
- `noise_frequency`: Sinusoidal interference (e.g., powerline)
|
||||
- `artifacts_amplitude`: Random spike artifacts
|
||||
- `artifacts_number`: Number of artifacts to add
|
||||
|
||||
**Use cases:**
|
||||
- Algorithm robustness testing
|
||||
- Preprocessing method evaluation
|
||||
- Realistic data simulation
|
||||
|
||||
### signal_interpolate()
|
||||
|
||||
Interpolate signal at new time points or fill gaps.
|
||||
|
||||
```python
|
||||
interpolated = nk.signal_interpolate(x_values, y_values, x_new=None, method='quadratic')
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'linear'`, `'quadratic'`, `'cubic'`: Polynomial interpolation
|
||||
- `'nearest'`: Nearest neighbor
|
||||
- `'monotone_cubic'`: Preserves monotonicity
|
||||
|
||||
**Use case:**
|
||||
- Convert irregular samples to regular grid
|
||||
- Upsample for visualization
|
||||
- Align signals with different time bases
|
||||
|
||||
### signal_merge()
|
||||
|
||||
Combine multiple signals with different sampling rates.
|
||||
|
||||
```python
|
||||
merged = nk.signal_merge(signal1, signal2, time1=None, time2=None, sampling_rate=None)
|
||||
```
|
||||
|
||||
**Use case:**
|
||||
- Multi-modal signal integration
|
||||
- Combine data from different devices
|
||||
- Synchronize based on timestamps
|
||||
|
||||
### signal_flatline()
|
||||
|
||||
Identify periods of constant signal (artifacts or sensor failure).
|
||||
|
||||
```python
|
||||
flatline_mask = nk.signal_flatline(signal, duration=5.0, sampling_rate=1000)
|
||||
```
|
||||
|
||||
**Returns:**
|
||||
- Binary mask where True indicates flatline periods
|
||||
- Duration threshold prevents false positives from normal stability
|
||||
|
||||
### signal_noise()
|
||||
|
||||
Add various types of noise to signal.
|
||||
|
||||
```python
|
||||
noisy = nk.signal_noise(signal, sampling_rate=1000, noise_type='gaussian',
|
||||
amplitude=0.1)
|
||||
```
|
||||
|
||||
**Noise types:**
|
||||
- `'gaussian'`: White noise
|
||||
- `'pink'`: 1/f noise (common in physiological signals)
|
||||
- `'brown'`: Brownian (random walk)
|
||||
- `'powerline'`: Sinusoidal interference (50/60 Hz)
|
||||
|
||||
### signal_surrogate()
|
||||
|
||||
Generate surrogate signals preserving certain properties.
|
||||
|
||||
```python
|
||||
surrogate = nk.signal_surrogate(signal, method='IAAFT')
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'IAAFT'`: Iterated Amplitude Adjusted Fourier Transform
|
||||
- Preserves amplitude distribution and power spectrum
|
||||
- `'random_shuffle'`: Random permutation (null hypothesis testing)
|
||||
|
||||
**Use case:**
|
||||
- Nonlinearity testing
|
||||
- Null hypothesis generation for statistical tests
|
||||
|
||||
## Peak Detection and Correction
|
||||
|
||||
### signal_findpeaks()
|
||||
|
||||
Detect local maxima (peaks) in signal.
|
||||
|
||||
```python
|
||||
peaks_dict = nk.signal_findpeaks(signal, height_min=None, height_max=None,
|
||||
relative_height_min=None, relative_height_max=None)
|
||||
```
|
||||
|
||||
**Key parameters:**
|
||||
- `height_min/max`: Absolute amplitude thresholds
|
||||
- `relative_height_min/max`: Relative to signal range (0-1)
|
||||
- `threshold`: Minimum prominence
|
||||
- `distance`: Minimum samples between peaks
|
||||
|
||||
**Returns:**
|
||||
- Dictionary with:
|
||||
- `'Peaks'`: Peak indices
|
||||
- `'Height'`: Peak amplitudes
|
||||
- `'Distance'`: Inter-peak intervals
|
||||
|
||||
**Use cases:**
|
||||
- Generic peak detection for any signal
|
||||
- R-peaks, respiratory peaks, pulse peaks
|
||||
- Event detection
|
||||
|
||||
### signal_fixpeaks()
|
||||
|
||||
Correct detected peaks for artifacts and anomalies.
|
||||
|
||||
```python
|
||||
corrected = nk.signal_fixpeaks(peaks, sampling_rate=1000, iterative=True,
|
||||
method='Kubios', interval_min=None, interval_max=None)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'Kubios'`: Kubios HRV software method (default)
|
||||
- `'Malik1996'`: Task Force Standards (1996)
|
||||
- `'Kamath1993'`: Kamath's approach
|
||||
|
||||
**Corrections:**
|
||||
- Remove physiologically implausible intervals
|
||||
- Interpolate missing peaks
|
||||
- Remove extra detected peaks (duplicates)
|
||||
|
||||
**Use case:**
|
||||
- Artifact correction in R-R intervals
|
||||
- Improve HRV analysis quality
|
||||
- Respiratory or pulse peak correction
|
||||
|
||||
## Analysis Functions
|
||||
|
||||
### signal_rate()
|
||||
|
||||
Compute instantaneous rate from event occurrences (peaks).
|
||||
|
||||
```python
|
||||
rate = nk.signal_rate(peaks, sampling_rate=1000, desired_length=None)
|
||||
```
|
||||
|
||||
**Method:**
|
||||
- Calculate inter-event intervals
|
||||
- Convert to events per minute
|
||||
- Interpolate to match desired length
|
||||
|
||||
**Use case:**
|
||||
- Heart rate from R-peaks
|
||||
- Breathing rate from respiratory peaks
|
||||
- Any periodic event rate
|
||||
|
||||
### signal_period()
|
||||
|
||||
Find dominant period/frequency in signal.
|
||||
|
||||
```python
|
||||
period = nk.signal_period(signal, sampling_rate=1000, method='autocorrelation',
|
||||
show=False)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'autocorrelation'`: Peak in autocorrelation function
|
||||
- `'powerspectraldensity'`: Peak in frequency spectrum
|
||||
|
||||
**Returns:**
|
||||
- Period in samples or seconds
|
||||
- Frequency (1/period) in Hz
|
||||
|
||||
**Use case:**
|
||||
- Detect dominant rhythm
|
||||
- Estimate fundamental frequency
|
||||
- Breathing rate, heart rate estimation
|
||||
|
||||
### signal_phase()
|
||||
|
||||
Compute instantaneous phase of signal.
|
||||
|
||||
```python
|
||||
phase = nk.signal_phase(signal, method='hilbert')
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'hilbert'`: Hilbert transform (analytic signal)
|
||||
- `'wavelet'`: Wavelet-based phase
|
||||
|
||||
**Returns:**
|
||||
- Phase in radians (-π to π) or 0 to 1 (normalized)
|
||||
|
||||
**Use cases:**
|
||||
- Phase-locked analysis
|
||||
- Synchronization measures
|
||||
- Phase-amplitude coupling
|
||||
|
||||
### signal_psd()
|
||||
|
||||
Compute Power Spectral Density.
|
||||
|
||||
```python
|
||||
psd, freqs = nk.signal_psd(signal, sampling_rate=1000, method='welch',
|
||||
max_frequency=None, show=False)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'welch'`: Welch's periodogram (windowed FFT, default)
|
||||
- `'multitapers'`: Multitaper method (superior spectral estimation)
|
||||
- `'lomb'`: Lomb-Scargle (unevenly sampled data)
|
||||
- `'burg'`: Autoregressive (parametric)
|
||||
|
||||
**Returns:**
|
||||
- `psd`: Power at each frequency (units²/Hz)
|
||||
- `freqs`: Frequency bins (Hz)
|
||||
|
||||
**Use case:**
|
||||
- Frequency content analysis
|
||||
- HRV frequency domain
|
||||
- Spectral signatures
|
||||
|
||||
### signal_power()
|
||||
|
||||
Compute power in specific frequency bands.
|
||||
|
||||
```python
|
||||
power_dict = nk.signal_power(signal, sampling_rate=1000, frequency_bands={
|
||||
'VLF': (0.003, 0.04),
|
||||
'LF': (0.04, 0.15),
|
||||
'HF': (0.15, 0.4)
|
||||
}, method='welch')
|
||||
```
|
||||
|
||||
**Returns:**
|
||||
- Dictionary with absolute and relative power per band
|
||||
- Peak frequencies
|
||||
|
||||
**Use case:**
|
||||
- HRV frequency analysis
|
||||
- EEG band power
|
||||
- Rhythm quantification
|
||||
|
||||
### signal_autocor()
|
||||
|
||||
Compute autocorrelation function.
|
||||
|
||||
```python
|
||||
autocorr = nk.signal_autocor(signal, lag=1000, show=False)
|
||||
```
|
||||
|
||||
**Interpretation:**
|
||||
- High autocorrelation at lag: signal repeats every lag samples
|
||||
- Periodic signals: peaks at multiples of period
|
||||
- Random signals: rapid decay to zero
|
||||
|
||||
**Use cases:**
|
||||
- Detect periodicity
|
||||
- Assess temporal structure
|
||||
- Memory in signal
|
||||
|
||||
### signal_zerocrossings()
|
||||
|
||||
Count zero crossings (sign changes).
|
||||
|
||||
```python
|
||||
n_crossings = nk.signal_zerocrossings(signal)
|
||||
```
|
||||
|
||||
**Interpretation:**
|
||||
- More crossings: higher frequency content
|
||||
- Related to dominant frequency (rough estimate)
|
||||
|
||||
**Use case:**
|
||||
- Simple frequency estimation
|
||||
- Signal regularity assessment
|
||||
|
||||
### signal_changepoints()
|
||||
|
||||
Detect abrupt changes in signal properties (mean, variance).
|
||||
|
||||
```python
|
||||
changepoints = nk.signal_changepoints(signal, penalty=10, method='pelt', show=False)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'pelt'`: Pruned Exact Linear Time (fast, exact)
|
||||
- `'binseg'`: Binary segmentation (faster, approximate)
|
||||
|
||||
**Parameters:**
|
||||
- `penalty`: Controls sensitivity (higher = fewer changepoints)
|
||||
|
||||
**Returns:**
|
||||
- Indices of detected changepoints
|
||||
- Segments between changepoints
|
||||
|
||||
**Use cases:**
|
||||
- Segment signal into states
|
||||
- Detect transitions (e.g., sleep stages, arousal states)
|
||||
- Automatic epoch definition
|
||||
|
||||
### signal_synchrony()
|
||||
|
||||
Assess synchronization between two signals.
|
||||
|
||||
```python
|
||||
sync = nk.signal_synchrony(signal1, signal2, method='correlation')
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'correlation'`: Pearson correlation
|
||||
- `'coherence'`: Frequency-domain coherence
|
||||
- `'mutual_information'`: Information-theoretic measure
|
||||
- `'phase'`: Phase locking value
|
||||
|
||||
**Use cases:**
|
||||
- Heart-brain coupling
|
||||
- Inter-brain synchrony
|
||||
- Multi-channel coordination
|
||||
|
||||
### signal_smooth()
|
||||
|
||||
Apply smoothing to reduce noise.
|
||||
|
||||
```python
|
||||
smoothed = nk.signal_smooth(signal, method='convolution', kernel='boxzen', size=10)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'convolution'`: Apply kernel (boxcar, Gaussian, etc.)
|
||||
- `'median'`: Median filter (robust to outliers)
|
||||
- `'savgol'`: Savitzky-Golay filter (preserves peaks)
|
||||
- `'loess'`: Locally weighted regression
|
||||
|
||||
**Kernel types (for convolution):**
|
||||
- `'boxcar'`: Simple moving average
|
||||
- `'gaussian'`: Gaussian-weighted average
|
||||
- `'hann'`, `'hamming'`, `'blackman'`: Windowing functions
|
||||
|
||||
**Use cases:**
|
||||
- Noise reduction
|
||||
- Trend extraction
|
||||
- Visualization enhancement
|
||||
|
||||
### signal_timefrequency()
|
||||
|
||||
Time-frequency representation (spectrogram).
|
||||
|
||||
```python
|
||||
tf, time, freq = nk.signal_timefrequency(signal, sampling_rate=1000, method='stft',
|
||||
max_frequency=50, show=False)
|
||||
```
|
||||
|
||||
**Methods:**
|
||||
- `'stft'`: Short-Time Fourier Transform
|
||||
- `'cwt'`: Continuous Wavelet Transform
|
||||
|
||||
**Returns:**
|
||||
- `tf`: Time-frequency matrix (power at each time-frequency point)
|
||||
- `time`: Time bins
|
||||
- `freq`: Frequency bins
|
||||
|
||||
**Use cases:**
|
||||
- Non-stationary signal analysis
|
||||
- Time-varying frequency content
|
||||
- EEG/MEG time-frequency analysis
|
||||
|
||||
## Simulation
|
||||
|
||||
### signal_simulate()
|
||||
|
||||
Generate various synthetic signals for testing.
|
||||
|
||||
```python
|
||||
signal = nk.signal_simulate(duration=10, sampling_rate=1000, frequency=[5, 10],
|
||||
amplitude=[1.0, 0.5], noise=0.1)
|
||||
```
|
||||
|
||||
**Signal types:**
|
||||
- Sinusoidal oscillations (specify frequencies)
|
||||
- Multiple frequency components
|
||||
- Gaussian noise
|
||||
- Combinations
|
||||
|
||||
**Use cases:**
|
||||
- Algorithm testing
|
||||
- Method validation
|
||||
- Educational demonstrations
|
||||
|
||||
## Visualization
|
||||
|
||||
### signal_plot()
|
||||
|
||||
Visualize signal and optional markers.
|
||||
|
||||
```python
|
||||
nk.signal_plot(signal, sampling_rate=1000, peaks=None, show=True)
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Time axis in seconds
|
||||
- Peak markers
|
||||
- Multiple subplots for signal arrays
|
||||
|
||||
## Practical Tips
|
||||
|
||||
**Choosing filter parameters:**
|
||||
- **Lowcut**: Set below lowest frequency of interest
|
||||
- **Highcut**: Set above highest frequency of interest
|
||||
- **Order**: Start with 2-5, increase if transition too slow
|
||||
- **Method**: Butterworth is safe default
|
||||
|
||||
**Handling edge effects:**
|
||||
- Filtering introduces artifacts at signal edges
|
||||
- Pad signal before filtering, then trim
|
||||
- Or discard initial/final seconds
|
||||
|
||||
**Dealing with gaps:**
|
||||
- Small gaps: `signal_fillmissing()` with interpolation
|
||||
- Large gaps: Segment signal, analyze separately
|
||||
- Mark gaps as NaN, use interpolation carefully
|
||||
|
||||
**Combining operations:**
|
||||
```python
|
||||
# Typical preprocessing pipeline
|
||||
signal = nk.signal_sanitize(raw_signal) # Remove invalid values
|
||||
signal = nk.signal_filter(signal, sampling_rate=1000, lowcut=0.5, highcut=40) # Bandpass
|
||||
signal = nk.signal_detrend(signal, method='polynomial', order=1) # Remove linear trend
|
||||
```
|
||||
|
||||
**Performance considerations:**
|
||||
- Filtering: FFT-based methods faster for long signals
|
||||
- Resampling: Downsample early in pipeline to speed up
|
||||
- Large datasets: Process in chunks if memory-limited
|
||||
|
||||
## References
|
||||
|
||||
- Virtanen, P., et al. (2020). SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature methods, 17(3), 261-272.
|
||||
- Tarvainen, M. P., Ranta-aho, P. O., & Karjalainen, P. A. (2002). An advanced detrending method with application to HRV analysis. IEEE Transactions on Biomedical Engineering, 49(2), 172-175.
|
||||
- Huang, N. E., et al. (1998). The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society of London A, 454(1971), 903-995.
|
||||
Reference in New Issue
Block a user