WO2011106527A2 - Procédé de filtrage du bruit fondé sur l'évaluation d'un intervalle de confiance - Google Patents

Procédé de filtrage du bruit fondé sur l'évaluation d'un intervalle de confiance Download PDF

Info

Publication number
WO2011106527A2
WO2011106527A2 PCT/US2011/026078 US2011026078W WO2011106527A2 WO 2011106527 A2 WO2011106527 A2 WO 2011106527A2 US 2011026078 W US2011026078 W US 2011026078W WO 2011106527 A2 WO2011106527 A2 WO 2011106527A2
Authority
WO
WIPO (PCT)
Prior art keywords
approximation
confidence interval
signal
noise
data array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2011/026078
Other languages
English (en)
Other versions
WO2011106527A3 (fr
Inventor
Yuri Kalambet
Sergey Maltsev
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
AMPERSAND INTERNATIONAL Inc
Original Assignee
AMPERSAND INTERNATIONAL Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by AMPERSAND INTERNATIONAL Inc filed Critical AMPERSAND INTERNATIONAL Inc
Publication of WO2011106527A2 publication Critical patent/WO2011106527A2/fr
Publication of WO2011106527A3 publication Critical patent/WO2011106527A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N30/00—Investigating or analysing materials by separation into components using adsorption, absorption or similar phenomena or using ion-exchange, e.g. chromatography or field flow fractionation
    • G01N30/02—Column chromatography
    • G01N30/26—Conditioning of the fluid carrier; Flow patterns
    • G01N30/28—Control of physical parameters of the fluid carrier
    • G01N30/32—Control of physical parameters of the fluid carrier of pressure or speed
    • G01N2030/324—Control of physical parameters of the fluid carrier of pressure or speed speed, flow rate
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N30/00—Investigating or analysing materials by separation into components using adsorption, absorption or similar phenomena or using ion-exchange, e.g. chromatography or field flow fractionation
    • G01N30/02—Column chromatography
    • G01N30/86—Signal analysis
    • G01N30/8603—Signal analysis with integration or differentiation
    • G01N30/8617—Filtering, e.g. Fourier filtering
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N30/00—Investigating or analysing materials by separation into components using adsorption, absorption or similar phenomena or using ion-exchange, e.g. chromatography or field flow fractionation
    • G01N30/02—Column chromatography
    • G01N30/86—Signal analysis
    • G01N30/8624—Detection of slopes or peaks; baseline correction
    • G01N30/8644—Data segmentation, e.g. time windows

Definitions

  • Measured signals are comprised of a signal portion and a random error portion, which may be caused by a variety of factors, such as from electronic components, a variation of surrounding ambient conditions, and/or radio interference. It may be beneficial to minimize the error portion of the signal as much as possible in order to achieve an accurate estimate of the measured signal.
  • Such noise reduction techniques may be based on polynomial approximation performed by linear smoothers (e.g., Savitzky- Golay (SG), Fourier transform based convolution, convolution with Gaussian, and/or moving average) or a non-linear noise manner (e.g., median filtering).
  • linear smoothers e.g., Savitzky- Golay (SG), Fourier transform based convolution, convolution with Gaussian, and/or moving average
  • a non-linear noise manner e.g., median filtering
  • results based on traditional noise filtering in chromatography or capillary electrophoresis may produce shape distortion and/or disturbances, which may be especially pronounced near sharp edges, baseline steps, and/or triangular peaks.
  • One or more methods, techniques, and/or systems are disclosed for a family of noise filtering methods based on a variety of approximating functions.
  • the confidence interval evaluation noise filtering is implemented by way of a polynomial filter, wherein a signal comprising an array or vector of original signal values corresponding to points within the signal may be evaluated using the confidence interval evaluation noise filtering technique.
  • the polynomial may have a fixed-degree, while other embodiments may comprise varying degree polynomials.
  • approximation polynomial is constructed using least-square method and approximated values of a signal may be evaluated as a set of points within an approximation window of the signal, along with a corresponding confidence interval.
  • the technique(s) presented herein involve re-evaluating the set of approximated values when the approximation window shifts by a point, along with new confidence intervals for the corresponding points.
  • An approximated value for the signal may be compared with the corresponding value from the output array of values, and depending on the confidence interval for that corresponding point, the approximated value may replace the stored estimate of signal value or alternatively, the stored approximation may be kept.
  • the approximated value of the signal replaces the original signal value within the array or vector if the confidence interval of the new signal value is lower than the current confidence interval.
  • confidence interval evaluation based noise filtering may be based at least in part on a least squares approximation and measurement signals, comprising objects that may be recoverable from noise by calculations utilizing confidence intervals.
  • Confidence interval evaluation noise filtering may be repeated iteratively, by repeatedly selecting different sets of approximating functions to improve overall accuracy. Therefore, confidence filtering provides desired (e.g., minimal) confidence intervals for respective approximation points.
  • polynomials may be accepted or rejected based on the standard deviation ⁇ of the noise portion of the signal measurement. For example, a polynomial exhibiting a low probability of fitting approximated data properly and/or accurately may be rejected due to poor modeling, and other polynomials may be selected for improved confidence interval estimates.
  • confidence interval evaluation based noise filtering may be applied to a variety of measurement scenarios, including but not limited to the fields of analytical chemistry, chromatography, spectrometry, and/or image processing.
  • Figure 1 is an illustration of a flow diagram illustrating an example embodiment where one or more techniques described herein may be implemented.
  • Figure 2 is an illustration of a flow diagram illustrating an example embodiment where one or more techniques described herein may be implemented.
  • Figure 3 is an illustration of a flow diagram illustrating an example embodiment where one or more techniques described herein may be implemented.
  • Figure 4 is an illustration of an example comparison between SG and confidence interval filtering.
  • Figure 5 is an illustration of an example approximation for the G ? value of Figure 4.
  • Figure 6 is an illustration of an example approximation for the G 2 value of Figure 4.
  • Figure 7 is an illustration of an example comparison between SG and confidence interval filtering as applied to electrophoresis data.
  • Figure 8 is an illustration of an example chromatographic analysis.
  • Figure 9 is an illustration of an example comparison between SG and confidence interval filtering.
  • Figure 10 is an illustration of an example comparison between SG and confidence interval filtering.
  • Figure 1 1 is an illustration of an example comparison between SG and confidence interval filtering.
  • Figure 12 is an illustration of an example comparison between SG and confidence interval filtering.
  • Figure 13 is an illustration of an example series of peaks.
  • Figure 14 is an illustration of an example comparison between a filtered signal and a pure signal.
  • Figure 15 is an illustration of an example comparison between SG and confidence interval filtering as applied to electromyography (EMG).
  • Figure 16 is an illustration of an example of pump pulsations.
  • Figure 17 is an illustration of an example comparison between SG and confidence interval filtering as applied to capillary electrophoresis (CE).
  • Figure 18 is an illustration of an exemplary computing environment wherein one or more of the provisions set forth herein may be implemented.
  • Figure 19 is an illustration of an exemplary computer-readable medium comprising processor-executable instructions configured to embody one or more of the provisions set forth herein.
  • One or more methods, techniques, and/or systems are disclosed for calculating an approximated value for a signal based on adaptive
  • Approximating functions are selected using criterion based on desired (e.g., minimal) confidence intervals of the approximation.
  • desired e.g., minimal
  • one such function capable of confidence interval calculation is a polynomial approximation using the least squares method, as will be discussed in greater detail herein.
  • Figure 7 demonstrate exemplary advantages of confidence interval filters by applying a third-degree adaptive polynomial filter to a set of chromatographic data and comparing the results with the popular Savitzky-Golay (SG) filtering method using a polynomial of the third degree. Therefore, the following comparisons, illustrations, and figures are based on confidence interval evaluation noise filtering compared with the SG filtering method, which is frequently used and currently considered one of the most reliable methods available.
  • Confidence interval evaluation based noise filtering has several advantages over other methods of noise filtering, and among others, an automatic adaptive reduction of the approximation window for accurate approximation of an object (e.g., a peak or a baseline region). Therefore, the shape of the peak within a signal does not suffer when a confidence interval filter is applied by comparison to other noise filtering methods.
  • Confidence interval filtering effectively suppresses baseline noise, significantly improves the detection and quantification limits, and even allows for the suppression of non-white noise (e.g., pump pulsations discussed in greater detail below, chemical noise, etc.) Further, confidence interval evaluation provides the advantages of non-central approximations of the signal in close proximity to object boundaries (e.g., a baseline step or a peak upslope). Improper approximation for known noise levels may also be avoided by using confidence interval evaluation due to the fact that noise filtering achieved through confidence interval evaluation may be based on the premise that the quality of a polynomial approximation may not be uniform through an analyzed signal over the entire range of the signal. According to one aspect, statistical methods and regression analysis enable a definition of a numerical evaluation of the quality of the approximation. Thus, a natural evaluation of the quality of an approximation is a confidence interval for an approximated value.
  • noise filtering based at least in part on confidence interval evaluation may pertain to a variety of measurement methods and systems, including but not limited to measurements related to analytical chemistry, chromatography, spectrometry, combined techniques, and/or image processing. A number of other applications or examples may be readily imagined (e.g., accurate handling of an object's boundaries in image processing).
  • confidence interval filtering may be applied to isocratic chromatography, where the peaks at the beginning are narrow, while the peaks tend to broaden toward the end of the chromatogram. Conventional techniques may make it difficult to filter such a signal correctly, since a small gap on a conventional filter produces low noise suppression, while a bigger gap produces distortions of narrow peaks.
  • adaptive confidence filters may resolve objects of various widths or different frequencies in a Fourier domain which may be present simultaneously in a signal.
  • the following example is based on single dimensional data (e.g., a chromatographic signal) and homoscedastic noise.
  • single dimensional data e.g., a chromatographic signal
  • homoscedastic noise e.g., a chromatographic signal
  • the techniques described herein may also be applied to two or multi-dimensional data filtering (e.g., image processing). Further, this single dimensional approach may be repeated using different single- dimensional subsets of multi-dimensional data arrays as desired.
  • the input array comprises an array of raw data, while the elements of the output array comprise an approximated data point and an estimate of the confidence interval corresponding to the point. Accordingly, definitions for variables and equations are outlined below:
  • X represents a matrix of values (as shown above) which runs along an independent axis (e.g., a time axis or a position axis).
  • C y represents the calculated confidence interval with respect to a corresponding point from the vector of detected response values, or in other words, a corresponding point for the measured signal. Further, this confidence interval is based on homoscedastic noise rather than
  • the raw data may be scaled to avoid complications. For example, one may take the square root of a signal before smoothing, and use square of the value in output array result as an estimate of the initial signal.
  • the confidence interval may be calculated for any x * within an approximated range for a given polynomial approximation (while SG relies on a central point). Further, original data is not required to be uniformly distributed along the position axis in order to calculate a confidence interval. Therefore, the positioning of x * for the approximated value is not required to coincide with a point where initial data was defined. Confidence intervals techniques may smooth irregular data accordingly and enable re-sampling of regular and irregular data. t m s
  • the Student's coefficient for confidence level (1 - ⁇ 5), where m represents the number of degrees of freedom and ⁇ represents a P-value.
  • n the number of data points used for the polynomial approximation (e.g., the gap of the filter)
  • an estimate S 2 of the population noise variance ⁇ 2 may be obtained from a local subset of n points.
  • the population noise variance ⁇ 2 may be represented as follows: where y, is a "true" signal value at position / ' .
  • Formula (a) may be applied, while if S 2 is greater than or equal to ⁇ 2 , Formula (b) may be used.
  • the noise variance may also provide the benefit of evaluating the quality of an approximation model.
  • an approximation model is correct, a larger number of points n in an approximation window may result in an estimate of variance S 2 which is closer to the true value of ⁇ 2 .
  • n-p white homoscedastic noise values
  • the polynomial may be treated as having a systematic error (which may include but is not limited to: an excessive number of points used to construct the polynomial, an insufficient power for the polynomial (e.g., the degree of the polynomial is too small), or a polynomial not fit for approximating the data).
  • a systematic error which may include but is not limited to: an excessive number of points used to construct the polynomial, an insufficient power for the polynomial (e.g., the degree of the polynomial is too small), or a polynomial not fit for approximating the data.
  • X (CC, m) is a distribution function
  • a represents the P-value
  • n the number of degrees of freedom.
  • An estimate of ⁇ 2 may be made using a data array, measured with a constant data rate, although alternative algorithms may be used in other cases.
  • a robust method of noise estimation may be selected by defining a width of the noise window n and a degree of the polynomial p- 1, Noise Definition Polynomial (NDP) to approximate the data.
  • the data array may be approximated numerous times, starting from the beginning of the array, and repeated when the approximation window is shifted to higher index points n/3. Likewise, at the respective approximation positions, the residual sum of squares RSS may be calculated and stored to the vector E. In one
  • vector E may be used for estimating average RSS values by way of iterative look-through passes.
  • an average of values in E may be calculated, and the first estimate for ⁇ 0 2 may be calculated as follows:
  • the outlier rejection process may be repeated until ⁇ 2 stops changing, but generally no more than 5 times, as the procedure effectively rejects all outliers originating from regions with a poor approximation of the data (e.g., baseline steps, jumps, and/or sharp peaks caused by sample injections). Therefore, an estimate of ⁇ 2 may be obtained in the last step of iterations, and accepted as the noise ⁇ 2 for the entire array.
  • the filtering and re-sampling functions are possible depending on the nature of the analyzed signal.
  • Techniques may be implemented where variations of the gap (n), power of the polynomial (p), and the non-central approximation are combined in different ways.
  • the gap (n) and power of the polynomial (p) are fixed and optimal approximations may be chosen from all possible non-central approximations for a given position x coast.
  • the power of the polynomial (p) is fixed while various sets of gaps (n) are used.
  • An optimal approximation is selected from all possible non-central approximations and from all gaps in the set for a given position x coast.
  • the gap (n) is fixed while power of the polynomial (p) may be varied within a range of [1 , p].
  • Optimal approximations may be chosen from possible non-central approximations and from all the powers for a given position x.
  • Figure 4 illustrates an example where gap (n) and power (p) are fixed, and the signal comprises a baseline, a section of a peak, and a steep slope.
  • Figures 5 and 6 illustrate approximations at G- and G 2 for Figure 4, where confidence intervals at a 0.95 confidence probability level are used.
  • Approximations at Gi are based on conventional central SG filtering, while approximations at G 2 are non-central. This gives a better approximation closer to the pure signal while maintaining a confidence interval less than the one associated with the central approximation.
  • non-central approximation G 2 produces efficient results for noise suppression analogous to a pure baseline.
  • Central approximation G ? produces a significant amount of distortion.
  • confidence intervals for both cases provide the criteria to decide which approximation may be better.
  • the confidence interval calculations may be applied to various noise reduction tasks. Therefore, any subsets of points comprising the raw data array may be approximated by a polynomial. Multiple estimates of the values and confidence intervals at the target position may be made using different data subsets (e.g., different sizes, widths, and/or positions for the approximation window). Additionally, multiple estimates may be calculated using varying degrees for the polynomials.
  • confidence filtering may be implemented in a method 1 00 based on the following description.
  • ⁇ 2 is evaluated for the noise distribution using the data array (alternatively, the evaluation may be optional if noise distribution parameters are known in advance or are being ignored).
  • a list of polynomial construction parameters (e.g., elements) to be applied is defined where the respective elements within the list comprise a polynomial degree and a subset of input data used to build a polynomial or data subset construction rule (e.g., window width, position, size).
  • the data is approximated based at least in part on data within the approximation window by the polynomial according to parameters of the respective list elements.
  • the RSS is evaluated based on the following conditions 1 1 0: Noise Not Ignored
  • a new confidence interval is calculated 1 12 based on one or more of the following formulae: t
  • the new confidence intervals are compared 1 14 with the values at the corresponding output array.
  • the new confidence interval and the corresponding approximation point may replace the old value stored in the output array if the condition that the new confidence interval is smaller than the old value is satisfied. Approximations may be made until each element of the list has been run 1 16. At 1 18 positions which were not evaluated are treated as outliers and data around these positions is re-evaluated. For example, if the noise is not
  • a portion of the input data array may not be analyzed. Additionally output data may be filled with estimates based on revised noise parameters or other principles.
  • approximations may be rejected based on contradictions with a priori information related to the raw data array (e.g., noise level). Therefore, the approximation comprising the window size, position, and polynomial degree with the desired (e.g., minimal) confidence interval may be accepted as a correct approximation for a target position.
  • a priori information related to the raw data array e.g., noise level
  • confidence intervals may be extended to data of higher dimensions (e.g., 2-D, by approximation of smoothed data in a different dimension).
  • the heteroscedastic noise model may be a benefit for the second dimension since regression may gain additional accuracy by accounting for different confidence intervals of data points.
  • the following example demonstrates chromatographic data measured with a constant data rate, where x is an integer (e.g., the index of data array values).
  • x is an integer (e.g., the index of data array values).
  • Non-central approximations may be especially effective in chromatographic data analysis where detector wavelengths or sensitivities may be switched between measurements.
  • re-sampling is not considered, and therefore the smoothed values of measured data points may be calculated, while the response values for non-integer x values may not be calculated.
  • Figure 3 illustrates an exemplary method 300, which begins at 302 and involves executing the acts of 304 on the processor instructions configured to perform the techniques presented herein. It will be appreciated the acts of exemplary method 300 may be based at least in part on the formulae presented herein ⁇ e.g., Variable Definitions, Confidence Interval Formulae). Exemplary method 300 demonstrates approximating a set of one or more approximated values for a signal based at least in part on receiving a signal comprising noise and an input data array of three or more signal values which correspond to a set of three or more points, the points comprising an independent X coordinate and a measured Y coordinate, as shown at 302.
  • the signal may be scaled with a scaling function and/or an inverse scaling function. Additionally, receiving the signal 302 may comprise sorting the input data array in an ascending order based at least in part on at least one of the X coordinates.
  • receiving the signal 302 may comprise sorting the input data array in an ascending order based at least in part on at least one of the X coordinates.
  • an output data array is approximated, comprising a number of elements based at least in part on a selected number of targets to approximate, the respective elements comprising an approximation and a confidence interval associated with the respective approximation, the approximation comprising an X coordinate and a Y coordinate, the number of targets defined by a nature of a task.
  • the subset of the input data array of 304 may be selected based at least in part on a defined number (approximation window, n) of adjacent input points. Further 304 may comprise a confidence level based at least in part on a user-defined parameter of the algorithm.
  • the instructions comprise calculating 306 at least a first set of respective approximations based at least in part on a subset of the input data array, one or more approximation functions (e.g., based on a least squares approach or a polynomial function).
  • the instructions may comprise re-calculating 308 at least a second set of respective
  • RSS residual sum of squares
  • (n-p) represents a number of degrees of freedom
  • p represents a number of functions used for approximation
  • n an approximation window
  • the exemplary method 300 may be implemented as a set of instructions stored on a memory component (such as a memory circuit, a platter of a hard disk drive, a solid-state storage device, or a magnetic or optical disc) of a device, where such instructions are configured to, when executed on a processor of the device, perform the elements of the techniques presented herein.
  • a memory component such as a memory circuit, a platter of a hard disk drive, a solid-state storage device, or a magnetic or optical disc
  • the exemplary method 300 may be implemented as a set of hardware components comprising a system, such as logic gates of an electronic circuit or
  • a smoothing method may comprise an input signal array (e.g., input data set), each element of the array having two coordinates: X (independent coordinate) and Y (result of measurement, dependent coordinate); an output signal array with at least one element, each element having two coordinates (X and Y similar to input array) and a confidence value associated with every element of the output array, X coordinates (target positions) of the input array are defined in advance by the nature of the approximation task; and a set of one or more procedures capable of producing an output signal value for a target position and its associated confidence interval value from the subset of values in said input signal array. Multiple (e.g., at least two) approximations using different functions or different input data subsets should be made and the correct approximation is selected using comparison of confidence intervals.
  • Figure 7 illustrates an example where original raw data is graphed using a dotted line, while SG filtering is shown as the thinner solid line, and an adaptive confidence interval non-central approximation filter is shown with the thicker solid line.
  • the filter gap is the same in both the SG and confidence interval case and equals 41 points.
  • the use of the SG filter with a reasonable gap produces artifacts which are difficult to handle.
  • the SG filter distorts peaks near steps and cannot be used to process the signal correctly. At the second peak of Figure 7, the distortion becomes especially
  • confidence interval noise filters with non-central approximations may provide more accurate results, especially for data comprising sudden, abrupt changes of signal levels.
  • a window size may be based on one or more locations of signal characteristics.
  • a maximum window size may be selected to avoid hiding small objects. Smaller windows may provide better estimates for steep slopes, steps, and/or jumps, while bigger windows may provide better noise reduction levels for lengthy baseline regions.
  • optimal window width may depend on the point neighborhood. Therefore, the larger the distance to the nearest peak, the wider the approximation window can be. Fit quality criteria based on noise estimates may also be applied to avoid effects of accidental good fits for small approximation windows and peak suppression for wider windows.
  • Filtering may be performed with the noise definition window (NDW) with a maximal gap G 0 .
  • NDW noise definition window
  • three steps are performed upwards and three downwards, increasing overall window width range to 8:
  • Figure 8 illustrates two peaks of different heights to simulate chromatographic analysis.
  • the model peaks have a shape of exponentially- modified Gaussian function:
  • t is a time or position coordinate
  • T is a parameter which defines the position of the peak
  • ⁇ and ⁇ are parameters which define width and asymmetry of the peak, respectively.
  • n 1 and n 2 are height of the first and second peaks, respectively;
  • Savitzky-Golay filter may be selected so that distortion for the second, higher peak is approximately twice as large as the background noise range at the baseline. For this sampling rate, the gap selected by this condition appeared to be equal to 31 . Thus, the maximal G 0 for the adaptive filter which produces approximately the same maximal distortion near the peak top may be found.
  • a series of reference signals with different noise samples may be analyzed and when G 0 is about 161 , the distortion of the highest peak is approximately the same as for previously used conventional SG filter, as illustrated by Figure 9.
  • Figure 10 illustrates an example which makes the comparison between the filtered and pure signals more apparent.
  • Both the SG and confidence filters produce approximately the same distortion for the second (higher) peak at the quantification limit.
  • the adaptive confidence filter may produce bigger distortions than the conventional Savitzky-Golay filter when large G 0 is used, although these distortions are much smaller than those that would be caused by SG with gap G 0 .
  • the difference e.g., signal distortion
  • dSG31 dotted line
  • dAGSG solid line
  • the distortion varies very little as G 0 varies from 141 to 241 . This is natural since, in all cases, the adaptive algorithm chooses an appropriate filtering gap for the top of the peak, as shown at Figure 1 1 . Meanwhile, the maximum gap is selected for noise suppression at the flat baseline piece of the signal. This produces the appropriate noise suppression. Gap values are selected based on best approximations (e.g., lowest confidence intervals).
  • the noise suppression generated by conventional Savitzky-Golay filter is: h 0
  • noise suppression of the adaptive filter is almost three times better than that of the conventional SG filter with the same distortion for the second peak.
  • Figure 13 illustrates a series of peaks with increasing heights for distortion investigation
  • Figure 14 illustrates the difference (e.g., signal distortion) between a filtered signal and a pure signal from Figure 13.
  • dSG31 represents filtering with conventional Savitzky-Golay at gap 31 (curve lowered with respect to 0)
  • One of the benefits of the confidence filter with an adaptive slit is that the absolute value of the peak distortion does not have a strong dependence on the peak height, as shown in Figures 13 and 14. Therefore, minimal statistical confidence intervals are defined by noise level rather than peak shape or peak height. The bigger the peak, the less the relative peak distortion because absolute distortion is nearly constant. Conventional SG filtering fails to provide this since its distortion directly relates to peak height. The peak at detection limit may be considered to be the worst case for the confidence filter; distortion being the price for significant improvement of baseline noise due to the bigger gap.
  • FIG. 5A demonstrates the confidence interval filter applied to a noisy signal to smooth an artificial chomatogram of an electromyography (EMG) peak with white noise.
  • EMG electromyography
  • the original data is represented by the solid line, while the dashed line represents the signal after the confidence interval filter has been applied.
  • the noise definition width for Figure 1 5A is 31 .
  • Figure 15B illustrates a graph of the approximation window which corresponds to the respective window position of Figure 1 5A. Baseline regions of Figure 1 5A are approximated with wider windows (e.g., in the 30-40 range) than the peak region. Since the approximation windows along the baseline are wider, this results in better noise reduction.
  • a comparison between SG filtering and confidence interval filtering can be made by evaluating the minimal and maximal window widths within a smoothing region.
  • the data should be smoothed with minimal window widths and additional noise reduction may be evaluated as follows:
  • non-white noise may be trickier to deal with in comparison to white noise, and results may depend on the particular noise model being used.
  • pump pulsations may frequently occur in chromatography.
  • NDW NDW ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇
  • Figure 1 7B illustrates the polynomial width (shown as a dashed line) versus the distance of the point used for approximation from the center of the polynomial (shown as the solid line).
  • Figure 1 7C illustrates the confidence interval of the approximations by the confidence filter.
  • the confidence filter produced an extremely clean result, as the baseline of the CE signal and the SG filtered signal are both slightly sinusoidal between 1400 and 1 500.
  • the confidence interval filter flattens the baseline into a desirable line shape and eliminates much of the unwanted noise while creating less of a disturbance near the steep end of the peak. It will be appreciated that the approximations near the top of the peak are based on non-central approximations. Additionally, the SG approximation corresponds to the smallest window of the implementation of the confidence interval filter. Most filters cannot be applied to such data, so the improvement in noise reduction may be evaluated as follows:
  • the confidence interval filter does not reject outliers, but simply avoids them.
  • a modified procedure based on robust regression methods may eliminate outliers.
  • drawbacks to such an approach may exist, as robust regressions may obscure the errors of selecting an improper approximation model. Therefore, it may be advantageous to perform object-dependent outlier elimination separately.
  • outlier elimination may be based on identification of outliers as points which were not estimated or points with the highest confidence intervals. These points may be re-evaluated and select points from the input data set may be excluded.
  • Still another embodiment involves a computer-readable medium comprising processor-executable instructions configured to implement one or more of the techniques presented herein.
  • An exemplary computer-readable medium that may be devised in these ways is illustrated in Figure 1 8, wherein the implementation 2000 comprises a computer-readable medium 2008 (e.g., a CD-R, DVD-R, or a platter of a hard disk drive), on which is encoded computer-readable data 2006.
  • This computer-readable data 2006 in turn comprises a set of computer instructions 2004 configured to operate according to one or more of the principles set forth herein.
  • the processor-executable instructions 2004 may be configured to perform a method, such as at least some of the exemplary method 1 00 of Figure 1 , for example.
  • processor-executable instructions 2004 may be configured to implement a system, such as at least some of an exemplary system, for example.
  • a system such as at least some of an exemplary system, for example.
  • Many such computer-readable media may be devised by those of ordinary skill in the art that are configured to operate in accordance with the techniques presented herein.
  • a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer.
  • a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer.
  • an application running on a controller and the controller can be a component.
  • One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
  • the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter.
  • article of manufacture as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media.
  • Figure 19 and the following discussion provide a brief, general description of a suitable computing environment to implement embodiments of one or more of the provisions set forth herein.
  • the operating environment of Figure 19 is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality of the operating environment.
  • Example computing devices include, but are not limited to, personal computers, server computers, hand-held or laptop devices, mobile devices (such as mobile phones, Personal Digital Assistants (PDAs), media players, and the like), multiprocessor systems, consumer electronics, mini computers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
  • Computer readable instructions may be distributed via computer readable media (discussed below).
  • Computer readable instructions may be implemented as program modules, such as functions, objects, Application Programming Interfaces (APIs), data structures, and the like, that perform particular tasks or implement particular abstract data types.
  • APIs Application Programming Interfaces
  • the functionality of the computer readable instructions may be combined or distributed as desired in various environments.
  • Figure 1 9 illustrates an example of a system 21 00 comprising a computing device 21 1 2 configured to implement one or more embodiments provided herein.
  • computing device 21 1 2 includes at least one processing unit 21 1 6 and memory 21 1 8.
  • memory 21 1 8 may be volatile (such as RAM, for example), non-volatile (such as ROM, flash memory, etc., for example), or some combination of the two. This configuration is illustrated in Figure 1 9 by dashed line 21 14.
  • device 21 1 2 may include additional features and/or functionality.
  • device 21 1 2 may also include additional storage (e.g., removable and/or non-removable) including, but not limited to, magnetic storage, optical storage, and the like.
  • additional storage e.g., removable and/or non-removable
  • FIG. 9 Such additional storage is illustrated in Figure 1 9 by storage 21 20.
  • computer readable instructions to implement one or more embodiments provided herein may be in storage 21 20.
  • Storage 21 20 may also store other computer readable instructions to implement an operating system, an application program, and the like. Computer readable instructions may be loaded in memory 21 1 8 for execution by processing unit 21 1 6, for example.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data.
  • Memory 21 18 and storage 2120 are examples of computer storage media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD- ROM, Digital Versatile Disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by device 21 1 2. Any such computer storage media may be part of device 21 1 2.
  • Computer readable media may include communication media.
  • Communication media typically embodies computer readable instructions or other data in a “modulated data signal” such as a carrier wave or other transport mechanism and includes any information delivery media.
  • modulated data signal may include a signal that has one or more of its characteristics set or changed in such a manner as to encode
  • Device 21 1 2 may also include communication connection(s) 21 26 that allows device 21 1 2 to communicate with other devices.
  • Communication connection(s) 2126 may include, but is not limited to, a modem, a Network Interface Card (NIC), an integrated network interface, a radio frequency transmitter/receiver, an infrared port, a USB connection, or other interfaces for connecting computing device 21 12 to other computing devices.
  • NIC Network Interface Card
  • NIC Network Interface Card
  • radio frequency transmitter/receiver an infrared port
  • USB connection or other interfaces for connecting computing device 21 12 to other computing devices.
  • Communication connection(s) 21 26 may include a wired connection or a wireless connection. Communication connection(s) 2126 may transmit and/or receive communication media.
  • Device 21 1 2 may include input device(s) 21 24 such as keyboard, mouse, pen, voice input device, touch input device, infrared cameras, video input devices, and/or any other input device.
  • Output device(s) 2122 such as one or more displays, speakers, printers, and/or any other output device may also be included in device 21 1 2.
  • Input device(s) 2124 and output device(s) 2122 may be connected to device 21 12 via a wired connection, wireless connection, or any combination thereof.
  • an input device or an output device from another computing device may be used as input device(s) 2124 or output device(s) 21 22 for computing device 21 12.
  • Components of computing device 21 12 may be connected by various interconnects, such as a bus.
  • Such interconnects may include a Peripheral Component Interconnect (PCI), such as PCI Express, a Universal Serial Bus (USB), firewire (IEEE 1394), an optical bus structure, and the like.
  • PCI Peripheral Component Interconnect
  • USB Universal Serial Bus
  • IEEE 1394 Firewire
  • optical bus structure and the like.
  • components of computing device 21 12 may be interconnected by a network.
  • memory 21 18 may be comprised of multiple physical memory units located in different physical locations interconnected by a network.
  • a computing device 2130 accessible via network 21 28 may store computer readable instructions to implement one or more embodiments provided herein.
  • Computing device 21 1 2 may access computing device 2130 and download a part or all of the computer readable instructions for execution.
  • computing device 21 12 may download pieces of the computer readable instructions, as needed, or some instructions may be executed at computing device 21 1 2 and some at computing device 2130.
  • one or more of the operations described may constitute computer readable instructions stored on one or more computer readable media, which if executed by a computing device, will cause the computing device to perform the operations described.
  • the order in which some or all of the operations are described should not be construed as to imply that these operations are necessarily order dependent. Alternative ordering will be appreciated by one skilled in the art having the benefit of this description. Further, it will be understood that not all operations are necessarily present in each embodiment provided herein.
  • the word "exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous over other aspects or designs. Rather, use of the word exemplary is intended to present concepts in a concrete fashion.
  • the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise, or clear from context, "X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Operations Research (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Algebra (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Image Processing (AREA)
  • Radar Systems Or Details Thereof (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)

Abstract

L'invention concerne un ou plusieurs procédés et techniques de filtrage du bruit fondés sur l'évaluation d'un intervalle de confiance. Un intervalle de confiance souhaité (par exemple minimal) est utilisé comme critère pour la sélection de paramètres appropriés d'une fonction d'approximation comprenant le degré d'un polynôme, un nombre de points utilisés dans l'approximation, et un positionnement du centre d'une fenêtre par rapport à un point d'approximation utilisé pour l'approximation polynomiale. La fonction d'approximation peut être mesurée avec une certaine erreur par le polynôme ou d'autres fonctions qui permettent l'estimation de l'intervalle de confiance. Le filtrage du bruit fondé sur l'évaluation d'un intervalle de confiance permet de tenir compte d'informations a priori concernant le niveau de bruit du signal.
PCT/US2011/026078 2010-02-24 2011-02-24 Procédé de filtrage du bruit fondé sur l'évaluation d'un intervalle de confiance Ceased WO2011106527A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US33881510P 2010-02-24 2010-02-24
US61/338,815 2010-02-24

Publications (2)

Publication Number Publication Date
WO2011106527A2 true WO2011106527A2 (fr) 2011-09-01
WO2011106527A3 WO2011106527A3 (fr) 2012-07-26

Family

ID=44507560

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/026078 Ceased WO2011106527A2 (fr) 2010-02-24 2011-02-24 Procédé de filtrage du bruit fondé sur l'évaluation d'un intervalle de confiance

Country Status (1)

Country Link
WO (1) WO2011106527A2 (fr)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2016120959A1 (ja) * 2015-01-26 2017-06-29 株式会社島津製作所 ノイズレベル推定方法、測定データ処理装置、及び測定データ処理用プログラム
CN112504426A (zh) * 2020-11-20 2021-03-16 中国直升机设计研究所 一种基于峰值搜索的旋翼桨涡干扰噪声整周期平均方法
CN112539142A (zh) * 2020-11-12 2021-03-23 中国电建集团华东勘测设计研究院有限公司 一种针对海上风电结构状态监测数据噪声的分析识别方法
US20210118102A1 (en) * 2018-03-29 2021-04-22 Leica Microsystems Cms Gmbh Apparatus and method using baseline estimation and half-quadratic minimization for the deblurring of signal data
CN113740802A (zh) * 2021-07-30 2021-12-03 香港中文大学（深圳） 以自适应噪声估计进行矩阵补全的信号源定位方法和系统
CN116484173A (zh) * 2023-02-22 2023-07-25 山东大学 一种基于预测滤波的隧道掘进机阵列声波处理方法及系统
CN116597170A (zh) * 2023-04-21 2023-08-15 哈尔滨工业大学 基于机器学习的暗光图像的特征点提取方法
CN118314673A (zh) * 2024-04-07 2024-07-09 河南唐都科技有限公司 基于互联网数据的智能火灾预警方法及系统
CN118366563A (zh) * 2024-06-19 2024-07-19 睿可净(上海)健康科技有限公司 基于人工智能的室内空气质量快速检测方法
RU2849142C1 (ru) * 2025-04-28 2025-10-22 Федеральное государственное автономное образовательное учреждение высшего образования "Национальный исследовательский Томский государственный университет" Способ адаптивной спектральной фильтрации с использованием фильтра Савицкого-Голея

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111258264B (zh) * 2020-02-24 2021-06-15 北京龙鼎源科技股份有限公司 现场噪声的滤波方法及装置、存储介质和处理器

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7580812B2 (en) * 2004-01-28 2009-08-25 Honeywell International Inc. Trending system and method using window filtering

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107209157A (zh) * 2015-01-26 2017-09-26 株式会社岛津制作所 噪声水平估计方法、测定数据处理装置以及测定数据处理用程序
EP3252465A4 (fr) * 2015-01-26 2018-01-17 Shimadzu Corporation Procédé d'estimation de niveau de bruit, dispositif de traitement de données de mesure et programme de traitement de données de mesure
CN107209157B (zh) * 2015-01-26 2019-04-02 株式会社岛津制作所 噪声水平估计方法和测定数据处理装置
US10359404B2 (en) 2015-01-26 2019-07-23 Shimadzu Corporation Noise level estimation method, measurement data processing device and program for processing measurement data
JPWO2016120959A1 (ja) * 2015-01-26 2017-06-29 株式会社島津製作所 ノイズレベル推定方法、測定データ処理装置、及び測定データ処理用プログラム
US11854166B2 (en) * 2018-03-29 2023-12-26 Leica Microsystems Cms Gmbh Apparatus and method using baseline estimation and half-quadratic minimization for the deblurring of signal data
US20210118102A1 (en) * 2018-03-29 2021-04-22 Leica Microsystems Cms Gmbh Apparatus and method using baseline estimation and half-quadratic minimization for the deblurring of signal data
CN112539142A (zh) * 2020-11-12 2021-03-23 中国电建集团华东勘测设计研究院有限公司 一种针对海上风电结构状态监测数据噪声的分析识别方法
CN112539142B (zh) * 2020-11-12 2022-09-13 中国电建集团华东勘测设计研究院有限公司 一种针对海上风电结构状态监测数据噪声的分析识别方法
CN112504426A (zh) * 2020-11-20 2021-03-16 中国直升机设计研究所 一种基于峰值搜索的旋翼桨涡干扰噪声整周期平均方法
CN112504426B (zh) * 2020-11-20 2022-10-18 中国直升机设计研究所 一种基于峰值搜索的旋翼桨涡干扰噪声整周期平均方法
CN113740802A (zh) * 2021-07-30 2021-12-03 香港中文大学（深圳） 以自适应噪声估计进行矩阵补全的信号源定位方法和系统
CN116484173A (zh) * 2023-02-22 2023-07-25 山东大学 一种基于预测滤波的隧道掘进机阵列声波处理方法及系统
CN116597170A (zh) * 2023-04-21 2023-08-15 哈尔滨工业大学 基于机器学习的暗光图像的特征点提取方法
CN118314673A (zh) * 2024-04-07 2024-07-09 河南唐都科技有限公司 基于互联网数据的智能火灾预警方法及系统
CN118366563A (zh) * 2024-06-19 2024-07-19 睿可净(上海)健康科技有限公司 基于人工智能的室内空气质量快速检测方法
RU2849142C1 (ru) * 2025-04-28 2025-10-22 Федеральное государственное автономное образовательное учреждение высшего образования "Национальный исследовательский Томский государственный университет" Способ адаптивной спектральной фильтрации с использованием фильтра Савицкого-Голея

Also Published As

Publication number Publication date
WO2011106527A3 (fr) 2012-07-26

Similar Documents

Publication Publication Date Title
Deledalle et al. Non-local methods with shape-adaptive patches (NLM-SAP)
Kusnik et al. Robust mean shift filter for mixed Gaussian and impulsive noise reduction in color digital images
KR101750671B1 (ko) 호스트 기반 터치 처리를 위한 최적화
CN108932301B (zh) 数据填充方法及装置
US20140368549A1 (en) Adaptive Patch-Based Image Upscaling
US10354142B2 (en) Method for holographic elements detection in video stream
US9619870B2 (en) Scale adaptive blind deblurring
CN106649877A (zh) 一种基于密度峰值的大数据挖掘方法及装置
US11913919B2 (en) Techniques for acquisition-stage peak width determination and baseline offset estimation
Yuan Improved anisotropic diffusion equation based on new non‐local information scheme for image denoising
Alkhouri et al. Sitcom: Step-wise triple-consistent diffusion sampling for inverse problems
CN104899589A (zh) 一种采用阈值二值化算法实现二维条码预处理的方法
Xiao et al. Image segmentation using Bayesian inference for convex variant Mumford–Shah variational model
CN110162552B (zh) 基于置信区间的时间序列特征提取方法及系统
CN106355600A (zh) 基于非线性结构张量的自适应类圆型结构元素构造形态学算子的方法
CN106683107A (zh) 基于roewa改进的sar图像边缘检测方法
Li et al. A novel signal enhancement method for overlapped peaks with noise immunity
CN110873606B (zh) 一种非均匀性校正方法及装置
US11055852B2 (en) Fast automatic trimap generation and optimization for segmentation refinement
US12530741B2 (en) Systems and methods for real-time preview of hyperspectral data during data acquisition
US20160232919A1 (en) Estimation of noise characteristics
CN117852156A (zh) 一种基于贝叶斯的道路平面线位智能重构的方法及系统
CN111727454A (zh) 用于处理分段平滑信号的装置、方法和计算机程序
Daniele et al. Deep Equilibrium models for Poisson imaging inverse problems via Mirror Descent
CN105894029A (zh) 一种基于费马点求解的自适应运动轨迹数据去噪方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11707959

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11707959

Country of ref document: EP

Kind code of ref document: A2