Optimization for grid-based batch scan

Please see write-up here
http://kakyolab.xicp.net/0.report/29.batch/Towards%20the%20automated%20batch%20scan.doc

Fitting the raw edge

Description

• Spline and polynomial fits were investigated; it turned out that spline fit is essentially piece-wise polynomial fit with bounded error by setting up knots/breaks and order.
• Previously there was a misunderstanding of mine that thinks the textbooks were trying to dismiss polynomial fitting. It turns out that they only dismiss global polynomial fitting, compared to spline fitting in the piece-wise polynomial sense. Therefore, piecewise polynomial fitting that Haber used is essentially valid.
• I made several tests and comparisons between spline and polynomial fitting using Matlab. See report:http://kakyolab.xicp.net/0.report/27.fitting/0_screenshots_1.htm
• See Detail

Detail

• For both polyfit and spline fit, two different approaches are used:
  
• For each point along the resampled edge angular position sequence, make each point the center and take a window around it, fit the real edge points that fall into the window to the resampled points, in a point-by-point fashion, shown in Row 2-5.
  
• Divide the resampled edge angular position sequence into windows (with overlap) and fit the whole group of resampled points within each window to the real edge points in that window, shown in Row 6-9
• It turned out that the fitted result from spline is very unlike sinewave, but pretty similar to the raw edge, meaning that spline actually has a better simulation and weak smoothing effect; polynomial, in comparison, has a better smoothing and gives us sine shape, but not necessarily preserve the original waveshape.
• I'm still not sure which one should be used.
  

Current solution to Saturation

Description

• The saturation under the condition of long scan depth is temporarily solved by combining two measures
• Increasing modulation threshold (rule out more invalid points).
  
• Dividing the entire scanning surface into smaller areas, scan them one by one and adjust the measurement parameters individually.
• The reason why threshold matters to the saturation is not understood yet.
• The scan may take another 2-3 days to finish (taking into account any possible re-scanning of bad data)
  

Grid Batch Measurement Pitfall

Grid Batch Measurement Pitfall

Description

• New measurment of test record encountered a difficulty regarding the global scan depth settings.
• Remember we had to apply a relatively large global depth to accommodate a wide range of depth gap due to the disc surface warp.
• However,
  it is discovered that such a setting causes scan saturation for quite a
  lot of FOVs, i.e., acquired depth data were distorted. The exact reason
  is not fully understood yet, although a few evidence was obtained
  through repeated scanning with different parameters, such as scan depth
  and intensity:
  • When this depth is large (e.g., 300um) and the
    turret tilt is not optimized for the FOVs, in which the groove depth
    range is located around the deepest portion of our scan depth range,
    the resulting depth image tend to get saturated.
  • Given a FOV
    that's naturally tilted relative to the turret. Let Scan #1 be a
    reference scan (normal result) with its depth range adapted to the
    target FOV's without tilt correction, Scan #2 a scan with long depth
    range regarding the target groove without tilt correction, and Scan #3
    with same depth range but with tilt correction; all three scans were
    performed on the same FOV. A quick test reveals that the distortion of
    depth data of #2 could be so large that the depth level of measured
    groove top edges of #2 could be as low as that of the groove bottom of
    #1. This is an obvious distortion. However, #3 generates the similar
    result with #1.
    
  • By chaning the intensity to a lower value but without tilt correction didn't remove the saturation.
  • Scan #1: http://kakyolab.xicp.net/0.report/26.saturation/saturation_1.tif
  • Scan #2: http://kakyolab.xicp.net/0.report/26.saturation/saturation_2.tif
    
  • Scan #3: http://kakyolab.xicp.net/0.report/26.saturation/saturation_3.tif
    

Plan

• Because
  tilt setting is static during batch measurement and it is empirically
  safe to assume depth consistency between surface areas that are close
  to each other, the ways to avoid saturation would probably one of the
  two approaches
  • Finely sub-divide the target region into smaller ones, and perform individual grid measurements on them:
    • Pros: this is the easiest way in terms of development
    • Cons: the surface situation is unpredictable; therefore the sub-division can hardly be designed systematically.
  • Dynamically
    monitoring the scanning intensity window, do a fancy image recognition
    to warn the scanning process about the absence of interference patterns
    (moving stripes), then automate the GUI program of Vision to restart
    the scanning.
  • Pros: the ideal way for mass production.
  • Cons: requires significant development effort, invoving image recognition and window process management.
    
• For now, I would go with the subdivision for the current data.

scanning long sine wave: vos grid calculation fixed

Description

• Last week I was about to start scanning a long sine wave, using Vision's grid stitching function. My previous script to calculate the grid ended up giving me a weird starting stage position: the Stage always went to the rightmost or rear-most point at the beginning and never seemed to come back. After a while of debugging, I realized
  
• This should be a bug of Visions grid function with a very large grid: by default the grid reference cell is the the grid top-left corner FOV; when setting up this corner as a reference cell, if this cell is not included (enabled) during the scanning, the stage will assume that the grid starts from wherever the stage was at the start of the scan. The solution is to always include and scan the grid top-left corner FOV in the scanning grid as the first FOV, even when it is meaningless to scan the corner FOV.
• About the selection of the experimental disc, remember in the previous discussion with Ich on the same topic, I insisted using another older disc because it gave me better groove bottom measurements across wide disc surfaces areas (still loses data uner certain circumstances); however, after seeing the jaggy edge data, I realized that currently we don't care about the depth as much, and a newer disc may give us a better edge data for the lateral extraction.
  
• Therefore I selected one newly purchased system disc by Ich.
• The new problem of the disc is that the surface is quite warped, so that across the long stereo (right channel only) sine signal (1.8 second ), the depth variation could be as large as 300 microns, which is pretty appalling if we would like to apply a global depth to the whole scanning, although the build-in depth adaptation function may help, but what it does is just linearly raise or lower the turret, which may be a cause of lens' crash. So to scan the whole area of data, a global/systematic depth variation might be incomplete.
• It's necessary to divide the disc surface into smaller areas to minimize the internal warp of that area, which may cause the overly large depth range.
• The scanning has yet to start before we make sure that the signal and setup is ok.
  

Bibliography on Numerical Differentiation

Description

• Previously, it was confirmed that raw groove edge of the scanned sine wave is not as smooth as an ideal audio sine wave, after testing with interpolation at various sampling rates; therefore, certain smoothing should be performed somewhere in the workflow prior to the audio output.

• Whether to perform smoothing before or after the differentiation is a question.

• A few literature on numerical differentiation (ND) are visited.

• See detail.

Detail

• From what I read, common ND all use a function to fit the data first, and then obtain the ND to that function.

• Smooth can be integrated with the UFO thing.

Bibliography

• Engeln-Müllges, G, and F. Uhlig. 1996. Numerical Algorithms with C. Berlin: Springer-Verlag.

• Indicated that among three common ND methods (interpolating polynomial, spline, Richardson extrapolation), the differentiation of an interpolating polynomial is least recommended. Note that Haber adopted polinomial interpolation.

• The rating from the higher to the lower methods in terms of accuracy is Richardson extrapolation, spline, and polynomial.

• Practical recipes of the first three methos are given.

• Numerical quadrature leads to better result than ND does, due to roughening effect of differentiation.

• Cheney, W., and D. Kincaid. 1999. Numerical Mathematics and Computing, 4^th ed. Brooks/Cole: Pacific Grove.

  • Talked about differentiation i the context of interpolation.

  • Indicate that the data for interpolation and differentiation usually contain noise.

  • Indiate that polynomial may overfit and create large variance. Spline is preferred over polynomial.

• Gautschi, W. 1997. Numerical Analysis: An Introduction. Birkhäuser: Boston.

  • Starting from general formula of differention of unequally spaced points, explained that symmetical difference quotient is one order more accurate than the one-sided one is.

  • Explained that in practice, purturbed data usually causes increasing error, especially as the h parameter gets smaller, i.e., fewer point differentiation. This is called truncation error, i.e., the finite approximation throws out the error term in the Taylor formulae.

• Maron, M. 1987. Numerical Analysis:A Practical Approach, 2^nd ed. New York: Macmillan.

  • Discussed approximation error in terms of truncation error (polynomial interpolation) and roundoff error, inversely proportional to the step-size h in 1^st-order derivatives , i.e., large h leads to large truncation error, but too small an h would cause roundoff error.

  • Discussed ND based on equispaced and unevenly spaced samples. Polynomial and Robinson extrapolation are efficient in the equispaced case; for unevenly cases, quadrature like Gauss quadrature and adaptive auadrature is preferred.

• Sharma, J. 2004. Numerical Methods for Engineers and Scientists. Pangbourne: Alpha Science International.

  • Pointed out that ND is essentially fitting the data to a function and then obtaining the approximate analytical differentiation of that function.

  • Gave definition for quadrature.

• Nakamura, S. 1991. Applied Numerical Methods with Software. Englewood Cliffs: Prentice-Hall.

  • Talked about forward, backward, and central difference approximation. Central ND is always more accurate than the other two, in the sense of Newton interpolation.

  • Talked about 3 approaches to deriving difference approximatios: Taylor expansion, difference operators, and differentiation of interpolating polynomials; their advantage and disadvantage are explained.

• Rice J. 1993. Numerical Methods Software and Analysis, 2^nd ed. Harcour Brace, and Jovanovich: Acdemic Press

  • simultaneously smooth and estimate derivative in Chi1.

  • Quadrature is lagacy term of approximate interpotion.

Raw edge resampled at various sampling rates

Description

• The edge smoothing is dismissed because we would like to see how optical approach can have different information extracted as opposed to turntables, which skip information during stylus reading.
• Results are at: http://kakyolab.xicp.net/0.report/25.interp_edge_raw/0_screenshots_1.htm
• each column represents one resampled edge at an angular sampling rate derived from an audio sampling rate.

Differentiation updated

Description

• Previously two errors in plotted analytical results of the differentiation of the sine-wave groove edge were found. Previous result: http://kakyolab.xicp.net/0.report/23.interp_diff_edge/0_screenshots_1.htm
1. The horizontal distribution of the points are uneven
2. Under 3-point differentiator (skip one point) The radius  variation (vertical axis) between close angular positions is too great, so that the graph looks nothing like a sine-wave but noise
• These two problems have been solved. Result is at:http://kakyolab.xicp.net/0.report/24.interp_diff_edge/0_screenshots_1.htm, the first row shows from the raw input edge to the differentiated result; the 2nd row shows the results of differentiation at various point-settings.
1. The source of error were not from the data but from my plotting program, it wrongly took the "arc" (radius times angle) as its X axis, so that the interval of the data became irregular. By replacing the X-axis with angles, the horizontal data spacing are showed to be even.
2. By using higher number of points for the differentiator, the differentiated edge data appear smoother and smoother and much resemble a sine-wave shape.

Plan

• Do some reading on numerical differentiation and prepare a bibliography, to figure out how to design a proper differentiator to achieve the desired sound according to the reference data.

Edge extraction and interpolation

Description

• We decided to use a specified audio rate to resample the original data, i.e, 96kHz from last meeting.
• The experiment was performed on an FOV's one CC's one edge that contains a sinewave (1kHz).
• The result is at: http://kakyolab.xicp.net/0.report/23.interp_diff_edge/0_screenshots_1.htm
• The procedure
• Low level groove edge detection by examining the neighbors (not the image processing routines, but by rules in our context).
• Smooth the data with a moving window.
• Interpolate the data at angular positions according to the specified audio sampling rate.
• Numerically differentiate the data, by skipping only one point.
• However, the result is noisy in display. With polynomial fitting, things would get much better.
• By skipping more points in differentiation, it may improve, but I haven't tested it yet.