Page 1 of 1

Generate->Y=f(X) X interval calculation off by 1

Posted: Fri Aug 28, 2009 2:16 pm
by toddbatzler
DPlot Version 2.2.6.5

Set from X = 0, to X = 0.049999, dx = 1e-006 and number of points reported is 50001. Generate waveform, edit data and waveform was generated up to and including X= 0.05, not 0.049999

Same behavior if numbers 0.1 to 0.199999 dx = 1e-6, but ok with 0.2 to 0.299999 or any subsequent interval where tenths digit is >1.

Posted: Fri Aug 28, 2009 3:17 pm
by DPlotAdmin
The trouble (in your example) is that even with double-precision floats, 49999 * 1.E-6 is equal to (roughly) 0.049999 - 6.9E-18. That's of course less than 0.049999, so a point is added. I'll modify the test to add a small (very small) bit of slop.