Search found 2 matches

by Don White
Fri May 25, 2007 11:27 am
Forum: Suggestions/Questions
Topic: Binary Type E uses single precision floats for time values.
Replies: 2
Views: 10680

Binary Type E uses single precision floats for time values

I will send email with the format for the Pacific Instruments BEU format.

Thanks for the fast response!

Don
by Don White
Thu May 24, 2007 5:18 pm
Forum: Suggestions/Questions
Topic: Binary Type E uses single precision floats for time values.
Replies: 2
Views: 10680

Binary Type E uses single precision floats for time values.

For type E binary plot files, long recordings at high rates cause time values that exceed the resolution of the single precision time values specified by the format.
It cannot properly represent time values with a mantissa that is too large.
Example using a recording rate of 50K samples per second ...