Rounding errors in digital computer arithmetic subroutines.

• Main Author: Lastman, Gary Joseph
• Language: English
• Published: University of British Columbia 2011
• Subjects: Error analysis (Mathematics)
• Online Access: http://hdl.handle.net/2429/39178

In this thesis we investigate arithmetic subroutines and round-off procedures. An error analysis of single operations in normalized floating point arithmetic leads us to the construction of an improved form of addition-subtraction subroutine. In addition the properties of several types of round-off procedures are examined (adding ½; adding random digits; dropping digits). The experimental work (using the above mentioned subroutines) with the system x• = y, y• = -x shows that the systematic accumulation of round-off error observed by Huskey is due to the type of rounding-off procedure used. Furthermore, Hartree's explanation of this effect is found to be inadequate because carrying extra digits throughout the calculations does not eliminate the systematic round-off. === Science, Faculty of === Mathematics, Department of === Graduate