Camera calibration and configuration for estimation of tennis racket position in 3D

• Main Author: Elliott, Nathan
• Other Authors: Allen, Tom ; Choppin, Simon ; Goodwill, Simon
• Published: Sheffield Hallam University 2015
• Online Access: https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.741532

Previously, stereo camera systems have been used to track markers attached to the racket frame, allowing for racket position to be measured in three-dimensions (3D). Typically, markers are manually selected on the image plane but this can be time consuming and inaccurate. The purpose of this project was to develop and validate a markerless method to estimate 3D racket position using a camera. The method relies on a silhouette of a racket captured with a camera whose relative pose (rotation and translation) is unknown. A candidate relative pose is used to measure the inconsistency between the silhouette and a set of racket silhouettes captured with a fully calibrated camera (known intrinsic and extrinsic parameters). The measure of inconsistency can be formulated as a cost function associated with the candidate relative pose. By adjusting parameters of the candidate relative pose to minimise the cost, an accurate estimation for the true 3D position of the racket can be made. Previous studies have found that silhouette-based pose optimisation methods depend on accurate camera calibration and silhouette extraction. Therefore, a repeatable and accurate camera calibration method to provide the relative pose of a camera with respect to a racket was developed. To facilitate silhouette extraction, the racket was painted black and a backlight was used. Synthetic camera poses and silhouette views associated with a 3D racket model were generated in Blender v2.70 and used to determine the optimum fully calibrated set configuration for a racket. A laboratory-based fully calibrated set (LFCS) consisting of 21 camera poses in a semispheric configuration was created. On average, using this set, racket position was reconstructed to within +/- 2 mm. This included systematic error arising from the calibration and error in the segmentation of silhouette boundaries. The maximum reconstruction error was 5.3 mm. Further synthetic testing demonstrated the methods ability to estimate 3D racket position during simulated real-play conditions. For racket silhouette orientations that simulated strokes expected to occur in tennis between 0 and 90°, mean RMSE for reconstruction of coordinates on the racket face plane was 1.5 +/- 1.8 mm. An RMSE of 2 mm was obtained from a camera positioned alongside the net, 14 m from the racket. Finally, this same camera position estimated 3D racket position to an accuracy of 1.9 +/- 0.14 mm using a fully calibrated set containing randomly orientated camera poses, during a simulated serve. This project developed and validated a novel markerless method to estimate 3D tennis racket position. A calibration method to obtain the relative pose of a camera with respect to a racket is presented and an appropriate configuration for a fully calibrated set is determined. The method has potential to be used alongside existing ball trajectory analysis tools to provide unprecedented information about player performance and to enhance tennis broadcasts. Future research should use the recommendations made in this project to inform and assist the development of the method for application during real tennis-play conditions.