On Selection Criteria for the Tuning Parameter in Robust Divergence

• Main Authors: Shonosuke Sugasawa, Shouto Yonekura
• Format: Article
• Language: English
• Published: MDPI AG 2021-09-01
• Series: Entropy
• Subjects: efficiency; Hyvarinen score; outlier; unnormalized model
• Online Access: https://www.mdpi.com/1099-4300/23/9/1147

Although robust divergence, such as density power divergence and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>γ</mi></semantics></math></inline-formula>-divergence, is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnormalized model defined by robust divergence. The proposed selection criterion only requires first and second-order partial derivatives of an assumed density function with respect to observations, which can be easily computed regardless of the number of parameters. We demonstrate the usefulness of the proposed method via numerical studies using normal distributions and regularized linear regression.