Affiliations: [a] National Meteorological Information Centre, Beijing 100081, China | [b] The College of Information Engineering, Xiangtan University, Hunan 411105, China | [c] Division of Nuclear Technology and Applications, Institute of High Energy Physics, Chinese Academy of Sciences, Beijing 100049, China
Correspondence:
[*]
Corresponding author: Songtao Ye, The College of Information Engineering, Xiangtan University, Hunan 411105, China. E-mail: [email protected].
Note: [1] These authors contributed equally to this work.
Abstract: In the era of big data, the data provided by multiple sources for the same entity may result in conflicting information. Because sources with higher reliability degrees provide true information more frequently, the truth can be obtained by estimating the weights or reliability degrees of the sources. Due to hostile websites or faulty sensors, some sources may occasionally provide outliers that deviate significantly from the truth. However, in the majority of existing truth discovery methods, each source should be uniformly assigned a weight at the initial stage. Therefore, the accuracy of truth estimation is degraded by outliers. Several previous studies have proposed kernel density estimation-based truth discovery algorithms to solve the problems caused by outliers. These approaches aim to estimate the probability distribution of observation values to assess the reliability of sources. Unfortunately, the data with outliers that are smoothed by Gaussian kernels may cause more deviations from the truth. Thus, we propose a local linear regression (LLR) method for addressing the problems caused by outliers. The proposed method can effectively estimate the source reliability and the truth of the datasets with outliers. Experiments on two real-world datasets demonstrate that the proposed method yields more accurate results than existing state-of-the-art methods.
Keywords: Truth discovery, local linear regression, kernel density estimation, outlier data, source reliability
