考虑工作者信誉的众包质量EM评估方法

Quality evaluation method for crowdsourcing based on workers’ reputation

摘要:

EM算法因其收敛稳定被广泛应用于众包质量评估中,但其评估结果极易受初始值影响,因在使用EM算法时较多赋予工作者相同的权重,导致结果容易陷入局部最优。然而,工作者提交答案的可信度不尽相同,信誉良好的工作者给出的答案可信度更高。为此,本文提出了一种考虑工作者信誉的众包质量评估方法。首先根据工作者在众包平台上的历史交易行为,建立工作者信誉模型；然后将信誉作为评价标准对工作者进行筛选；并以工作者信誉值为权重加入到EM算法,改进其初始值确定过程,改善评估结果。通过实例验证了该方法的有效性,并对提出方法的阈值参数进行了率定。

Abstract:

Because of its stable convergence,EM algorithm is widely used in crowdsourcing quality evaluation, but the evaluation results are easily affected by the initial value. The result easily gets into the local extremum because EM algorithm always gives workers the same weight. However, crowdsourcing workers come from an undefined public, and the credibility of their answers is not the same. The worker with high reputation gives more credible answers. Therefore, a quality evaluation method considering workers’ reputation is proposed. First, build the reputation model based on the workers’ past performance. Then, select the workers according to the reputation of workers and put the reputation as the weight into the EM algorithm optimizing the initial value selection method. Finally, verify the validity of this method through numerical experiment. And the threshold of parameters is rated.