Crowdsourcing yields a new standard for kinks in protein helices

Abstract

Kinks are functionally important structural features found in the α-helices of proteins. Structurally, they are points at which a helix abruptly changes direction. Current kink definition and identification methods often disagree with one another. Here we describe a crowdsourcing approach to obtain a reliable gold standard set of kinks. Using an online interface, we collected more than 10 000 classifications of 300 helices into straight, curved, or kinked categories. We found that participants were better at discriminating between straight and not-straight helices than between kinked and curved helices. Surprisingly, more obvious kinks were not necessarily identified as more localized within the helix. We present a set of 252 helices where more than 50% of the participants agree on a classification. This set can be used as a reliable gold standard to develop, train, and compare computational methods. An interactive visualization of the results is available online at http://opig.stats.ox.ac.uk/webapps/ahah/php/experiment_results.php.