Computers, Coders, and Voters: Comparing Automated Methods for Estimating Party Positions

Abstract

Assigning political actors positions in ideological space is a task of key importance to political scientists. In this paper we compare estimates obtained using the automated Wordscores and Wordfish techniques, along with estimates from voters and the Comparative Manifesto Project (CMP), against expert placements. We estimate the positions of 254 manifestos across 33 elections in Germany and Denmark, two cases with very different textual data available. We find that Wordscores approximately replicates the CMP, voter, and expert assessments of party positions in both cases, whereas Wordfish replicates the positions in the German manifestos only. The results demonstrate that automated methods can produce valid estimates of party positions, but also that the appropriateness of each method hinges on the quality of the textual data. Additional analyses suggest that Wordfish requires both longer texts and a more ideologically charged vocabulary in order to produce estimates comparable to Wordscores. The paper contributes to the literature on automated content analysis by providing a comprehensive test of convergent validation, in terms of both number of cases analyzed and number of validation measures.