West Ham United, the English football (soccer) club I support, hasn’t seen much recent success. (Written in 2014, as I revise this in 2021 things are better). The fans, in a fine example of optimism bias, expect the team to win while playing with a certain élan. In 2011, after a disastrous few years, Sam Allardyce was installed as manager. Under Allardyce results were far from magical. Still, West Ham has been promoted to, and survived two seasons in, the top tier of English football. The supporters were not universally happy. Allardyce had a famously pragmatic, non-entertaining, style of play. A number of fan websites, therefore, collaborated to ask about Allardyce’s future. I use this survey to illustrate problems with survey methodology and the future of West Ham united.
Subtle Question Choices Change Survey Responses
Firstly, choice of question has a surprisingly strong impact on the result. The question used was simple. “Should Sam Allardyce be retained as manager of West Ham for the 2014-15 season?” Yet, this was not really the right question given Sam Allardyce already had a contract to manage West Ham for the year. The choice wasn’t really whether to “retain” Allardyce but whether to fire him as manager and so break his employment contract. Survey writers should always make sure the wording reflects the reality of what is suggested. People often tend to have status quo bias so may be more willing to not break the contract (i.e., keep the status quo) rather than take positive action and actively seek to retain a relatively unpopular manager.
Pay To Fire Or Pay To Acquire
Respondents would, understandably, have little sympathy for Allardyce if he was fired. That happens to football managers, they get fired at an alarming rate. Still, they are usually given massive payoffs and often just get another similar job. Even if you don’t have qualms about breaking the employment contract firing a manager costs several million pounds. A better question, therefore, might be: “Would you like to fire Sam Allardyce instead of recruiting a multi-million-pound player?”
The question makes no mention of a replacement manager. The decision the board faced was between Allardyce and any alternative managers who would take the vacant role. The consultation question doesn’t mirror the board’s decision so is of limited value in informing the decision.
Timing And Population Sampled Matter
The West Ham fans survey followed closely a string of disappointing results on the pitch. Survey timing matters. When considering a survey always ask: Did people think the same last month? Will they feel the same next month?
16 supporter websites provided the respondents. Many of these websites criticized Allardyce and his style of play. Polling stations ban potentially biasing materials to avoid influencing the voters. Websites do the exact opposite.
Finally, recruiting volunteer respondents through websites does not give a random sample. The website’s visitors are probably atypical of all supporters. Those filling in the survey may not even be typical of those who visit each website. Non-scientifically selected samples are dangerous. This is because they appear informative but the results are always highly suspect.
Survey Methodology And The Future Of West Ham United
In the poll 77% voted against Sam Allardyce. He maintained that only a minority of supporters wanted him fired. This did not go down well with his critics. Still, Allardyce was right to say that the evidence against him was weak. We really can’t conclude much from this survey. Allardyce will always be welcome in my class as a guest speaker critiquing survey methodology.
PS Allardyce stayed for the year. At the end of the year he left (contract not renewed). Allardyce even went on to manage England for a game before a scandal ended his tenure.
For more on survey approaches see here, here, and here.
Read: One of many West Ham fan sites. My favourite is West Ham Till I Die: https://www.westhamtillidie.com/posts/2014/04/28/poll-result-77-of-west-ham-fans-say-sam-allardyce-should-go