Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5

Favourite tool for measuring implicit bias is still mired in data controversy

#1
C C Offline
https://digest.bps.org.uk/2018/12/05/psy...ntroversy/

INTRO: It has been a long and bumpy road for the implicit association test (IAT), the reaction-time-based psychological instrument whose co-creators, Mahzarin Banaji and Anthony Greenwald — among others in their orbit — claimed measures test-takers’ levels of unconscious social biases and their propensity to act in a biased and discriminatory manner, be that via racism, sexism, ageism, or some other category, depending on the context. The test’s advocates claimed this was a revelatory development, not least because the IAT supposedly measures aspects of an individual’s bias even beyond what that individual was consciously aware of themselves.

As I explained in a lengthy feature published on New York Magazine’s website last year, many doubts have emerged about these claims, ranging from the question of what the IAT is really measuring (as in, can a reaction-time difference measured in milliseconds really be considered, on its face, evidence of real-world-relevant bias?) to the algorithms used to generate scores to, perhaps most importantly (given that the IAT has become a mainstay of a wide variety of diversity training and educational programmes), whether the test really does predict real-world behaviour.

On that last key point, there is surprising agreement. In 2015 Greenwald, Banaji, and their coauthor Brian Nosek stated that the psychometric issues associated with various IATs “render them problematic to use to classify persons as likely to engage in discrimination”. Indeed, these days IAT evangelist and critic alike mostly agree that the test is too noisy to usefully and accurately gauge people’s likelihood of engaging in discrimination — a finding supported by a series of meta-analyses showing unimpressive correlations between IAT scores and behavioral outcomes (mostly in labs). Race IAT scores appear to account for only about 1 per cent of the variance in measured behavioural outcomes, reports an important meta-analysis available in preprint, co-authored by Nosek. (That meta-analysis also looked at IAT-based interventions, finding that while implicit bias as measured by the IAT “is malleable… changing implicit bias does not necessarily lead to changes in explicit bias or behavior.”)

So where does this leave the IAT? In a new paper in Current Directions in Psychological Science called “The IAT Is Dead, Long Live The Iat: Context-Sensitive Measures of Implicit Attitudes Are Indispensable to Social and Political Psychology”, John Jost, a social psychologist at New York University and a leading IAT researcher, seeks to draw a clear line between the “dead” diagnostic-version of the IAT, and what he sees as the test’s real-world version – a sensitive, context-specific measure that shouldn’t be used for diagnostic purposes, but which has potential in various research and educational contexts.

Does this represent a constructive manifesto for the future of this controversial psychological tool? Unfortunately, I don’t think it does...

MORE (details): https://digest.bps.org.uk/2018/12/05/psy...ntroversy/
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Article Flawed body of studies shows true ‘long COVID’ risk likely exaggerated (data bias) C C 0 349 Sep 26, 2023 02:23 AM
Last Post: C C
  Do we have a ‘bias bias’? C C 2 124 Jan 20, 2022 02:13 AM
Last Post: Syne
  5 types of cat owners (survey data) + Breast cancer link to hair dye? (data project) C C 0 414 Sep 7, 2020 01:54 AM
Last Post: C C
  New tool aims to assist military logistics in evacuating noncombatants C C 0 149 Mar 2, 2020 08:40 PM
Last Post: C C
  Amazon scrapped 'sexist AI' tool (data training) C C 1 482 Oct 10, 2018 04:50 PM
Last Post: Syne
  Data on preferences: Is gender inequality inevitable? + Data ethics is more than what C C 1 665 Jun 28, 2018 02:30 AM
Last Post: Syne
  Data’s intangiblility & ownership claims + Kant according to quantitative data C C 0 455 Jun 21, 2018 05:22 PM
Last Post: C C
  Data thugs + ‘Still working’ on the data: Astronomers explain why they don’t publish C C 0 810 Feb 20, 2018 08:33 PM
Last Post: C C
  In data analysis, just how bad has cognitive bias become? C C 0 588 Oct 11, 2015 03:37 PM
Last Post: C C



Users browsing this thread: 1 Guest(s)