Only 5 weeks ago I warned Facial Recognition Technology bias in the Foodstuffs trial was technically likely and not a matter of if, but when and that Māori women would likely be discriminated against.
That same week Māori Television first reported “Māori women and women of colour will be targeted’ in Foodstuffs’ facial recognition trial – Māori tech expert to balance the primary narrative by two white men in authority.
Then most New Zealand mainstream and many Māori media companies began providing coverage giving a cultural perspective including Radio Waatea, NZ Herald, 1news, J.R.Bruning Talking Risk, and international coverage including OpenGov and BNN.
Sadly, only two days ago, about a month after the trail started, the NZ Herald reported that a Māori women has been falsely identified as a trespassed thief by FoodStuffs Facial Recognition Technology in Rotorua, despite the victim offering her ID a telling two staff that the photo was not her. All of this despite FoodStuffs making public claims that it was safe and its staff were trained.
Foodstuffs have stated they have specifically trained staff to verify images that are a 90% or higher match.
The technology scans customers’ faces and compares these images to those on the store’s databases of known offenders or suspects.
All appropriate staff would receive further training and the incident would be reported to the Office of the Privacy Commissioner.
Despite the innocent victim offering three forms of ID that would have proved she was not the person trespassed and staff checking a photo of the offender, the staff insisted that she was lying.
She said they insisted she leave, even when she offered photo identification.
The supermarket, New World Westend, says it was a “genuine case of human error” and would apologise.
The issue is this is a common phenomenon seen in America where the human beings doing manual checks of positive matches from Facial Recognition Technologies still rely on the AI and not common sense or obvious facts, as there is a perception by the human beings that the AI can not be wrong. The impacts of physiological damage are immense.
Before an alert from the facial recognition system was acted on, two trained team members must verify the matched image was of a person enrolled in the system, they said.
“In this case, our team didn’t err on the side of caution and got it wrong.”
It is also a proven common trait of the human beings doing the checks being racist and in some instances not being in social situations with people other than their own race and culture.
She said she felt “racially discriminated” against by the men and described their conduct as “unconscionable”.
She said it scared her that this happened despite staff having training for the trial.
“I feel like I’m at risk everywhere I go now, it’s horrible.”
FoodStuffs have previously publicly stated that their staff would be trained so these situations would not occur. The question is what kind of training was offered and were the staff racist and stereotyping a Māori woman in a supermarket? Will the be an excuse for FoodStuffs and other retailers who are watching this trial to ask customers for fingerprints or biological samples to prove who they are?
It is most likely that this is not the first false accusation and it will certainly not be the last.
If this has also happened to you and you want to share your experience or need assistance to raise an issue to FoodStuffs, please message me.
Possible solutions to prevent this occurring again in New Zealand could include:
- Employing staff who have lived knowledge and experiences in various ethnic communities and with various religious beliefs from both locally, nationally but also international experience.
- Customised human recognition training using New Zealand data sets that will test for (un) conscious bias and racism of human staff.
- Better signage that Facial Recognition Technologies are being used in the store
- Employing local staff from ethnic communities
- Training in de-escalation so that people are not yelled at and have their personal spaces invaded
- Policies in place if there is a dispute, perhaps the most senior manager at the venue to have the final decision
- Relationships with community leaders who could assist with disputes about identity or cultural practices such as wearing niqab, burka and other items that may hide the face.
- Engagement with the disability community about how best to engage with vision and hearing impaired customers.
Leave a Reply