Google AI Flagged Parent Accounts Over Naked Kids’ Photos

The platforms’ Artificial Intelligence (AI) systems are increasingly refined, in order to guarantee the safety of users. This time, Google’s AI flagged parent accounts for potential abuse.

The reason? They had pictures of their naked children.

<!–

PUB

–>

According to The Verge, citing the The New York Timesa father claimed that after using his Android smartphone to capture his son's groin infection, he saw the images flagged as child sexual abuse material by Google.
Following these images, Google closed its accounts, filed a complaint with the Center for Missing and Exploited Children (NCMEC) and launched a police investigation.  However, the photographs were taken to help doctors diagnose the infections.
Alongside the investigation, Google highlighted the complications associated with distinguishing between potential abuse and a photograph with no malicious intent.

Father flagged by Google for having pictures of sick son naked
The New York Times explained that the main incident happened in February 2021, when some medical centers were still closed due to the pandemic.  The child's father noticed a swelling in the child's groin and, at the request of a nurse, sent images of the region before the consultation, which took place by video call.  After that, the doctor prescribed antibiotics and the infection was cured.

Although everything went well, two days after the images were captured, the father, named Mark, received a notification from Google that his accounts had been blocked, due to “harmful content” which was a “serious violation of the policies”. from Google”.

We identify and report child sexual abuse material with expert teams and state-of-the-art technology, including machine learning classifiers and hash matching technology, which creates a "hash", or unique fingerprint, for an image or video so that can be compared to hashes of known child sexual abuse material.  When we find these materials, we report it to the NCMEC, which liaises with law enforcement agencies around the world.

The father, who ended up losing access to his e-mails, contacts, photos and his phone number, was acquitted when the San Francisco Police Department, responsible for the case, “did not find the elements of a crime” and concluded that "No crime has taken place."