Emma Doe’s lawsuit alleges that Google has taken advantage of a dominant position it has maintained in schools since 2007 to introduce AI to children and seize their data without meaningfully protecting them from the worst potential harms of its chatbot’s sycophantic design. In Chicago, CPS signed a contract with Google in 2020, “long before Gemini was launched,” Creswell found. During the next two extensions of that contract, no updates were introduced to protect student privacy as AI advanced.
For Creswell, it remained “totally unclear” in the contracts how Google’s processing of student data may have changed, beyond updates to the Google Workspace for Education privacy policy.
Currently, that policy specifies that “we receive customer data through the Core Services and process it in accordance with the instructions of the school (the customer). This customer data includes anything submitted, stored, sent or received through the Core Services by you or your school.” Google collects that data about that activity, as well as account information “such as name, email address, and password” and location information, according to the policy.
The policy did not talk about Gemini, but noted that data from the use of core services helps maintain services, improve personalization, and increase security. And “the data we collect in Additional Services is used across our services to provide, maintain and improve our services; develop new services; provide personalized services; measure performance; communicate with you; and protect Google, our users and the public.”
For parents, the policy offers no guarantee that their children’s data is not part of Google’s plan to dominate the artificial intelligence industry.
“We should not allow any children’s data to be collected in a school setting for commercial purposes,” Creswell said.
