What is AI slop at work?

Artificial intelligence tools including Copilot, Google Gemini and ChatGPT can help perfect writing a paragraph or transcribe a meeting, but if the content generated is not checked or verified you may be capturing “AI slop” as an official record.

For South Australian state and local government workers, not verifying the data can have significant consequences, according to information management (external site) (external site) experts at State Records of South Australia.

Photo: Adam Ryan uses Copilot in his state government job but always checks the AI tool’s output for accuracy himself.

Senior Policy Officer Adam Ryan said AI without context will make mistakes.

“Not verifying AI outputs can cause headaches… and in the public sector, that might directly impact members of the public,” he said.

“When AI uses personal information it can have many benefits, especially in the health sector, however it also has the potential to wreak havoc on individual lives if it’s not managed properly,” said Manager Information Governance, Janine Silvestri.

According to the Federal Government’s eSafety Commissioner, “AI slop” is “low-quality AI-generated content which often contains errors”.¹

The South Australian government’s Office for Artificial Intelligence said AI learns from data, which “can be flawed”, does not know when it is wrong, and “is not a replacement for human judgement”.²

AI in meetings and job applications: Not great

While AI generating pictures and videos with flaws such as people with six fingers is common knowledge, a good example of AI slop in a business scenario can be found in meeting transcripts. Janine said AI will often record conversations inaccurately, especially when people speak quickly or if two people are speaking.

“There’s been times when I’ve been in meetings and the AI [transcript] has said something, and I know it didn’t happen in that order,” she said.

Janine and Adam have seen examples where people submitted feedback or job applications to State Records’ just using AI, with some telling signs.

“AI will structure its responses in a certain way, and it’s clear it has been copied and pasted without enough oversight,” Adam said.

Benefits of AI at work

But they said AI has its benefits; Adam has used Copilot to help clarify complex sentences in documents and asked it to identify gaps in State Records policies. But he always reviews and fact checks the content generated by AI himself before progressing further.

Janine is equally cautious and encourages people to be familiar with the original content and subject matter to start with, not use AI as a research tool.

“You need to know what’s right, or where to find credible information,” she said.

“If you don’t know anything about the topic or original data, you may run the risk of taking the AI-produced content as gospel.”

Human oversight is best for accuracy

Janine and Adam said it is important to create accurate records and review it with human expertise.

“Accuracy builds trust and helps inform better decision making,” Adam said.

“AI will give you a starting point… but it won’t have the necessary context. AI will make mistakes or assumptions based on missing context.”

This emphasised the need for government agencies to maintain accurate records, including being transparent and declaring when AI has been used to create a document.

“You need compliant records management so agencies can identify when, where and why in order to investigate when something has gone wrong,” Adam said.

“The evidence needs to be clear, transparent and available to audit.”

Where to get help with using AI at work

State Records’ guidance on Artificial Intelligence and Information Management (external site) (external site) explains the data used to train AI systems should be a copy of the original source data, kept in compliant systems.

“This will ensure the source of truth is not inadvertently manipulated,” it advises.

“All other official records relating to an AI system… including the decisions made and content generated must be stored in compliant systems as well.[1]”

More information about State Records’ guidance on Artificial Intelligence and Information Management can be found on its website (external site) (external site).

Find out about Artificial Intelligence in South Australian state and local government here: Office for Artificial Intelligence. (external site) (external site)