This article documents an instance of this problem:
- https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gym-website-aus-cyber-attack/107007986
Fupio is the easiest way to share online Learn more »
Join Fupio#chatbots
This article documents an instance of this problem:
tantek.comCSF_03: today’s Cybersecurity Friday post: the effective security of small business websites has likely gotten worse due to LLMs and “AI agents” (with or without safeguards) and what actions you may want to consider.
This article documents an instance of this problem:
* https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gym-website-aus-cyber-attack/107007986
Small business websites tend to be sloppily written (no pun intended though that may also be true) and are likely riddled with numerous very fundamental security holes. There are many possible explanations (economics) from poor initial construction, perhaps using the latest new trendy framework rather than established hardened libraries, to lack of maintenance after initial setup. They have obvious holes like lack of server-side form validation (people being able to change values in forms using browser dev tools), and less obvious like buggy APIs allowing more access than they should.
In the past, many of these holes didn’t really matter because those sites were “not worth attacking” for the incentive models of human-based cyber-attackers or collectives thereof.
However, now that there are LLMs that have likely been trained on any number of common security holes in websites (and how to exploit them), when an “AI agent” is given a task, it may very well use any “tool” at its disposal, including website vulnerabilities to accomplish its goals, as illustrated by the example in the Australia ABC news article above.
Since such chatbots are now essentially "hack websites as a service", I expect we will see LOTS more of this happening, likely unintentionally, or sometimes with mild intention like “can you get me higher on the waitlist”.
Ultimately I think both the human giving instructions to (prompting) such chatbots and the creators of such chatbots should be held responsible for any such intrusions and any damage they cause, even if/when unintended.
There are a few things you can do about this emerging phenomenon:
1. If you use such “agents”, be very careful about what you ask it/them to do, avoiding asking for anything that’s morally gray or questionable at all, even something as “minor” as cutting the line in an online waitlist.
2. If you run a small business site, you have your work cut out for you. Pay a professional web developer to audit the security of your website, document what they find, and patch holes / repair it accordingly.
3. If you have accounts on small business sites you rarely or ever use, consider exporting any data (receipts, transactions), replacing your profile details (name, addresses, photos) with noise, and then deleting your account. If you need to use the site again, use a different email address (as recommended in https://tantek.com/2025/122/b1/more-steps-indieweb-cybersecurity) to create a new account.
That last tip is also helpful for reducing your own personal “attack surface”. By pruning your online accounts, you both reduce the number potential data breaches that you’re in, and reduce the places and ways that attackers can cause you trouble (or that you have to double-check if you’re ever the target of a cyber-attack)
Previously: https://tantek.com/2025/122/b1/more-steps-indieweb-cybersecurity
#CyberSecurity #Friday #cyber #security #cyberAttack #cyberAttacker #chatBot #chatBots #LLM #LLMs #AI #agent #agents #AIagent #AIagents
#Blaugust #Blaugust2026 - Tantek
“Won't that be grand? Computers and the programs will start thinking and the people will stop.” — Walter
2026 students:
“How do you know an assignment is AI proof? Because when it is, students cannot do it.”
- https://www.reddit.com/r/Professors/comments/1vkw6by/how%5Fdo%5Fyou%5Fknow%5Fan%5Fassignment%5Fis%5Fai%5Fproof%5Fbecause/
tantek.com1982 TRON prescience:
“Won't that be grand? Computers and the programs will start thinking and the people will stop.” — Walter
2026 students:
“How do you know an assignment is AI proof? Because when it is, students cannot do it.”
* https://www.reddit.com/r/Professors/comments/1vkw6by/how_do_you_know_an_assignment_is_ai_proof_because/
“I assign a profile essay. Have for 30 years. Student interviews a person, records it, writes a paper based off the recording. Citations are timestamps. Should be pretty easy. Has been easy pre-AI. Now, students cannot do it.”
Snippet from professor/student dialog:
====
Prof: ”Start with what you found interesting.”
Student: ”How do I know if something’s interesting?”
Prof: ”You’re asking me how you find something interesting?”
Student: ”Yeah.”
====
Chatbots / LLMs are not just "the latest tools" like using a calculator instead of doing arithmetic yourself.
Students are being robbed of curiosity, learning to discover and notice novelty, how to “find something interesting.”
This isn't cognitive enhancement, this is a tool causing brain rot (loss of curiosity), even a developmental disability. Unclear if it is reversible.
TRON quote citation: https://www.imdb.com/title/tt0084827/quotes/?item=qt0406255
Update, see also:
* 2026-08-19 “Teachers Warn That Students Are Losing the Ability to Think as They Lean on AI for Everything” https://futurism.com/future-society/students-lose-ability-think-ai
#TRON #TRON1982 #AIproof #interesting #chatBot #chatBots #LLM #LLMs #AI #brainRot #AIbrainRot #disability #Blaugust #Blaugust2026 - Tantek