Working on some poison-as-a-service (PaaS).
-
How do you differentiate Ai bots from search engine bots from sploit scrapers?
I've seen sploit scrapers in my logs, but I wouldn't be able to disambiguate Ai from indexer.
@n_dimension I do not see a way to differentiate, and suspect there is no way.
-
Also working on a zip bomb, to randomly scatter in among the links.
Thanks to @anaiscrosby I came across this excellent method, using LZ77:
https://natechoe.dev/blog/2025-08-04.html
TBH I was just going to `dd if=/dev/urandom` my way to a titanic RAM flooding *.gz, but am getting great results with the above, and with bonus site data honey inside to keep bots on the chase.
@anaiscrosby After seeing ChatGPTBot blow 123 seconds on my drip-feed poison tarpit and then never come back, I got reading on how modern LLM scrapers might employ mechanisms to detect tarpits and blacklist.
During research I came across this tarpit evading scraper that provides some interesting insights into how modern LLM scrapers might do this.
https://github.com/Draconiator/Ipema
This gives me pause and has me looking at other solutions for counter-detection.
The GeoCities CSS is going nowhere.
-
@anaiscrosby After seeing ChatGPTBot blow 123 seconds on my drip-feed poison tarpit and then never come back, I got reading on how modern LLM scrapers might employ mechanisms to detect tarpits and blacklist.
During research I came across this tarpit evading scraper that provides some interesting insights into how modern LLM scrapers might do this.
https://github.com/Draconiator/Ipema
This gives me pause and has me looking at other solutions for counter-detection.
The GeoCities CSS is going nowhere.
@JulianOliver @anaiscrosby thank you
-
@anaiscrosby After seeing ChatGPTBot blow 123 seconds on my drip-feed poison tarpit and then never come back, I got reading on how modern LLM scrapers might employ mechanisms to detect tarpits and blacklist.
During research I came across this tarpit evading scraper that provides some interesting insights into how modern LLM scrapers might do this.
https://github.com/Draconiator/Ipema
This gives me pause and has me looking at other solutions for counter-detection.
The GeoCities CSS is going nowhere.
Interesting!! Based on my little experience implementing a similar tarpit using spigot (https://github.com/gw1urf/spigot) via @pengfold, I’ve noticed something pretty similar - bursts of activity (millions of hits/day) followed by long stretches of silence. From the intensity and patterns, it does seem like many scrapers aren’t consistently avoiding the tarpit, at least initially.
That said, I’d be a bit cautious about that conclusion. What you might be seeing isn’t necessarily "they can’t avoid it," but more like:
- some scrapers don’t try to detect tarpits (they just brute-force crawl and eat the cost)
- others probe once, flag it, and then blacklist it, hence the sudden silence
- and some operate in waves (rotating IPs / infrastructure), which can look like on/off behavior -
@anaiscrosby After seeing ChatGPTBot blow 123 seconds on my drip-feed poison tarpit and then never come back, I got reading on how modern LLM scrapers might employ mechanisms to detect tarpits and blacklist.
During research I came across this tarpit evading scraper that provides some interesting insights into how modern LLM scrapers might do this.
https://github.com/Draconiator/Ipema
This gives me pause and has me looking at other solutions for counter-detection.
The GeoCities CSS is going nowhere.
@JulianOliver Did you see this paper by Anthropic researchers? https://arxiv.org/abs/2510.07192
250 samples can poison even the largest models. That’s one webring! Even if detectable, might be a good way to avoid getting scraped?
-
@anaiscrosby After seeing ChatGPTBot blow 123 seconds on my drip-feed poison tarpit and then never come back, I got reading on how modern LLM scrapers might employ mechanisms to detect tarpits and blacklist.
During research I came across this tarpit evading scraper that provides some interesting insights into how modern LLM scrapers might do this.
https://github.com/Draconiator/Ipema
This gives me pause and has me looking at other solutions for counter-detection.
The GeoCities CSS is going nowhere.
@JulianOliver @anaiscrosby I haven't looked into tarpits but it smells to me very much like an "arms race" situation and there's no reason to think your side could prevail.
-
@JulianOliver @anaiscrosby I haven't looked into tarpits but it smells to me very much like an "arms race" situation and there's no reason to think your side could prevail.
@danstowell @anaiscrosby Winning would be nice, but I don't think it's always about prevailing. Just as a likelihood of failing need not undermine the will to act. Resistance, doing something, standing ground, rather than letting this predatorial broligarchy have their way.
Much of the time it's just about pushing back. If concerted, and at scale, it can indeed bring about tangible change.
-
@JulianOliver @anaiscrosby I haven't looked into tarpits but it smells to me very much like an "arms race" situation and there's no reason to think your side could prevail.
@danstowell @JulianOliver It’s about pushing back, not prevailing. “Tarpitting” has already emerged as a widely adopted response to AI, both a strategic approach and a meaningful act of resistance.
-
@danstowell @JulianOliver It’s about pushing back, not prevailing. “Tarpitting” has already emerged as a widely adopted response to AI, both a strategic approach and a meaningful act of resistance.
@anaiscrosby @JulianOliver Thanks. I see that it's been adopted. My concern is that it might cost us a lot developing these tarpits that have very little strategic effect if they become outmoded v quickly. But I really don't know - it's a very murky phase rn
-
@anaiscrosby After seeing ChatGPTBot blow 123 seconds on my drip-feed poison tarpit and then never come back, I got reading on how modern LLM scrapers might employ mechanisms to detect tarpits and blacklist.
During research I came across this tarpit evading scraper that provides some interesting insights into how modern LLM scrapers might do this.
https://github.com/Draconiator/Ipema
This gives me pause and has me looking at other solutions for counter-detection.
The GeoCities CSS is going nowhere.
@anaiscrosby Running a non-Markov tarpit for half an hour on one public link, and already have Claude lost in my swamp. Waiting to see if it runs into my ZIP bomb
---
216.73.216.124 - - [07/Apr/2026:03:28:49 +0200] "GET /tarpit/until/same/drive/harmattan_leftmost_intranscalency_few_ministries_few_between HTTP/2.0" 200 10132 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "-"
--- -
@anaiscrosby Running a non-Markov tarpit for half an hour on one public link, and already have Claude lost in my swamp. Waiting to see if it runs into my ZIP bomb
---
216.73.216.124 - - [07/Apr/2026:03:28:49 +0200] "GET /tarpit/until/same/drive/harmattan_leftmost_intranscalency_few_ministries_few_between HTTP/2.0" 200 10132 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "-"
---@anaiscrosby It hit it, but I guess decompressed in a thread. It's a 127M archive that decompresses to 128GB. The bot kept scraping for a bit and then dropped off. Difficult to know if it was a discouragement.
Strange is that soon after other IPs were reaching statistically non-guessable randomly generated URL paths, without touching the webroot or another other tarpit URL prior. They all had iOS UA strings (readily forged).
-
@anaiscrosby It hit it, but I guess decompressed in a thread. It's a 127M archive that decompresses to 128GB. The bot kept scraping for a bit and then dropped off. Difficult to know if it was a discouragement.
Strange is that soon after other IPs were reaching statistically non-guessable randomly generated URL paths, without touching the webroot or another other tarpit URL prior. They all had iOS UA strings (readily forged).
@JulianOliver @anaiscrosby
I've learnt about poison fountain and zip bomb before but never deployed any. It's interesting to read about your results.
Can I ask why you use Markov chain ? Wouldn't be enough, and simpler, to spit words at random ? Is that because totally random text would be easier to detect as a poison fountain ? Also, have you considered adding images (just coherent noise with irrelevant alt text) to the text ?
About the zip bomb, how do you filter to avoid accidentally bombing an innocent guy ? Checking the user-agent isn't an option, right ? Bad bots forge it anyway.
What about the legal aspect of these two methods, do you know ? -
@anaiscrosby It hit it, but I guess decompressed in a thread. It's a 127M archive that decompresses to 128GB. The bot kept scraping for a bit and then dropped off. Difficult to know if it was a discouragement.
Strange is that soon after other IPs were reaching statistically non-guessable randomly generated URL paths, without touching the webroot or another other tarpit URL prior. They all had iOS UA strings (readily forged).
It is quite wild how persistent Claude is, and an eerie feeling watching it just roam ever deeper into the endless rhizome of generated linked pages. It's been like this for a couple of hours now, and is not touching any other pages on the server, solely those in the tarpit. So that PoC does seem to check out.
CPU spikes are worrying, so will need to work the threading a bit and provision a couple more cores.
It has a rhythm of ~10-15s gorging, then a pause for 20-30s, and then at it again
-
M monkee@other.li shared this topic