<![CDATA[RE: https://tldr.nettime.org/@tante/116605858023186072]]>RE: https://tldr.nettime.org/@tante/116605858023186072

Google Search rests on a social contract: their bots can crawl our sites, they can index our sites, and they can show excerpts of our sites because

and •only because•

they send people to our sites. •Our• sites, our words, with our design, with our links, with our context and our aesthetics, shared the way we want to share them.

Google is announcing — unambiguously and with great fanfare — that they are now fully breaking that already-ragged contract. We should reciprocate.

1/2

]]>https://forum.other.li/topic/a24315f0-d3ed-487d-869c-74d65e991495/re-https-tldr.nettime.org-@tante-116605858023186072RSS for NodeFri, 05 Jun 2026 01:06:23 GMTWed, 20 May 2026 14:49:24 GMT60<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 08:51:31 GMT]]>@Coho @inthehands So the music site exists not for humans, but solely to fuck up AI scrapers?

]]>https://forum.other.li/post/https://wandering.shop/users/ergative/statuses/116611710398202760https://forum.other.li/post/https://wandering.shop/users/ergative/statuses/116611710398202760Thu, 21 May 2026 08:51:31 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 07:09:06 GMT]]>@amyworrall @inthehands I don't get the "Swiper, no swiping"..?
The effect is "Hey, here you see a page which must not be indexed in google search" -> google sees it, reads it and won't index it, regardless of other signals (links, sitemap etc.). robots.txt forbids access to the page. -> Google knows nothing about page content, but may index the URL, if its found elsewhere (external link, sitemap, whatever). My experience: with no further information -> You get a very ugly SERP snippet 🙂

]]>https://forum.other.li/post/https://nrw.social/users/korrupt/statuses/116611307682387277https://forum.other.li/post/https://nrw.social/users/korrupt/statuses/116611307682387277Thu, 21 May 2026 07:09:06 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 07:04:51 GMT]]>@inthehands
Some websites are being deindexed but they're still being crawled by Google
https://social.wikidex.net/@ciencia/116611028083997779

]]>https://forum.other.li/post/https://social.wikidex.net/users/ciencia/statuses/116611291010154853https://forum.other.li/post/https://social.wikidex.net/users/ciencia/statuses/116611291010154853Thu, 21 May 2026 07:04:51 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 06:37:42 GMT]]>@inthehands

I’m mostly worried about their “agentic” part, because that sounds like new infrastructure with possibly different user agents etc., so harder to ban, and I’m 💯 sure it will DEFINITELY have no “social contract” whatsoever.

]]>https://forum.other.li/post/https://infosec.exchange/users/avuko/statuses/116611184245515522https://forum.other.li/post/https://infosec.exchange/users/avuko/statuses/116611184245515522Thu, 21 May 2026 06:37:42 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 06:37:00 GMT]]>@inthehands all the bots that are used to train AI, don't care about robots.txt, or meta stuff. Otherwise, it will be much more harder to train stuff.

]]>https://forum.other.li/post/https://mastodon.social/users/eickot/statuses/116611181480233328https://forum.other.li/post/https://mastodon.social/users/eickot/statuses/116611181480233328Thu, 21 May 2026 06:37:00 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 05:59:02 GMT]]>@joe @ShadSterling @inthehands there is one in Java?

]]>https://forum.other.li/post/https://todon.eu/users/FantasmitaAsex/statuses/116611032191320440https://forum.other.li/post/https://todon.eu/users/FantasmitaAsex/statuses/116611032191320440Thu, 21 May 2026 05:59:02 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 05:41:55 GMT]]>@commons_protocol @inthehands the problem isn't our own usage, the problem is the hegemonic place of Chrome. Forbidding Chrome browser from accessing one's site is almost completely cutting it from the web, from nearly everybody.

]]>https://forum.other.li/post/https://ludosphere.fr/users/gherhartd/statuses/116610964865212648https://forum.other.li/post/https://ludosphere.fr/users/gherhartd/statuses/116610964865212648Thu, 21 May 2026 05:41:55 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 05:37:45 GMT]]>@inthehands hi! Did anyone answer to your question here? I'm very interested in blocking the bots, but everyone seems so eager to give their fucking opinions instead of answering this, that I couldn't find a proper way of blocking said bots. Thanks!

]]>https://forum.other.li/post/https://xarxa.cloud/users/iakobsdesamos/statuses/116610948495126651https://forum.other.li/post/https://xarxa.cloud/users/iakobsdesamos/statuses/116610948495126651Thu, 21 May 2026 05:37:45 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 04:52:13 GMT]]>@Coho @inthehands

How is the data laundering machine not *also* raping earth? Bad things aren't mutually exclusive

]]>https://forum.other.li/post/https://ieji.de/ap/users/116176293715755380/statuses/116610769435614995https://forum.other.li/post/https://ieji.de/ap/users/116176293715755380/statuses/116610769435614995Thu, 21 May 2026 04:52:13 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:51:11 GMT]]>@glassresistor @inthehands oh okay I was misinterpreting the statement, my bad

]]>https://forum.other.li/post/https://mastodon.social/users/rooneymcnibnug/statuses/116610529460142541https://forum.other.li/post/https://mastodon.social/users/rooneymcnibnug/statuses/116610529460142541Thu, 21 May 2026 03:51:11 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:47:58 GMT]]>@rooneymcnibnug @inthehands filtering by user agent, ip address, cloudfront no bots acl config, by load

]]>https://forum.other.li/post/https://sfba.social/users/glassresistor/statuses/116610516843720991https://forum.other.li/post/https://sfba.social/users/glassresistor/statuses/116610516843720991Thu, 21 May 2026 03:47:58 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Fri, 29 May 2026 22:08:34 GMT]]>@inthehands@hachyderm.io @datarama@hachyderm.io

Here's how I've seen the response to Google's latest bullshit:

]]>https://forum.other.li/post/https://app.wafrn.net/fediverse/post/9a26b7d9-186d-4e9a-bb44-3f7e9b7545bahttps://forum.other.li/post/https://app.wafrn.net/fediverse/post/9a26b7d9-186d-4e9a-bb44-3f7e9b7545baFri, 29 May 2026 22:08:34 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:42:52 GMT]]>@inthehands annoy people into doing better.

]]>https://forum.other.li/post/https://lgbtqia.space/users/MxVerda/statuses/116610496750777599https://forum.other.li/post/https://lgbtqia.space/users/MxVerda/statuses/116610496750777599Thu, 21 May 2026 03:42:52 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:36:51 GMT]]>@inthehands I found a google note on meta tag so saying that if you use robots.txt to exclude googlebot, it won’t be able to crawl your pages to honor noindex, so if another site google does index points to a page on your site, the link will still be indexed.

https://developers.google.com/search/docs/crawling-indexing/block-indexing

]]>https://forum.other.li/post/https://zirk.us/users/cratermoon/statuses/116610473115461887https://forum.other.li/post/https://zirk.us/users/cratermoon/statuses/116610473115461887Thu, 21 May 2026 03:36:51 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:36:45 GMT]]>@glassresistor @inthehands are you doing this in a particular way? Basically looking for different approaches.

]]>https://forum.other.li/post/https://mastodon.social/users/rooneymcnibnug/statuses/116610472717541960https://forum.other.li/post/https://mastodon.social/users/rooneymcnibnug/statuses/116610472717541960Thu, 21 May 2026 03:36:45 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:27:51 GMT]]>@ShadowJonathan @inthehands what if we honeypotted them? Get them to ingest gigabytes upon gigabytes of LLM generated text to ruin their.efforts?

]]>https://forum.other.li/post/https://furry.engineer/users/fluffykittycat/statuses/116610437727482336https://forum.other.li/post/https://furry.engineer/users/fluffykittycat/statuses/116610437727482336Thu, 21 May 2026 03:27:51 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:11:10 GMT]]>@inthehands I needed to read this today.

]]>https://forum.other.li/post/https://infosec.exchange/users/NosirrahSec/statuses/116610372127851609https://forum.other.li/post/https://infosec.exchange/users/NosirrahSec/statuses/116610372127851609Thu, 21 May 2026 03:11:10 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:09:02 GMT]]>@inthehands In that case, it's better if we send them poisoned data instead, using iocaine [0] or nepentheses [1].

[0]: https://iocaine.madhouse-project.org
[1]: https://zadzmo.org/code/nepenthes/

]]>https://forum.other.li/post/https://indieweb.social/users/andrew2point5k/statuses/116610363697815800https://forum.other.li/post/https://indieweb.social/users/andrew2point5k/statuses/116610363697815800Thu, 21 May 2026 03:09:02 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 03:04:56 GMT]]>@jargoggles @inthehands finally my chance to join the "ate my balls" web ring

https://iowastatedaily.com/218801/uncategorized/eating-balls-on-the-internet/

]]>https://forum.other.li/post/https://nyan.lol/users/waffle_iron/statuses/116610347583024138https://forum.other.li/post/https://nyan.lol/users/waffle_iron/statuses/116610347583024138Thu, 21 May 2026 03:04:56 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 02:44:39 GMT]]>@inthehands How to keep Google from indexing your website

Official docs from Google:
<https://developers.google.com/search/docs/crawling-indexing/block-indexing>

Other ways:
<https://exertpro.com/how-can-i-keep-google-from-indexing-my-website/>

Other specific search engines may have their own tools for blocking their engine from indexing a web site or specific pages within it.

]]>https://forum.other.li/post/https://mastodon.social/users/dancingtreefrog/statuses/116610267826299857https://forum.other.li/post/https://mastodon.social/users/dancingtreefrog/statuses/116610267826299857Thu, 21 May 2026 02:44:39 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 02:14:47 GMT]]>@inthehands Google executives don't care about consent. Google executives care about their bottom line.

Copyright class action seems most promising. Along with adversarial material to poison their scrapers

]]>https://forum.other.li/post/https://mas.to/users/cykonot/statuses/116610150407862898https://forum.other.li/post/https://mas.to/users/cykonot/statuses/116610150407862898Thu, 21 May 2026 02:14:47 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 02:05:34 GMT]]>@goetic

Thank you!! My site is a labor of love, handmade from the ground up for the few people who will find it.

]]>https://forum.other.li/post/https://hachyderm.io/users/inthehands/statuses/116610114159187809https://forum.other.li/post/https://hachyderm.io/users/inthehands/statuses/116610114159187809Thu, 21 May 2026 02:05:34 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 02:02:59 GMT]]>@commons_protocol @gherhartd

Brave’s leadership is too personally despicable for me to consider their products

]]>https://forum.other.li/post/https://hachyderm.io/users/inthehands/statuses/116610104012669317https://forum.other.li/post/https://hachyderm.io/users/inthehands/statuses/116610104012669317Thu, 21 May 2026 02:02:59 GMT<![CDATA[Reply to RE: https://tldr.nettime.org/@tante/116605858023186072 on Thu, 21 May 2026 02:01:47 GMT]]>@inthehands
"Get in, loser. We're doing web rings again."

]]>https://forum.other.li/post/https://kolektiva.social/users/jargoggles/statuses/116610099286457516https://forum.other.li/post/https://kolektiva.social/users/jargoggles/statuses/116610099286457516Thu, 21 May 2026 02:01:47 GMT