RE: https://tldr.nettime.org/@tante/116605858023186072
-
@commons_protocol Brave is just Google Chrome in a fancy overcoat - it's still contributing to Google's dominance over the web (even if you ignore the other problems with it)
@irina My understanding is that Brave is an independent instance and remains privacy focussed with an established, fully independent search index, remaining unaffected by Google's algorithmic updates or interface changes.
I understand it uses its own distinct features and privacy-focused architecture, without adopting Google’s specific changes.
Nevertheless, google has dominated the web for about 25 years already and I don't doubt there are systemic issues at foot either way and the whole sea is shifting!
-
Quick strategy discussion, for those who understand Google indexing and SEO:
If I want to yank a web site out of Google’s now-fully-extractive search, should I (1) disallow googlebot in robots.txt or (2) add `<meta name="googlebot" content="noindex">` to all the page headers?
The goal here is not just to remove my contributions to the commons from Google’s results, but to •make Google aware• that sites are pulling consent. What will best do that?
2/2
@inthehands if you want to remove it immediately, as unintuative as it sounds, claiming ownership of the site in Google Search Console will allow you to remove any URL in their index from being included in results, but only temporarily. I think you have to renew the exclude every 45 days or something link that. If a random blog excludes their content, it won't impact Google at all. If Wikipedia were to do that, the trust in the accuracy of Google's results would start to be impacted.
-
RE: https://tldr.nettime.org/@tante/116605858023186072
Google Search rests on a social contract: their bots can crawl our sites, they can index our sites, and they can show excerpts of our sites because
and •only because•
they send people to our sites. •Our• sites, our words, with our design, with our links, with our context and our aesthetics, shared the way we want to share them.
Google is announcing — unambiguously and with great fanfare — that they are now fully breaking that already-ragged contract. We should reciprocate.
1/2
@inthehands
"Get in, loser. We're doing web rings again." -
@gherhartd @inthehands Use Brave browser and Brave search, or equivalent, might be one way
Brave’s leadership is too personally despicable for me to consider their products
-
@inthehands My site is tiny, but I had to make sure and add that into my own. I don't care if Google never happens to come across it, the idea of them using my site to train their LLM is sickening.
I also have to say, your site is absolutely gorgeous. -
OK, a •lot• of replies need this reponse:
Yes, of •course• they will start ignoring robots.txt etc as soon as they think it hurts their business. Of course.
It is important to •force that fight•, rather than just capitulating in advance.
@inthehands Google executives don't care about consent. Google executives care about their bottom line.
Copyright class action seems most promising. Along with adversarial material to poison their scrapers
-
RE: https://tldr.nettime.org/@tante/116605858023186072
Google Search rests on a social contract: their bots can crawl our sites, they can index our sites, and they can show excerpts of our sites because
and •only because•
they send people to our sites. •Our• sites, our words, with our design, with our links, with our context and our aesthetics, shared the way we want to share them.
Google is announcing — unambiguously and with great fanfare — that they are now fully breaking that already-ragged contract. We should reciprocate.
1/2
@inthehands How to keep Google from indexing your website
Official docs from Google:
<https://developers.google.com/search/docs/crawling-indexing/block-indexing>Other ways:
<https://exertpro.com/how-can-i-keep-google-from-indexing-my-website/>Other specific search engines may have their own tools for blocking their engine from indexing a web site or specific pages within it.
-
@inthehands
"Get in, loser. We're doing web rings again."@jargoggles @inthehands finally my chance to join the "ate my balls" web ring
https://iowastatedaily.com/218801/uncategorized/eating-balls-on-the-internet/
-
OK, a •lot• of replies need this reponse:
Yes, of •course• they will start ignoring robots.txt etc as soon as they think it hurts their business. Of course.
It is important to •force that fight•, rather than just capitulating in advance.
@inthehands In that case, it's better if we send them poisoned data instead, using iocaine [0] or nepentheses [1].
[0]: https://iocaine.madhouse-project.org
[1]: https://zadzmo.org/code/nepenthes/ -
Defeatism is a form of surrender. Cynicism is surrender. Despair is surrender. Nihilism is surrender.
Our job is to •care• and to •keep caring• and to •keep doing and keep building• and to •endure• longer than them.
@inthehands I needed to read this today.
-
@inthehands this is a fence-post defense against this, google Will Not Care
just start poisoning the data once you detect that google is the one fetching it, just absolutely fucking destroy their LLM output
@ShadowJonathan @inthehands what if we honeypotted them? Get them to ingest gigabytes upon gigabytes of LLM generated text to ruin their.efforts?
-
@inthehands you can block their bots at the network level
@glassresistor @inthehands are you doing this in a particular way? Basically looking for different approaches.
-
Going with meta noindex for now. My thinking is that this actively tells Google to yank already-crawled content from their index, whereas they might take a robots.txt entry to mean “do not update, but keep showing last fetched.”
@inthehands I found a google note on meta tag so saying that if you use robots.txt to exclude googlebot, it won’t be able to crawl your pages to honor noindex, so if another site google does index points to a page on your site, the link will still be indexed.
https://developers.google.com/search/docs/crawling-indexing/block-indexing
-
Defeatism is a form of surrender. Cynicism is surrender. Despair is surrender. Nihilism is surrender.
Our job is to •care• and to •keep caring• and to •keep doing and keep building• and to •endure• longer than them.
@inthehands annoy people into doing better.
-
@inthehands As I said just a while ago: Every big tech press event these last few years have felt like "Announcing our exciting plans for oligarchs to strip-mine the entire world and immiserate all of humanity! Get on board, and also death to the unbelievers!"
@inthehands@hachyderm.io @datarama@hachyderm.io
Here's how I've seen the response to Google's latest bullshit:
-
@glassresistor @inthehands are you doing this in a particular way? Basically looking for different approaches.
@rooneymcnibnug @inthehands filtering by user agent, ip address, cloudfront no bots acl config, by load
-
@rooneymcnibnug @inthehands filtering by user agent, ip address, cloudfront no bots acl config, by load
@glassresistor @inthehands oh okay I was misinterpreting the statement, my bad
-
@inthehands
There is a new fad called "data poisoning" that web sites are using to foil ai scraping. One music site put a Homer Simpson monologue into every track in its online data base. It starts a few seconds in and continues to the end. That's only one way it's being used. We need a generation of ai "monkey wrench gangs " to start sabotaging. It's really no different than what Edward Abbey talked about, instead of extractive earth raping machinery being targeted , it's data mining machinery. -
Quick strategy discussion, for those who understand Google indexing and SEO:
If I want to yank a web site out of Google’s now-fully-extractive search, should I (1) disallow googlebot in robots.txt or (2) add `<meta name="googlebot" content="noindex">` to all the page headers?
The goal here is not just to remove my contributions to the commons from Google’s results, but to •make Google aware• that sites are pulling consent. What will best do that?
2/2
@inthehands hi! Did anyone answer to your question here? I'm very interested in blocking the bots, but everyone seems so eager to give their fucking opinions instead of answering this, that I couldn't find a proper way of blocking said bots. Thanks!
-
@gherhartd @inthehands Use Brave browser and Brave search, or equivalent, might be one way
@commons_protocol @inthehands the problem isn't our own usage, the problem is the hegemonic place of Chrome. Forbidding Chrome browser from accessing one's site is almost completely cutting it from the web, from nearly everybody.
