gedaliyah@lemmy.world to Technology@lemmy.worldEnglish · 4 months agoReddit blocking all major search engines, except Googlereadwrite.comexternal-linkmessage-square187fedilinkarrow-up1679arrow-down118
arrow-up1661arrow-down1external-linkReddit blocking all major search engines, except Googlereadwrite.comgedaliyah@lemmy.world to Technology@lemmy.worldEnglish · 4 months agomessage-square187fedilink
minus-squaredaniskarma@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up2·4 months agoIf you publish your data in a friendly manner bots would have no need to crawl your site. Data that is more interesting and requested a lot could even be served over p2p. This moderl would generate less cost that dealing with constant bot scrappers. It is not a technical discussion. Or a discussion about associated cost. It’s a discussion about morals and economic models.
If you publish your data in a friendly manner bots would have no need to crawl your site.
Data that is more interesting and requested a lot could even be served over p2p.
This moderl would generate less cost that dealing with constant bot scrappers.
It is not a technical discussion. Or a discussion about associated cost. It’s a discussion about morals and economic models.