<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/">
	<channel>
		<title><![CDATA[Scivillage.com Casual Discussion Science Forum - Bots-Spiders-Scrapers ]]></title>
		<link>https://www.scivillage.com/</link>
		<description><![CDATA[Scivillage.com Casual Discussion Science Forum - https://www.scivillage.com]]></description>
		<pubDate>Sun, 03 May 2026 13:27:10 +0000</pubDate>
		<generator>MyBB</generator>
		<item>
			<title><![CDATA[Bots-Spiders-Scrapers]]></title>
			<link>https://www.scivillage.com/thread-19617.html</link>
			<pubDate>Fri, 16 Jan 2026 01:59:30 +0000</pubDate>
			<dc:creator><![CDATA[<a href="https://www.scivillage.com/member.php?action=profile&uid=1">stryder</a>]]></dc:creator>
			<guid isPermaLink="false">https://www.scivillage.com/thread-19617.html</guid>
			<description><![CDATA[If you are a the owner/developer of a bot/agent or AI,<br />
<br />
Please consider that if you abuse sites with over-scrapeing it doesn't just jeopardise your access to this one site.  <br />
Poor ettiquette can lead to poorly trained models since they likely find themselves with partial or no data at all.  <br />
<br />
I'd advise using or developing to be shared a read-only mirror server to scrape from, as this would lower site traffic from continual scrapeing.]]></description>
			<content:encoded><![CDATA[If you are a the owner/developer of a bot/agent or AI,<br />
<br />
Please consider that if you abuse sites with over-scrapeing it doesn't just jeopardise your access to this one site.  <br />
Poor ettiquette can lead to poorly trained models since they likely find themselves with partial or no data at all.  <br />
<br />
I'd advise using or developing to be shared a read-only mirror server to scrape from, as this would lower site traffic from continual scrapeing.]]></content:encoded>
		</item>
	</channel>
</rss>