No. Robots.txt is a set of polite instructions with no enforcement whatsoever. Well-behaved crawlers read it and comply. Anything built to deceive ignores it completely, because there is nothing to make it comply. So it shapes the behaviour of the bots you were never worried about and does nothing to the ones you are. It is not a security control.
Does robots.txt stop bad bots?Block Click Fraud2026-09-20T07:12:04+00:00