GenZ News

Article View

AI site Perplexity uses “stealth tactics” to flout no-crawl edicts, Cloudflare says

204d ago
Technology
Ars Technica
Technology and innovation
AI-powered search platform Perplexity is facing renewed allegations of scraping websites without permission. Cloudflare has accused Perplexity of using "stealth tactics" to circumvent website restrictions, specifically no-crawl directives. These directives, often implemented via robots.txt files, instruct web crawlers to avoid indexing certain content. The accusations add to a growing list of concerns regarding Perplexity's data acquisition methods and raise questions about ethical web crawling practices. This controversy highlights the ongoing debate surrounding AI companies' reliance on web data and the need for clearer guidelines on data usage and attribution.
Source