GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Google to make robots.txt an Internet standard after 25 years

trevesty · 07-02-2019, 05:47 AM

Been running websites for over 15 years and making money from it. Tens of thousands of sites at least...

And every single one of them has had a robots.txt file. I don't see the issue.

07-02-2019, 05:47 AM
trevesty Confirmed User Industry Role: Join Date: Aug 2006 Location: Midwest Posts: 3,789	Been running websites for over 15 years and making money from it. Tens of thousands of sites at least... And every single one of them has had a robots.txt file. I don't see the issue.