Tech How to index AVS tube? (robots.txt issue) - GoFuckYourself.com

Ramirez · 02-15-2016, 12:15 PM

Hi guys,

I added a tube (with AVS script) in Google Webmaster Tools, but the issue is with the robots.txt file which is blocking everything.

Here is my AVS tube robots.txt:

Code:

User-agent: *
Disallow: /siteadmin/
Disallow: /cache/
Disallow: /include/
Disallow: /install/
Disallow: /language/
Disallow: /media/
Disallow: /tmp/
Disallow: /classes/
Disallow: /templates/
Disallow: /scripts/
Disallow: /addons/
Disallow: /cron/
Disallow: /contrib/

Any idea how to fix this and get the tube indexed?

Thank you!

magneto664 · 02-15-2016, 12:43 PM

Code:

Allow: /

Ramirez · 02-17-2016, 01:36 AM

Thanks for the reply.

I've added:

Code:

Allow: /

And didn't change anything else in the robots.txt, only added "Allow: /"

I still get errors in Webmaster Tools.

Code:

Warnings 

Url blocked by robots.txt.

Sitemap contains urls which are blocked by robots.txt.

Issues: 272

Value: http://www.domain.com/media/videos/tmb/1/1.jpg
-
Feb 17, 2016
Value: http://www.domain.com/media/videos/tmb/10/1.jpg
-
Feb 17, 2016
Value: http://www.domain.com/media/videos/tmb/100/1.jpg
-
Feb 17, 2016

Now, I have to mention I've added both versions of site, www. and non-www.

02-15-2016, 12:15 PM	#1
Ramirez Confirmed User Industry Role: Join Date: May 2011 Location: Europe Posts: 1,590	How to index AVS tube? (robots.txt issue) Hi guys, I added a tube (with AVS script) in Google Webmaster Tools, but the issue is with the robots.txt file which is blocking everything. Here is my AVS tube robots.txt: Code: User-agent: * Disallow: /siteadmin/ Disallow: /cache/ Disallow: /include/ Disallow: /install/ Disallow: /language/ Disallow: /media/ Disallow: /tmp/ Disallow: /classes/ Disallow: /templates/ Disallow: /scripts/ Disallow: /addons/ Disallow: /cron/ Disallow: /contrib/ Any idea how to fix this and get the tube indexed? Thank you! __________________ Adult SEO Best Cam Affiliate Program!

02-15-2016, 12:43 PM	#2
magneto664 God Bless You Industry Role: Join Date: Aug 2014 Location: Glasgow, $cotland Posts: 1,467	Code: Allow: / __________________ magneto664 📧 gmail.com Adult Backlinks 💘Best Website Stats 💘 Best CDN for Adult Content My Fav: 👍Chaturbate 👍 Stripchat 👍 Dateprofits 👍 AdultFriendFinder

02-17-2016, 01:36 AM	#3
Ramirez Confirmed User Industry Role: Join Date: May 2011 Location: Europe Posts: 1,590	Thanks for the reply. I've added: Code: Allow: / And didn't change anything else in the robots.txt, only added "Allow: /" I still get errors in Webmaster Tools. Code: Warnings Url blocked by robots.txt. Sitemap contains urls which are blocked by robots.txt. Issues: 272 Value: http://www.domain.com/media/videos/tmb/1/1.jpg - Feb 17, 2016 Value: http://www.domain.com/media/videos/tmb/10/1.jpg - Feb 17, 2016 Value: http://www.domain.com/media/videos/tmb/100/1.jpg - Feb 17, 2016 Now, I have to mention I've added both versions of site, www. and non-www. __________________ Adult SEO Best Cam Affiliate Program!