View Single Post
Old 10-20-2017, 08:02 PM  
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 380
t's the first thing I always do, I ask here when I find nothing

I found a lot of solutions for wordpres, but my sites are not WordPress, and I understand very little url_rewriter...

I wrote these rules a long time ago, following guides, and I never saw this error before, until I started to manage my server...

My url_revriter is really simple and stupid, and the error does not give many clues to understand what creates it.

Code:
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Yandex [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Baidu [NC]
RewriteRule .*  - [L,F]


RewriteRule ^page-([^/]+)/$ index.php?page=$1 [L]

RewriteRule ^([^/]+)\.html$ video.php?title=$1 [L]

RewriteRule channels/ channels.php [L]

RewriteRule tags/ tags.php [L]

RewriteRule ^tags-([^/]+)\/$ tags.php?letter=$1 [L]

RewriteRule search/ search.php [L]

RewriteRule ^search-([^/]+)$ search.php?query=$1 [L]
RewriteRule ^search-([^/]+)\/$ search.php?query=$1 [L]
RewriteRule ^page-([^/]+)/search-([^/]+)/$ /search.php?page=$1&query=$2 [L]

RewriteRule ^search-([^/]+)\/([^/]+)\/$ search.php?query=$1&sort=$2 [L]
RewriteRule ^page-([^/]+)/search-([^/]+)/([^/]+)/$ /search.php?page=$1&query=$2&sort=$3 [L]

RewriteRule ^([0-9]+)$ out.php?id_photo=$1 [L]
RewriteRule ^([0-9]+)/$ out.php?id_photo=$1 [L]
RewriteRule ^([0-9]+)-([^/]+)$ out.php?id_photo=$1 [L]

But I also found another thing... Last week Google decided to scan my sites, all together, and a lot of pages of each site...

Now I'm thinking, I don't know if...

Wait a few days, maybe, when Google has finished this scan the server returns to a normal regime.

Or

Try to limit its consumption by Google, adding for example if-midified-since and Last-Modified in the headers of my pages

There were already, but time ago I had to comment on why they created problems with some crappy VPS
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote