small script help to veify websites...

[qwe]

anyone can hook me up with a simple php script where you can load a list of websites... then the script loads a website and checks view source for whatever text I specify, and if it finds text that I specified it will save that URL as a valid url into a valid.txt file, if it does not find what I specify in view source it will ignore it and put it into bad.txt ...

[Voodoo]

<?php
error_reporting(0);

$url=$_GET['url'];
$string='THE TEXT YOU WANT TO CHECK FOR';
$handle = fopen("$url", "rb");
$contents = stream_get_contents($handle);
$count=substr_count("$contents", "$string");



if($count > 0){
	echo "PASS";
}else{
	echo "FAIL";
}

?>

This should get you started.

[qwe]

Originally Posted by Voodoo

<?php
error_reporting(0);

$url=$_GET['url'];
$string='THE TEXT YOU WANT TO CHECK FOR';
$handle = fopen("$url", "rb");
$contents = stream_get_contents($handle);
$count=substr_count("$contents", "$string");



if($count > 0){
	echo "PASS";
}else{
	echo "FAIL";
}

?>

This should get you started.

I don't see a code where I load up .txt file with urls ?

Originally Posted by qwe

I don't see a code where I load up .txt file with urls ?

Off Voodoo's code:

PHP Code:

<?php
error_reporting(0);
$pass = array();
$fail = array();
if($urls = file('textfile.txt'))
{
    foreach($urls as $url)
    {
        $string = 'THE TEXT YOU WANT TO CHECK FOR';
        $handle = fopen($url, "rb");
        $contents = stream_get_contents($handle);
        $count=substr_count($contents, $string);
        if($count > 0)
        {
            $pass[] = $url;
        }else{
            $fail[] = $url;
        }
    }
}
?>

All you have to do is write $pass array to another text file for the pass urls, and same with $fail array to a fail text file.

[qwe]

Originally Posted by Killswitch

Off Voodoo's code:

PHP Code:

<?php
error_reporting(0);
$pass = array();
$fail = array();
if($urls = file('textfile.txt'))
{
    foreach($urls as $url)
    {
        $string = 'THE TEXT YOU WANT TO CHECK FOR';
        $handle = fopen($url, "rb");
        $contents = stream_get_contents($handle);
        $count=substr_count($contents, $string);
        if($count > 0)
        {
            $pass[] = $url;
        }else{
            $fail[] = $url;
        }
    }
}
?>

All you have to do is write $pass array to another text file for the pass urls, and same with $fail array to a fail text file.

ok i put in
$pass = array(pass.txt);
$fail = array(fail.txt);


when I run it, it's doing something, but i don't see good/bad urls going to txt files

That's because your doing it wrong.

Read this, should help: http://us2.php.net/fwrite

[qwe]

Originally Posted by Killswitch

That's because your doing it wrong.

Read this, should help: http://us2.php.net/fwrite

bro i dont know anything about php, even following that link I don't understand a thing

Well if you don't mind either waiting for someone to complete it for you, or until I finish writing my class that does exactly what you want, I'll give you the url to either view it (if I decide to give it free), or to buy it.

[GrouchyAdmin]

At least substr_count() isn't strstr(), which most people don't count '0' as being valid. But lots of luck with finding an exact pattern match. Might as well be doing 'view source' by hand.

If you're trying to confirm a link trade, just use a pre-existing script. There's tons to choose from.

[qwe]

can somebody finish the script, i'll epass $10 to whoever can get it right... script should be able to handle 2-5000 urls

[Iron Fist]

LOL $10 bucks.... that's awesome offers.

[qwe]

Originally Posted by sharphead

LOL $10 bucks.... that's awesome offers.

how much you want for few lines of code, especially with more then half done already ?

[ProG]

I'll chime in...

PHP Code:

<?php
    $search = "google"; // Your seach string
    $pass = "";
    $fail = "";
    
    if ( ( $websites = file( "websites.txt" ) ) !== false )
    {
        foreach( $websites as $url )
        {
            $url = trim( $url );
            $contents = file_get_contents( $url );
            if ( ( strpos( $contents, $search ) ) !== false )
            {
                $pass .= "{$url}\n";
            }
            else
            {
                $fail .= "{$url}\n";
            }
        }
    }
    
    if ( ( $handle = fopen( "valid.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $pass );
        fclose( $handle );
    }
    
    if ( ( $handle = fopen( "bad.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $fail );
        fclose( $handle );
    }
?>

You will need openssl. Click your Wampserver icon, goto PHP settings, goto PHP extensions, and click on php_openssl. Then restart the Apache service. Enjoy.

[qwe]

Originally Posted by ProG

I'll chime in...

PHP Code:

<?php
    $search = "google"; // Your seach string
    $pass = "";
    $fail = "";
    
    if ( ( $websites = file( "websites.txt" ) ) !== false )
    {
        foreach( $websites as $url )
        {
            $url = trim( $url );
            $contents = file_get_contents( $url );
            if ( ( strpos( $contents, $search ) ) !== false )
            {
                $pass .= "{$url}\n";
            }
            else
            {
                $fail .= "{$url}\n";
            }
        }
    }
    
    if ( ( $handle = fopen( "valid.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $pass );
        fclose( $handle );
    }
    
    if ( ( $handle = fopen( "bad.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $fail );
        fclose( $handle );
    }
?>

You will need openssl. Click your Wampserver icon, goto PHP settings, goto PHP extensions, and click on php_openssl. Then restart the Apache service. Enjoy.

hey bro thanks again one small thing, anyway to make it load 1 url at a time instead of all at once? like it will load one url and checks it before going to next one? also anyway to add a timeout between each check?

leave your epass id as well

[ProG]

What problem are you having? It is loading one URL at a time and a timeout seems unnecessary?

[qwe]

Originally Posted by ProG

What problem are you having? It is loading one URL at a time and a timeout seems unnecessary?

yah it's doing it real quick, i just need timeout option just in case for a project if you don't mind... like if timeout=0 then there's no timeout, if timeout=50 it's 50 seconds, etc...

[ProG]

You can use the sleep() function.

PHP Code:

<?php
    $search = "google"; // Your seach string
    $pass = "";
    $fail = "";
    
    if ( ( $websites = file( "websites.txt" ) ) !== false )
    {
        foreach( $websites as $url )
        {
            $url = trim( $url );
            $contents = file_get_contents( $url );
            if ( ( strpos( $contents, $search ) ) !== false )
            {
                $pass .= "{$url}\n";
            }
            else
            {
                $fail .= "{$url}\n";
            }
            sleep( 10 ); // 10 second timeout between URLs
        }
    }
    
    if ( ( $handle = fopen( "valid.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $pass );
        fclose( $handle );
    }
    
    if ( ( $handle = fopen( "bad.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $fail );
        fclose( $handle );
    }
?>

[qwe]

Originally Posted by ProG

You can use the sleep() function.

PHP Code:

<?php
    $search = "google"; // Your seach string
    $pass = "";
    $fail = "";
    
    if ( ( $websites = file( "websites.txt" ) ) !== false )
    {
        foreach( $websites as $url )
        {
            $url = trim( $url );
            $contents = file_get_contents( $url );
            if ( ( strpos( $contents, $search ) ) !== false )
            {
                $pass .= "{$url}\n";
            }
            else
            {
                $fail .= "{$url}\n";
            }
            sleep( 10 ); // 10 second timeout between URLs
        }
    }
    
    if ( ( $handle = fopen( "valid.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $pass );
        fclose( $handle );
    }
    
    if ( ( $handle = fopen( "bad.txt", 'at' ) ) !== false )
    {
        fwrite( $handle, $fail );
        fclose( $handle );
    }
?>

hrmm they don't add to txt every 10 seconds, like script loads, and after like 2-3 minutes do I get results in .txt ... does the script output results after it went through all of them? it doesn't add live ? just wondering

[ProG]

Yes, it creates the full text file before writing it. You could write each time but at 5000+ times that could cause some issues. Unless absolutely necessary, I wouldn't do it that way. Here is the code anyways..

PHP Code:

<?php
    $search = "google"; // Your seach string
    
    if ( ( $websites = file( "websites.txt" ) ) !== false )
    {
        foreach( $websites as $url )
        {
            $url = trim( $url );
            $contents = file_get_contents( $url );
            if ( ( strpos( $contents, $search ) ) !== false )
            {
                if ( ( $handle = fopen( "valid.txt", 'at' ) ) !== false )
                {
                    fwrite( $handle, "{$url}\n" );
                    fclose( $handle );
                }
            }
            else
            {
                if ( ( $handle = fopen( "bad.txt", 'at' ) ) !== false )
                {
                    fwrite( $handle, "{$url}\n" );
                    fclose( $handle );
                }
            }
            sleep( 10 ); // 10 second timeout between URLs
        }
    }
?>

[ProG]

Sorry that code has a problem in Windows, this should do the trick.

PHP Code:

<?php
    $search = "google"; // Your seach string
    
    if ( ( $websites = file( "websites.txt" ) ) !== false )
    {
        foreach( $websites as $url )
        {
            $url = trim( $url );
            $contents = file_get_contents( $url );
            if ( ( strpos( $contents, $search ) ) !== false )
            {
                if ( ( $handle = fopen( "valid.txt", 'at' ) ) !== false )
                {
                    $write = "{$url}\n";
                    fwrite( $handle, $write );
                    fclose( $handle );
                }
            }
            else
            {
                if ( ( $handle = fopen( "bad.txt", 'at' ) ) !== false )
                {
                    $write = "{$url}\n";
                    fwrite( $handle, $write );
                    fclose( $handle );
                }
            }
            sleep( 10 ); // 10 second timeout between URLs
        }
    }
?>

[GrouchyAdmin]

I've gotta admit, I've never seen a filemode of 'at' before.

[StaceyJo]

I suggest you hire a developer.

[ProG]

Windows offers a text-mode translation flag ('t') which will transparently translate \n to \r\n when working with the file. In contrast, you can also use 'b' to force binary mode, which will not translate your data. To use these flags, specify either 'b' or 't' as the last character of the mode  parameter.

[GrouchyAdmin]

Originally Posted by ProG

Windows offers a text-mode translation flag ('t') which will transparently translate \n to \r\n when working with the file. In contrast, you can also use 'b' to force binary mode, which will not translate your data. To use these flags, specify either 'b' or 't' as the last character of the mode  parameter.

Again, that's the first time I've ever seen it in practice. Most people who expect to run multiplatform - or even on Windows, specifically, use the double-escaped-posix path and hardcode the \r\n into some of the print logic. I suppose 't' has it's place but as un-UTF as PHP can be, I'd really, really hate to allow for translation. That also explains the use of trim().

From a design standpoint - other than it since being asked for, I'm curious why you killed the pass/fail arrays - that was the only part of the code that I thought was salvageable.

[qwe]

Originally Posted by ProG

Windows offers a text-mode translation flag ('t') which will transparently translate \n to \r\n when working with the file. In contrast, you can also use 'b' to force binary mode, which will not translate your data. To use these flags, specify either 'b' or 't' as the last character of the mode  parameter.

that's like reading in Chinese to me btw, that last script is just like first, only after it finished its task, it writes to .txt but that's fine (i was just thinking what if it hangs up sometime, that way whatever it checked would be already saved in txt instead of waiting until it's done) thanks again, leave your epass i'll throw you some $ for all your help

[ProG]

Originally Posted by GrouchyAdmin

Again, that's the first time I've ever seen it in practice. Most people who expect to run multiplatform - or even on Windows, specifically, use the double-escaped-posix path and hardcode the \r\n into some of the print logic. I suppose 't' has it's place but as un-UTF as PHP can be, I'd really, really hate to allow for translation. That also explains the use of trim().

From a design standpoint - other than it since being asked for, I'm curious why you killed the pass/fail arrays - that was the only part of the code that I thought was salvageable.

Yes it's the first time I've used it as well. I do not typically code for Windows ;) As for the arrays, I really see no need to build two arrays then convert them to a string for writing, why not just build a string? Unless the arrays are used for something later, such as removing duplicates, why build them?

[qwe]

i don't get it, why did it stop working, everything goes to invalid txt now.... hrm

PHP Code:

<?php
$Sites = array('google.com', 'search.yahoo.com', 'godaddy.com');
$Scan = new CheckURL($Sites, 'search');
foreach($Scan->Pass() as $ID => $Site)
{
    echo $Site."\n";
}
?>

Mmmm, looking sexy!