Quote:
Originally Posted by freecartoonporn
what search engine you guys using ?
i used sphinx before but it uses cpu a lot. and i had only ~7 mil records
|
Sphinx won't do the job.
This is clustered nodes of individual components. Crawling is seperate from Indexing. Ranking is seperate from Indexing. Crawls are performed through caching proxies on their own servers.
You could do this with Nutch and ElasticSearch but the overhead would be much greater than this system has.