The huge status of the Search Engine Giant – Google has always  led me to explore the technologies it uses to power itself. This has led me to explore the all new world of Web Crawlers ,  Indexers,  Full Text Search,  Map Reduce File Systems like Hadoop and lots of more techy stuff. The first thing I wanted to start with was writing a web crawler...