Only Google is really allowed to crawl the web

957 points by skinkestek

Read the article

Discussion on Hacker News