Source : Free On-Line Dictionary of Computing

standard for robot exclusion
     
         A proposal to try to prevent the havoc
        wreaked by many of the early {World-Wide Web} {robots} when
        they retrieved documents too rapidly or retrieved documents
        that had side effects (such as voting).  The proposed standard
        for robot exclusion offers a solution to these problems.
     
        {(http://info.webcrawler.com/mak/projects/robots/norobots.html)}.
     
        (1996-03-23)