.Google confines robots.txt help to 4 areas, clarifying its posture on unsupported directives.Google.com simply supports 4 details robots.txt fields.Unsupported instructions in robots.txt will definitely be overlooked.Take into consideration bookkeeping your robots.txt documents taking into account this upgrade.