fix(app): Add Disallow: to robots.txt

arthurvr · arthurvr · commit fd942ce9a29b · 2015-04-13T14:28:26.000+02:00
According to the robots.txt standard: " The record starts with one or more User-agent lines, followed by one or more Disallow lines, as detailed below. " This is also encouraged by the major search engines. - Baidu: https://www.baidu.com/search/robots_english.html - Google: https://developers.google.com/webmasters/control-crawl-index/docs/getting_started https://www.youtube.com/watch?v=P7GY1fE5JQQ - Yandex: help.yandex.com/webmaster/controlling-robot/robots-txt.xml Ref yeoman/generator-webapp#220 h5bp/html5-boilerplate#1487 https://www.robotstxt.org/orig.html
diff --git a/templates/common/app/robots.txt b/templates/common/app/robots.txt
@@ -1,3 +1,4 @@
 # robotstxt.org
 
 User-agent: *
+Disallow:

Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,4 @@`
`1`	`1`	`# robotstxt.org`
`2`	`2`
`3`	`3`	`User-agent: *`
	`4`	`+Disallow:`