Robots.txt Disallows All User-Agents

Sep 01, 2016 18:20


Title
Robots.txt Disallows All User-Agents

Short, concise description of the idea
Robots.txt disallows all user-agents, which means that my livejournal cannot be indexed by search engines.

Full description of the idea
Hi,

Despite setting my journal to public, and unchecking "minimize inclusion to search engines," my robots.txt remains the same, it blocks all search engines. What's the use of having a blog if not crawled by search engines anyway? How would one get new visitors?

My LiveJournal URL is: http://skillsfuture.livejournal.com/
Robots.txt: http://skillsfuture.livejournal.com/robots.txt

Regards,
An ordered list of benefits
  • It's user-friendly.
An ordered list of problems/issues involved
  • Robots.txt is not changed despite setting privacy setting to PUBLIC.
Previous post Next post
Up