Fix: Set common robots.txt to all virtual servers in Virtualmin

[Saman]

Go to Server Templates and see whether the skeleton folder it located at /etc/skel. Create a subfolder within it as public_html and store your robots.txt file. I will attach a sample robots.txt file. When you do that, robots.txt will be copied to all your newly created servers.

Sample robots.txt

Code: Select all

User-agent: Applebot
Allow: /

User-agent: baiduspider
Allow: /

User-agent: Bingbot
Allow: /

User-agent: Googlebot
Allow: /

User-agent: ia_archiver
Allow: /

User-agent: msnbot
Allow: /

User-agent: Naverbot
Allow: /

User-agent: seznambot
Allow: /

User-agent: Slurp
Allow: /

User-agent: teoma
Allow: /

User-agent: Twitterbot
Allow: /

User-agent: Yandex
Allow: /

User-agent: Yeti
Allow: /

User-agent: *
Disallow: /