Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Googlebot actually used to support a noindex rule in robots.txt, but they are removing it.

https://webmasters.googleblog.com/2019/07/a-note-on-unsuppor...



Yes, that was linked above. It looks like this is part of reducing support to what's in the spec?


Oops, yep. I didn't see that context.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: