View Post [edit]
Poster: | gojomo | Date: | Apr 7, 2011 2:36pm |
Forum: | web | Subject: | Forum reminders: adds, removes |
This post was modified by gojomo on 2011-04-07 21:36:52
Reply [edit]
Poster: | purplemath | Date: | Jan 23, 2011 5:33am |
Forum: | web | Subject: | Re: Forum reminders: adds, removes |
Thank you.
Reply [edit]
Poster: | gojomo | Date: | Jan 24, 2011 3:30pm |
Forum: | web | Subject: | Re: Forum reminders: adds, removes |
http://waybackmachine.org
...for the most recent material.
- Gordon @ IA
Reply [edit]
Poster: | schwarma | Date: | Feb 21, 2011 9:45am |
Forum: | web | Subject: | Re: Forum reminders: adds, removes |
Reply [edit]
Poster: | gojomo | Date: | Feb 22, 2011 3:23pm |
Forum: | web | Subject: | Re: Forum reminders: adds, removes |
Any changes to your robots.txt should be noticed within 24 hours, perhaps much sooner. (Active crawling usually requests robots.txt just before visiting a site, and then every 6-12 hours thereafter. The Wayback Machine generally checks robots.txt as needed, but can remember a previous version for up to 24 hours.)
If it has been more than 24 hours and your changes do not seem to have been noticed, and using an online robots.txt validator suggests all is well with your format, please let us know and we'll investigate further.
- Gordon @ IA