|
Poster:
|
Jeff Kaplan |
Date:
|
September 12, 2011 10:38:20pm |
|
Forum:
|
web
|
Subject:
|
Re: Why is Archived content purged retroactively? [Was: Re: Ezboard content suddenly not available in the new system - why?] |
i was referring to questions regarding robots.txt blocking site. i revised it to better state that. sorry if you misunderstood.
This post was modified by Jeff Kaplan on 2011-09-13 05:38:20
|
Poster:
|
jory2 |
Date:
|
September 13, 2011 06:33:11am |
|
Forum:
|
web
|
Subject:
|
Re: Why is Archived content purged retroactively? [Was: Re: Ezboard content suddenly not available in the new system - why?] |
Jeff you wrote the following:
"media is not being deleted. it just becomes unavailable for viewing in the Wayback Machine."
there was nothing to "misunderstand" about that.
Changing your post to read:
"My understanding is that when a robots.txt is in place all pages from that site are blocked." is pointing out the obvious and answers no ones questions.
We already know that sites are "blocked" when a robot.txt is respected, the questions is, is the content deleted from the Internet Archive, or simply made unavailable for public viewing?
