|
Question:
All of a sudden, Google quits crawling my website! Help
|
Report abuse
|
at www.eritreadaily.net, we are also facing the same problem, We have been long time participants in Google News but all of a sudden, since beginnnig of July 2009, our articles are not picked up by Google News Crawlers. Our site meets all the technical requirements. Please help, thanks.
Best regards,
Brin
Answers:
1 of 49
All answers
Ahh, sorry. I missed this thread somehow.
Do you have a news sitemap? If not, it would be cool if you could submit one in webmaster tools. If you can't for some reason create one, let me know and i create a temporary one for you which you can submit.
Also, please try to mess a bit with crawl rate in webmaster tools. It might be that the Crawler doesn't visits your site as often as before. You could set the crawl rate a bit higher.
If these things won't help, you will have to wait for a Googler to look into your issue.
Have a nice day
Dear methode:
I appreciate your feedback greatly, thanks bud!
Yes, I do have a news sitemap that I submitted since November 2008 is and has been working perfect and even downloaded by Google yesterday. Will mess with the crawl rate again, ty.
You have a nice day too, my friend.
OK, if you have a news sitemap, then please check on the detailed sitemap page (sitemaps>> click [sitemap_name]) whether the Type and the Format are both listed as "News"; this is very important, so pay attention to it. Please report back what you see.
Hi methode, yes, both Type and Format listed as "News". Thanks
Hi methode:
I miss your feedback, bud! checkout my site: www.eritreadaily.net I would appreciate it if you would get some feedback from Google Folks in this matter. ty.
Nice day
Did you change anything in your site's layout, address, URL of articles or anything in the HTML code?
dear methode, never, never bro. Still hope you would be able to contact Googlee folks, please. Thanks.
Greatful for your prompt response.
Nice Day.
They don't work on weekends and wouldn't ruin their weekend by email but will point them to this thread Monday morning (EDT).
methode, you are great! Appreciate your efforts in this matter greatly. God bless you.
Have a nice weekend, my friend
OK, a few more things I would like from you to check out. In webmaster tools>>diagnostics, is anything interesting? Can you tell me where's your sitemap located? In webmaster tools, is any error reported on the detailed sitemap page?
I wouldn't really want to bother the Google guys if it's something too simple and we can also fix.
mithode, diagnostic is unremarkable: no error reporting, sitemap downloaded by Google 10 July 2009, status OK. I checked and rechecke everything, every thing is AOK. Even my Google Analytic is functioning perfect. sitemap is located in the index directory: www.eritreadaily.net/. My friend, am not IT savvy as you are but I know I met all the technical requirements of Google. I hope that the location of the sitemap is right, ty. Some thing is blocking all of a sudden the crawling and picking up of my news items after long years of functioning. I am stupified!
Great thanks
Hi methode, here the location of my sitemap: www.eritreadaily.net/public_html/sitemap.xml
That is not a news sitemap but a web sitemap. Again, do you have a News sitemap[1]?
That is all I have been using and worked sofar. I would appreciate it if you would help with a News sitemap please. ty
k,
Download it and save it on your webserver. Then submit it in webmaster tools just like you'd submit a normal sitemap, but pay particular attention to submit it as "News". This can be selected after you pressed the "Add sitemap" button from a drop-down menu.
If this solves your issue momentarily, you will have to code a sitemap generator which is capable of generating a news sitemap. Another publisher already hired me to create one, but that's only capable to create a news sitemap from an RSS feed, since you have no RSS feed, you will have to find another way.
By the way. You can download the above sitemap by right clicking on it and then clicking on the "Save target as", "Save link as" or similar menu item.
methode, great thanks. let me try it. will report back asap. ty
Hi methode, I already have a website map. Now, in order to add a News site map, do I have to delete the website or is it possible to add the News site map separately. Please help. ty
methode, I mean do I have to give up (delete) my website sitemap inorder to submit the News sitemap? Thanks
No, you should have the option of submitting a news sitemap. See the help entry I linked to below
methode, won't allow a second site map cause it prompts me to enter a website url but won't accept it cause it is already indexed, Hope you get in touch with Googlefolks about this issue tomorrow and await anxiously for your valuable feedback, ty
Nice evening
Hi methode: I was able to submit the News sitemap you created successfully this morning and Google status is OK. But the news items were not picked by Googlebot or may be the site not crawled, Thanks for creating the News citemap but I would appreciate it if you would get Google folks to look into this issue. Great thanks.
Nice day
methode, you checkout my sitemaps here: https://www.google.com/webmasters/tools/sitemap-list?hl=en&siteUrl=http%3A%2F%2Fwww.eritreadaily.net%2F
methode, you there! miss your feedback, bro! Don't let me down, need your professional and valuable feedbacks now than ever, ty.
Nice day
https://www.google.com/webmasters/tools/sitemap-list?hl=en&siteUrl=http%3A%2F%2Fwww.eritreadaily.net%2F
Am sorry, but I don't work for Google but for another corporation and sometimes I have to do that job as well.
Also, I can't check out your news sitemap on that location because it's protected by your login.
One more thing and then I report the thread to the Google engineers. Please follow the URL you posted above then click on the sitemap's filename you have recently added and which should be a news sitemap. A table will show up which several details of the news sitemap. Which is interesting for us is the "Type" and "Format" data. Please tell me these two values.
methode, the one site has the format: Web, the other which you created and is accepted by Goog has the forma: News
OK, and type? Click on the News sitemap's name then see what it says for "Type"
Type too: website and News bro, ty
K, I report the site now. Please doublecheck the news sitemap's format and type.
We will still have to figure out how to create for you an always up-to-date news sitemap.
methode, thanks bro. That would be great with the always up-to-date sitemap, God help you, buddy. Will check with you over the day. ty again for your help
Nice day
Hi methode, sorry to bother you again. Some one said this is the problem; <META HTTP-EQUIV="EXPIRES" CONTENT="-1"><META HTTP-EQUIV="CACHE-CONTROL" CONTENT="NO-CACHE"> Would please help figure that out? How do I fix that. Thanks
As far as I know that causes no issue. Just think: it didn't cause issue till now, why would now?
methode, my friend, that is exactly how I responded too! Yes, that didn't cause issue for years, why now? Any way, were able to get response from Google folks yet? Thanks for hangign on. Hope to hear from you again. ty
Have a nice day
No, I let them know and much likely they are working on a solution or its on the to-do list Please, do not expect a 2-minute-fix. They have loads of work.
methode, apology, don't mean to hustle them at all. Just curious if they at respond! You know, the site is crawled but the new items are not posted on Google News page, that is my worry! Once again great thanks and you have a wonderful day.
|
Inbal
Google Employee
7/15/09
|
methode
(Top Contributor)
+
2
other people
say this answers the question:
Hi edaily,
Apologies for the confusion regarding your news content stopped being indexed in Google News. The reason why we stopped crawling your content since July is that we periodically review news sources, particularly following user
complaints, to ensure Google News offers a high quality experience for
our users. When we reviewed your site we found that we can no longer
include it in Google News at this time.
We currently only include articles from sources that could be considered organizations, generally characterized by
multiple writers and editors, availability of organizational
information, and accessible contact information. When we reviewed your
site we weren't able to find this evidence of an organization.
If you have other information about your site that we should consider, please contact us via the link below.
Best Regards, Inbal
Eritrea Daily is a news website, we post news articles reported by various News Agencies like, AFP, AP, BBC, CNN, other websites like shabait.com, zhinua and international organizations like UN, AU, EU and more. Moreover we post our own op-ed. Basically, our website is a New trove for news about Eritrea in particular and horn of Africa in general and world affairs. Our contac address webmaster@eritreadaily.net, or mfilli@eritreadaily.net telofon: 732-397-8849. Let me know if you need more info ty. We are hurt by the action you took. Hopefully you give the essence of our website due consideration and reverse your decision because it also hurts revenues over GoogleAdsense that we have been participating in for so long. Hope to hear from you soon in this matter, ty
PS-there are so many contat possibilities, I would appreciate it if you would direct me the right issue that pertains to my issue, ty
inbal, do you include articles from News troves? Eritrea Daily is a news trove: It posts news articles reported by various new agencies, organizations and others, ty
Regards,
edaily
Dear inbal, I would have hoped Google would warn me about the danger of not being included in its Google News and necessary remedy would have be taken before the action taken so abruptly. Moreover, wish I knew if News Trove such as my site would met the requirement of organization. ty hope to hear from you how long the exclusion would last, ty.
Best regards,
Brin for edaily
|
Inbal
Google Employee
7/15/09
|
1
person
says this answers the question:
Dear Brin,
Please be assured that we tried to contact Eritrea Daily via email before removing it from Google News to offer the necessary remedy, however, alas, our emails bounced back, and due to some user complaints about no evidence for an organization behind the source we took this action, as we weren't able to find multiple writers and editors and accessible contact information on your site. However, as you've provided us with this additional information about your site, we'll be re-adding it into our index for Google News.
Best regards, Inbal
Additionally, You will still have to provide some form of contact information, list of editors and organizational info on your website. Just think with your visitor's mind: they want to know more about the news source they read articles from, providing the missing information makes your site to be a more credible news source.
Have a nice day
Hi methode, finally good news. Just above your latest entry, Inbla posted an answer where he accepted the additional info about my site and decided "we'll be re-adding it into our index for Google News." I appreciate all your efforts in this matter and hopefully you will find a way to turn the temporary News sitemap you created for me into permanent one.
Have a great day my friend
Brin
1
person
says this answers the question:
We had to see whether there's a problem which we can fix ourself or is something beyond us; that's why the thread grew this long. The Googlers have enough work already, no need to bother them when there's something we can also fix. Yes, I saw that post, I responded to add a bit more info. Note that if the information is not present on your website someone could report it again and not necessarily Inbal will process the report thus your site might get removed again. Please, do provide the info and form of working contact on your website
As for creating a permanent news sitemap, it's not necessarily a need. If the system you were using till now worked, it will in the future as well. The temporary sitemap was created to test whether the crawler has issues identifying individual articles, nothing more. When I have time, i will create a tool like the one you are using to generate the web sitemap, though.
methode, thanks again. There was already contact link, and Mission statement on our site. Any way I have now included the necessary infos on the site. Yea, you are right, never needed News sitemap for so long and I am about to remove the test sitemap. Appreciate all efforts and concerns, God bless you! Amen
Best regards,
Brin
Inbal great thanks for your decision to re-include my site in Google News. Appreciate it greatly. I know you got a ton of emails to respond to, but certainly hope that it won't be too long before our news items appear in Google News page again. ty
Best regards,
Brin
1
person
says this answers the question:
Your site will be processed again in a matter of few days, max a week.
Dear methode: Two thumbs up for your tireless efforts to help others and for boosting my moral. Great thanks and you have a great day.
Besst regards,
Brin
Your welcome.
If you have any more question, let me know; I'd be happy to answer them if I can or am allowed. Though, you should start a new thread for them as this has been already answered.
|
|
|