J!Extensions Store™
Forum
Welcome, Guest
Please Login to access forum.
Re:Clear out old sitemap-generate a brand new one (1 viewing) 
Go to bottom
TOPIC: Re:Clear out old sitemap-generate a brand new one
#646
CW WebworX
Junior Boarder
Posts: 29
User Offline
Clear out old sitemap-generate a brand new one Karma: 1  
How do I get this extension to recreate a brand new sitemap? Right now with caching turned on it only updates the existing sitemap with new items or articles added.

What has happened is we have changed a lot of category names and have moved articles to the new renamed categories. We are getting a ton of 404s now from Google. So what we need to do is generate a brand new sitemap and somehow get Google to erase all indexed sitemap data and add all the new data from a new sitemap.

1. How do we create a new complete sitemap with over 6000 articles? Everytime I try to export the sitemap it only exports newest added articles not a complete sitemap.

2. How do we get Google to clear old sitemap data and add the new one?

Will this extension do this? We bought the Pro version and love the extension but are confused on how to do this now. We have a Google webmasters account so we can submit sitemaps or remove old ones.

Any help would be appreciated.
Thanks!
 
Logged Logged  
  The administrator has disabled public write access.
#647
John Dagelmore
Admin
Posts: 3716
User Online Now
Re:Clear out old sitemap-generate a brand new one Karma: 79  
Hello, if you changed urls on your site, the main issue is not the sitemap.

A sitemap is a mirror of links on your site and its purpouse is reflect exactly all this links that are valid, i mean generate HTTP 200.

If you have an old cached sitemap you can clear the Joomla cache to force a new generation, but the concept of brand new sitemap doesn't exist, because the sitemap is always new being generated in realtime if not cached, or everytime the cache is stale.

if your sitemap doesn't include all 6000 articles, ensure that articles are all published and not expired, and also that in the data source of type 'content' there are no restrictions such as limit to most recent articles.

You don't need to clear sitemap and resubmit in your GWT, the link to sitemap never changes and Google will fetch updated sitemaps every 2/3 days.

To remove old links, ensure that they generate all 404s, in this way Google will remove automatically from the index.

Cheers!

John
 
Logged Logged  
  The administrator has disabled public write access.
#650
CW WebworX
Junior Boarder
Posts: 29
User Offline
Re:Clear out old sitemap-generate a brand new one Karma: 1  
Thanks! I made sure all items in data source 'content' is selected. I was able to generate a new complete sitemap by turning off the cache as well.

I was under the impression from the documentation that once I exported the sitemap.xml file I should then copy it to the public_html folder on the host. Which I did. I then added this sitemap.xml to GWT to index. Was that correct?

Now the 500 error is back. When I try to view the html sitemap the server times out. In order for me to stop this I will have to modify data source 'content; and change it to only add articles from the last month and then the 500 errors stop. Is this normal and is that what I need to do?

Thanks for the help and quick support. Cheers!

John Dagelmore wrote:
QUOTE:
Hello, if you changed urls on your site, the main issue is not the sitemap.

A sitemap is a mirror of links on your site and its purpose is reflect exactly all this links that are valid, i mean generate HTTP 200.

If you have an old cached sitemap you can clear the Joomla cache to force a new generation, but the concept of brand new sitemap doesn't exist, because the sitemap is always new being generated in realtime if not cached, or everytime the cache is stale.

if your sitemap doesn't include all 6000 articles, ensure that articles are all published and not expired, and also that in the data source of type 'content' there are no restrictions such as limit to most recent articles.

You don't need to clear sitemap and resubmit in your GWT, the link to sitemap never changes and Google will fetch updated sitemaps every 2/3 days.

To remove old links, ensure that they generate all 404s, in this way Google will remove automatically from the index.

Cheers!

John
 
Logged Logged  
  The administrator has disabled public write access.
#651
John Dagelmore
Admin
Posts: 3716
User Online Now
Re:Clear out old sitemap-generate a brand new one Karma: 79  
Handling a whole sitemap containing 6000 links is that it could require too much time to complete generation and the server could timeout.

For XML you could use AJAX precaching with AJAX generation to generate sitemap by steps.

About the HTML sitemap, always avoid to include thousands of links, because also if the 500 error would not happen, too much data would hang the browser.

In your case i would suggest a strategy.

1- Generate a full XML sitemap, export as sitemap.xml then after positioning in public_html submit to GWT
2- Limit the articles to most recent and submit the link for sitemap dynamic generation to GWT

This allows you to have an old big static part of sitemap and another part managed in realtime quickly.

Cheers!

John
 
Logged Logged  
  The administrator has disabled public write access.
#652
CW WebworX
Junior Boarder
Posts: 29
User Offline
Re:Clear out old sitemap-generate a brand new one Karma: 1  
Ok I did as you suggested. The only problem is because articles have been re-indexed under new category names, even limiting the content source to items created in the last 1 month, it still hangs and times out. It treats all the re-indexed articles as new. So in viewing the html sitemap the system does hang as you suggested it might. Any further suggestions?

Cheers!

John Dagelmore wrote:
QUOTE:
Handling a whole sitemap containing 6000 links is that it could require too much time to complete generation and the server could timeout.

For XML you could use AJAX precaching with AJAX generation to generate sitemap by steps.

About the HTML sitemap, always avoid to include thousands of links, because also if the 500 error would not happen, too much data would hang the browser.

In your case i would suggest a strategy.

1- Generate a full XML sitemap, export as sitemap.xml then after positioning in public_html submit to GWT
2- Limit the articles to most recent and submit the link for sitemap dynamic generation to GWT

This allows you to have an old big static part of sitemap and another part managed in realtime quickly.

Cheers!

John
 
Logged Logged  
  The administrator has disabled public write access.
#653
John Dagelmore
Admin
Posts: 3716
User Online Now
Re:Clear out old sitemap-generate a brand new one Karma: 79  
This probably happens because all the creation date of recreated articles has been updated
I think the trick would be set the original date for those articles.
Otherwise you need to exclude the content data source from the HTML sitemap leaving it enabled only for XML and indexing.

Cheers

John
 
Logged Logged  
  The administrator has disabled public write access.
Go to top