Best way to generate a sitemap for a large site? - GoFuckYourself.com

Mikeh · 04-22-2011, 11:17 PM

I have 15,000+ pages on my site what is the best way to generate a sitemap for google? I have downloaded a few freebie sitemap generators but they only do 5000 pages...I was hoping for a free program anyone know of any?

robber · 04-22-2011, 11:32 PM

do they run from a database or are they all static?

Mikeh · 04-22-2011, 11:36 PM

well they run from a database, but they have been assigned static .html urls

Mikeh · 04-22-2011, 11:41 PM

I am using prestashop if that helps? which comes with a sitemap module but I am on a shared account with hostgator which only allows so much time for any given process to finish otherwise it kills it.

barcodes · 04-23-2011, 12:03 AM

http://code.google.com/p/sitemap-gen...emapGenerators

hope this helps

robber · 04-23-2011, 12:14 AM

if it needs to keep up then you may have to rerun it every now and then, but you could script a direct feed from your back-end and build the file that way, if the site i small(ish) then you could have it update automatically, providing it doesn't time your server out

I would be happy to help you put one together like: http://foxydrop.com/sitemap.xml, this automatically pulls the info from the db, as it is mainly used by spiders I have not yet made it a standalone file that generates itself. You can extend your sitemap to include as much information as you want, but it would be the time to construct it that I think you're having a few issues with if I'm not mistaken.

Rob

Mikeh · 04-23-2011, 12:16 AM

thanks that was what i was looking for =)

Mikeh · 04-23-2011, 12:19 AM

Quote:

Originally Posted by robber

if it needs to keep up then you may have to rerun it every now and then, but you could script a direct feed from your back-end and build the file that way, if the site i small(ish) then you could have it update automatically, providing it doesn't time your server out

I would be happy to help you put one together like: , this automatically pulls the info from the db, as it is mainly used by spiders I have not yet made it a standalone file that generates itself. You can extend your sitemap to include as much information as you want, but it would be the time to construct it that I think you're having a few issues with if I'm not mistaken.

Rob

I am running a crawler right now, i'll see how it goes...

robber · 04-23-2011, 05:04 AM

Quote:

Originally Posted by Mikeh

I am running a crawler right now, i'll see how it goes...

how did the crawler go?

enginecash2 · 04-24-2011, 04:30 AM

Have a look at xmlsitemaps.org

Adraco · 05-07-2011, 01:33 PM

I use this one:
http://www.xml-sitemaps.com/

Kafka · 05-08-2011, 05:09 PM

Don't use a sitemap. Google will find your pages.

04-22-2011, 11:17 PM	#1
Mikeh Registered User Industry Role: Join Date: Feb 2011 Location: Flo Rida! Posts: 21	Best way to generate a sitemap for a large site? I have 15,000+ pages on my site what is the best way to generate a sitemap for google? I have downloaded a few freebie sitemap generators but they only do 5000 pages...I was hoping for a free program anyone know of any?

05-07-2011, 01:33 PM	#11
Adraco Confirmed User Industry Role: Join Date: May 2009 Location: Onboard an airplane around the globe Posts: 3,737	I use this one: http://www.xml-sitemaps.com/ __________________ ---------------------------------------------------------------------------------- The truth is not affected by the beliefs, or doubts, of the majority.

04-22-2011, 11:32 PM	#2
robber Web Developer Industry Role: Join Date: Jan 2011 Location: UK Posts: 264	do they run from a database or are they all static?

04-22-2011, 11:36 PM	#3
Mikeh Registered User Industry Role: Join Date: Feb 2011 Location: Flo Rida! Posts: 21	well they run from a database, but they have been assigned static .html urls

04-22-2011, 11:41 PM	#4
Mikeh Registered User Industry Role: Join Date: Feb 2011 Location: Flo Rida! Posts: 21	I am using prestashop if that helps? which comes with a sitemap module but I am on a shared account with hostgator which only allows so much time for any given process to finish otherwise it kills it.

04-23-2011, 12:03 AM	#5
barcodes Confirmed User Industry Role: Join Date: Mar 2011 Location: Florida Posts: 2,040	http://code.google.com/p/sitemap-gen...emapGenerators hope this helps

04-23-2011, 12:14 AM	#6
robber Web Developer Industry Role: Join Date: Jan 2011 Location: UK Posts: 264	if it needs to keep up then you may have to rerun it every now and then, but you could script a direct feed from your back-end and build the file that way, if the site i small(ish) then you could have it update automatically, providing it doesn't time your server out I would be happy to help you put one together like: http://foxydrop.com/sitemap.xml, this automatically pulls the info from the db, as it is mainly used by spiders I have not yet made it a standalone file that generates itself. You can extend your sitemap to include as much information as you want, but it would be the time to construct it that I think you're having a few issues with if I'm not mistaken. Rob

04-23-2011, 12:16 AM	#7
Mikeh Registered User Industry Role: Join Date: Feb 2011 Location: Flo Rida! Posts: 21	thanks that was what i was looking for =)

04-24-2011, 04:30 AM	#10
enginecash2 Confirmed User Industry Role: Join Date: Apr 2011 Posts: 256	Have a look at xmlsitemaps.org

05-08-2011, 05:09 PM	#12
Kafka Confirmed User Join Date: Oct 2002 Location: Holland Posts: 466	Don't use a sitemap. Google will find your pages.