Tweet
I am using the BV sitemap tool and have a question.
I noticed that when I crawl, it will pick up a page or 2 that is not in the directory or uploaded to the directory (it may have been at one time but is now deleted) Does anyone know why this happens? Obviously I can remove it from sitemap creator, however when I have 600 pages it can be complex to know which is there is which really is not.
If it is not on the server and is not even on my pc, how is it the crawler pics it up?
webmasters of course tells me when I have a issue (some times), but is hard to detect the exact problem.
I also notice on kortia.com it crawls and picks up a kortia.com/index and then one with www.kortia.com.index, so it duplicates it. How can this happen? It does it also with one other page, one with www. and one with out www.
Sorry I know I ask complicated questions.
I noticed that when I crawl, it will pick up a page or 2 that is not in the directory or uploaded to the directory (it may have been at one time but is now deleted) Does anyone know why this happens? Obviously I can remove it from sitemap creator, however when I have 600 pages it can be complex to know which is there is which really is not.
If it is not on the server and is not even on my pc, how is it the crawler pics it up?
webmasters of course tells me when I have a issue (some times), but is hard to detect the exact problem.
I also notice on kortia.com it crawls and picks up a kortia.com/index and then one with www.kortia.com.index, so it duplicates it. How can this happen? It does it also with one other page, one with www. and one with out www.
Sorry I know I ask complicated questions.
Comment