robots.txt

[vinny1957]

Now that my site is up, I'm looking to promote it and I ran up on robots.txt. I've looked at it and it helps or directs search engine spiders when they crawl your site. Now, my question is, does this have to be put in manually, or does BV put it in automaticly? If I do, next step is how and what to include?
Thanks all.
Vinny

[Steven.H]

One of the best ways to promote your site is to be sure you have good metatags. After you have those established, there are also many programs, such as exitexchange, which help you get more traffic.

I am not too familiar with robots.txt, but I have read somewhere that it can actually prevent robots from entering your website. I will have to look further into it.

I will get back to you about it.

[vinny1957]

Thanks a lot Steven, I happened accross this and was confused.

[Stroke.nz]

Hi Vinny

Here is an Interesting site I was looking at http://www.websight.net.au/search-en...r-friendly.asp

[vinny1957]

Thanks a lot it is a good site for reference.

[pet121]

robot text

I am learning every day about page rank ect... I will share with something with this forum I may be right or I may be wrong. I belong to many forums. this one forum and its members were pretty angry at this one site that was basically giving them a hard time. Anyway within a week there comments were posted on google and yahoo in the number two position right under the company they were angry with. The company ranked number 1 in there catagory the forums comments number 2. How did they get such a fast page rank? I think it was because of the robot txt inserted into their source code. In fact about 80% of pages that rank high seem to have robot txt. take a look. This is something i am still investingating. pet121

[vinny1957]

Thanks a lot, I'll take a look. The more I look into this the more questions a have. But there's something to it.
Vinny

[Stroke.nz]

Add this to your meta

<meta name="ROBOTS" content="INDEX,FOLLOW">

[vinny1957]

Thanks Dave, I'll do it today.

[Maddog]

Add this to your meta

<meta name="ROBOTS" content="INDEX,FOLLOW">

vinny make sure you place the above meta tag in each page

[vinny1957]

vinny make sure you place the above meta tag in each page

Thanks Maddog, wiil do!

[vinny1957]

Just thought of something - duh! I don't have any meta tags set up for my link page. Should I? It just links from my home page.

[rdarko]

Another way that worked for me is to put as many links to popular sites on your website and do lots of link exchanges with popular sites. As with Google...it sometimes comes across a new website via websites already in its indexes.

Meta tags are good.

Robots.txt prevent google adsense and other things from working properly.

Just make sure you've submitted your site to as many search engines and that you've used Keyword Digger in BV to create your meta tags and you should be fine in a couple of weeks or so.

Good luck!

[vinny1957]

Another way that worked for me is to put as many links to popular sites on your website and do lots of link exchanges with popular sites. As with Google...it sometimes comes across a new website via websites already in its indexes.

Meta tags are good.

Robots.txt prevent google adsense and other things from working properly.

Just make sure you've submitted your site to as many search engines and that you've used Keyword Digger in BV to create your meta tags and you should be fine in a couple of weeks or so.

Good luck!

Thanks all!

[pet121]

my advice

Robot text is useful and has no bearing on googlebots. Googlebots search for content and are very slow. As for keywords use the keywords from your content page. Because your keyword must be in your content. If it is not you are penalized for it. Most important be honest with your website content and be sure it is relevant to your tiltle. I am no expert pet121