Robot.txt - Question

Collapse
X
Collapse
 
  • Page of 1
  • Filter
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Nite Producer
    Lieutenant Colonel

    • May 2006
    • 593
    #1

    Robot.txt - Question

    Hi People,

    I'm still reading through the SE sections etc.

    I just have a question about the file Robot.txt and why I always find in my awstat an IP address got an error trying to find the Robot.txt file?

    Thanks in advance.
    The Producer
    Tags: None
  • Bethers
    Major General & Forum Moderator

    • Feb 2006
    • 5224
    #2
    Re: Robot.txt - Question

    Because you don't have one and Google always checks for one. Not having it won't hurt you unless there is something you don't want the se's to spider. They look for it for any specific directions you might have for them - especially things like no follows.
    Beth
    A Child's Palace - Pinata Palace - Moxie Enterprises

    SEO and Marketing Tools
    SEO - The Basics

    Comment

    • Nite Producer
      Lieutenant Colonel

      • May 2006
      • 593
      #3
      Re: Robot.txt - Question

      Thank you bethers, I knew it will be you who will answer.

      Thank you again.
      The Producer

      Comment

      • ragamuffin
        Corporal

        • Jan 2007
        • 13
        #4
        Re: Robot.txt - Question

        Hello. When you go into the page html, and enter the meta tags for the robots, does it matter what position it enters the lines? Just wondering.

        And did I read somewhere that the Character lines in the tags should be below the other tags?

        I may have this wrong, but mine appear at the top of my meta tags lists. Yes, using BV website bldr.
        Thanks
        ragamuffin
        http://www.learnaboutwebhosting.com

        Comment

        • Bethers
          Major General & Forum Moderator

          • Feb 2006
          • 5224
          #5
          Re: Robot.txt - Question

          I don't follow the question - what character lines?
          Beth
          A Child's Palace - Pinata Palace - Moxie Enterprises

          SEO and Marketing Tools
          SEO - The Basics

          Comment

          • JoeHollinger
            Corporal

            • Sep 2006
            • 12
            #6
            Re: Robot.txt and meta tags - Question

            I have a question concerning Robot.txt and meta tags. Where and how do I go to edit page html in the <head></head> section and the <body></body> section of a web page?

            Is there a tool that I can use to check my robot.txt and meta tags to be sure I haven't made errors and/or ommissions in typing, format, etc.?

            Thanks,
            Joe Hollinger
            Webland Security

            Comment

            • Karen Mac
              General

              • Apr 2006
              • 8332
              #7
              Re: Robot.txt - Question

              Joe..

              Try sitereportcard.com and have it check your site for meta tags. Go to google webmaster tools and if you dont have an account open one and it will tell you rather or not its reading your robot.txt file

              Karen

              VodaHost
              Your Website People!
              1-302-283-3777 North America / International
              02036089024 / United Kingdom
              291916438 / Australia

              ------------------------

              Top 3 Best Sellers

              Web Hosting - Unlimited disk space & bandwidth.

              Reseller Hosting - Start your own web hosting business.

              Search Engine & Directory Submission - 300 directories + (Google,Yahoo,Bing)


              Comment

              • JoeHollinger
                Corporal

                • Sep 2006
                • 12
                #8
                Re: Robot.txt - Question

                Karen Mac Thank you for your help. I went to Google Webmaster Tools and got a report. Just what I wanted to see. Now I have a sort of road map that shows me where some additions and corrections are needed in my robots META tags. Speaking of META tags, are there tools that will check the META tags on a page and provide a report of what was found? Thanks again! Joe Hollinger Webland Security

                Comment

                Previous template Next

                Find your perfect domain name

                Working...
                X