blocking robots

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts
  • mattyfergus
    Second Lieutenant

    • Sep 2008
    • 121

    blocking robots

    How do i block robots from crawling certain pages on my site?

    Thanks in advance
    Simple Video Production Liverpool - Website video's, corporate video's, event filming, music video's, viral's, motion graphics & 3D animation
  • Collectors-info
    General

    • Feb 2006
    • 8703

    #2
    Re: blocking robots

    Hi, depends on what you want the robot to do. IE: Just not index the page, but still follow the links on the page? Or not index anything.
    Place one of these in the head of each of the pages to not index.



    <META name="robots" content="noindex">

    0r

    <META name="robots" content="noindex,nofollow">

    Regards Chris.

    Collectables, Collecting, collectors-info.com

    www.chrismorris.co.uk

    House build project

    Comment

    • mattyfergus
      Second Lieutenant

      • Sep 2008
      • 121

      #3
      Re: blocking robots

      Thanks for the reply.

      When you say the HEAD OF THE PAGE, where exactly do you mean?
      Simple Video Production Liverpool - Website video's, corporate video's, event filming, music video's, viral's, motion graphics & 3D animation

      Comment

      • mattyfergus
        Second Lieutenant

        • Sep 2008
        • 121

        #4
        Re: blocking robots

        Also, I dont want the robots to see the text as it is duplicate text and google says this will degrade the site. So what code should I use?

        Thanks again.
        Simple Video Production Liverpool - Website video's, corporate video's, event filming, music video's, viral's, motion graphics & 3D animation

        Comment

        • mattyfergus
          Second Lieutenant

          • Sep 2008
          • 121

          #5
          Re: blocking robots

          Anybody?
          Simple Video Production Liverpool - Website video's, corporate video's, event filming, music video's, viral's, motion graphics & 3D animation

          Comment

          • Collectors-info
            General

            • Feb 2006
            • 8703

            #6
            Re: blocking robots

            <META name="robots" content="noindex,nofollow">

            The code above should do it. place it in the head of all page,s via. Main menu>View>Page HTML>Between head tag, paste the code in, & press ok.
            Regards Chris.

            Collectables, Collecting, collectors-info.com

            www.chrismorris.co.uk

            House build project

            Comment

            • Karen Mac
              General

              • Apr 2006
              • 8332

              #7
              Re: blocking robots

              Rather you index those pages or not, or no follow, you can still be penalized for duplicate content if it exists. No index means the robot wont spider and add it to cache, doesnt mean they wont rank it.

              No Follow doesnt mean they wont INDEX.. they will, it just means you dont VOUCH for the content of the link..

              Karen

              VodaHost

              Your Website People!
              1-302-283-3777 North America / International
              02036089024 / United Kingdom
              291916438 / Australia

              ------------------------

              Top 3 Best Sellers

              Web Hosting - Unlimited disk space & bandwidth.

              Reseller Hosting - Start your own web hosting business.

              Search Engine & Directory Submission - 300 directories + (Google,Yahoo,Bing)


              Comment

              • Collectors-info
                General

                • Feb 2006
                • 8703

                #8
                Re: blocking robots

                Originally posted by Karen Mac View Post
                Rather you index those pages or not, or no follow, you can still be penalized for duplicate content if it exists. No index means the robot wont spider and add it to cache, doesnt mean they wont rank it.

                Karen
                So would you say to stop the robots on the page that links are coming from to that page that is not to be indexed, or add a tag like this <meta name="robots" content="none"> so its not looked at by the robots??
                Regards Chris.

                Collectables, Collecting, collectors-info.com

                www.chrismorris.co.uk

                House build project

                Comment

                • Karen Mac
                  General

                  • Apr 2006
                  • 8332

                  #9
                  Re: blocking robots

                  To stop the robots yes... its NO INDEX.... or deny them access in htaccess file, however, duplicate content is duplicate content.. and we all know GOOGLE will look and find it.

                  If you want to remove a url in cache or index, or search results.. you can contact google to remove the url from their index... still doesnt mean they dont have a record of it

                  NO FOLLOW doesnt mean they wont index.. they will.. it means you dont vouch or vote for the content where the link leads


                  Karen

                  VodaHost

                  Your Website People!
                  1-302-283-3777 North America / International
                  02036089024 / United Kingdom
                  291916438 / Australia

                  ------------------------

                  Top 3 Best Sellers

                  Web Hosting - Unlimited disk space & bandwidth.

                  Reseller Hosting - Start your own web hosting business.

                  Search Engine & Directory Submission - 300 directories + (Google,Yahoo,Bing)


                  Comment

                  • Collectors-info
                    General

                    • Feb 2006
                    • 8703

                    #10
                    Re: blocking robots

                    Thanks Karen, good one to keep in mind.

                    Mattyfurgus, we have been told. Ya just cant hide from the little devils. LOL
                    Regards Chris.

                    Collectables, Collecting, collectors-info.com

                    www.chrismorris.co.uk

                    House build project

                    Comment

                    • mattyfergus
                      Second Lieutenant

                      • Sep 2008
                      • 121

                      #11
                      Re: blocking robots

                      Thanks people.
                      Simple Video Production Liverpool - Website video's, corporate video's, event filming, music video's, viral's, motion graphics & 3D animation

                      Comment

                      Working...
                      X