Today we will learn how to make your website easy to crawl and indexed by the search engine. In crawling your website, SEO is very important with Content creation, Social media and Link builder. In SEO word, crawling means following your links and crawling your Website. If the search engines cannot crawl your website efficiently, you’re unlikely to rank. Even links and social shares won’t solve accessibility issues of your website.
An accessible website means that all target pages will be indexed and have the opportunity to rank for your target keywords. The conclusions for SEOs working at any level used to these essential facts:
- It’s all about the crawl
- It’s all about the index
- It’s all about the Rank
Crawling
Crawling means the visit of Google to track your website target page. This process is done by Spider of Google crawler. Our best practise should be to stop the web page those we didn’t want to crawl.
Controlling the crawl :
“Nofollow” provides a way for webmasters to tell search engines “Don’t follow links on this page” or “Don’t follow this specific link.” Usually we stop crawling of a certain part of our site is by using Robots.txt file.
- Add the rel=”nofollow” tag to links to page you do not want crawled.
- Use <a href=”signin.php” rel=”nofollow”>sign in</a> to stop crawling links.
As well as rel=”nofollow” and robots.txt, you can use META tags to control how Google crawls your website. These are placed in the <head> section of your page and can do a number of things including:
- Tell Google not to index the page.
- Tell Google now to crawl any links on the page.
- Tell Google not to index images on the page.
- Tell Google not to use a snippet from the page in their search results.
Keep a Simple URL Structure :
A site’s URL structure should be as simple as possible. For example, if you’re searching for information about Opencart Marketplace, a URL like https://store.webkul.com/OpenCart-MarketPlace.html will help you decide whether to click that link. A URL like http://www.example.com/index.php?id_sezione=360&sid=3a5ebc944f41daa6f849f730f1, is much less appealing to users.
Consider using punctuation in your URLs. The URL http://oc.webkul.com/Booking-Reservation/ is much more useful to us than http://oc.webkul.com/BookingReservation/. Google recommend that you use hyphens (-) instead of underscores (_) in your URLs.
Indicate paginated content :
Sites paginate content in various ways. For example:
- News and/or publishing sites often divide a long article into several shorter pages.
- Retail sites may divide the list of items in a large product category into multiple pages.
- Discussion forums often break threads into sequential URLs.
If you paginate content on your site, and you want that content to appear in search results, use rel="next" and rel="prev" attributes to links.
In the <head> section of the first page (http://www.example.com/article-part1.html), add a link tag pointing to the next page in the sequence, like this:
<link rel=”next” href=”http://www.example.com/article-part2.html”>
Indexing
Indexing is the process of adding Webpages into Google search. Depending upon what meta tag you used for your page (Index or NO-index), Google crawl and index your pages. No-index means that page will not be added in Web search. By default any WordPress post or pages are indexable.
A good idea for better search engine ranking is to let only vital part of your blog/Website be in index and do not index un-necessary archive like tags, categories and all such pages.