Easy SEO Introduction (2)-Crawling & Indexing

 Search engines have two main functions: Crawling and Indexing. In order to answer users’ questions, by crawling billions of documents, web pages, files, messages, videos, media, etc. on the WWW network, and providing the correct index to provide search users with the most relevant replies and website ranking lists

What are Crawling and Indexing used for?

Imagine the virtual network is a subway system in a big city. The way the search engine moves is Links. The link structure of the web page is used to tie all web pages together (metro network). Each site is a page (usually a web page, sometimes PDF, JPG or other) file). Search engines use a special automated robot, referred to as "Crawlers" or "Spiders" to "crawl" the entire city, find all the sites, and retrieve billions of interconnected files on the Internet.

After the search engines find these web pages, they decipher the code of the web pages and store the relevant information in a huge database. When someone searches for related words in the information in the future, the search engine can search the huge database within a few hundredths of a second and immediately provide the most relevant information. In order to accomplish such a difficult task, search engine companies have filled data centers all over the world. These huge storage facilities can hold thousands of computers to quickly process large amounts of information. When a person searches on any major search engine, they can get immediate results-any one-second delay is not allowed.

 

A search engine is a machine that provides answers

A search engine is a machine that provides answers. When a person types a word in the search bar, the search engine organizes billions of documents and vocabulary, and starts to do two things:

1. Return to previous user related searches and find effective query results
2. Rank the results of the search based on the popularity of the website providing relevant information

From the above two actions, we can know that search engines will evaluate search rankings based on the relevance and popularity of information , and SEO actions are affecting the final results of search engines according to each search.

 

How do search engines judge the relevance and popularity of information?

For search engines, relevance refers to finding the correct words within a page. In the early days of the Internet, there was no better search method than finding the correct words as search results, so the accuracy and value of search results were limited. However, after years of continuous development and improvement by engineers, search engines now have smarter ways to provide improved matching results for searches. Today, there are hundreds of factors that allow search engines to determine whether the data can be provided as correct search results. In this volume, we will discuss some of the most important ones.

First, search engines usually believe that a popular website, web page, or document is bound to provide more useful information for retrieval. This hypothesis has been successfully proved in terms of user search satisfaction, the more popular the website, the more effective the information provided to the user (and user feedback).

However, website popularity and relevance cannot be adjusted manually. The above will be calculated by the engine using complex mathematical equations and then sorted by relevance and popularity. The equation usually contains hundreds of variables. In the field of SEM (Search Engine Marketing), we call these variables "ranking factors".

The complex algorithms of search engines seem indestructible. In fact, they do not provide complete knowledge of how to obtain better rankings or traffic. According to different calculation methods, there are still some common principles and general directions to be based on. Please see the website optimization guide provided by Google and Bing webmasters.

 

Example-Website optimization guide provided by Google webmaster:

• Do not build a website for search engines, understand the needs of users, and create web pages for users.
• Do not deceive users, do not provide users with two different information from search engines on the website. This practice is often referred to as "cloaking", which has been recognized by search engines as a black hat technique for deducting points.
• The website needs to have a clear structure and text links
• Every page should have at least one static link
• Create a useful, informative website and provide pages that accurately describe the content
• Ensure that the description of the <title> element and ALT attribute in the website is accurate and correct
• Use keywords to create highly descriptive and understandable URLs
• Provide a URL for the file and use 301 redirects or rel="canonical" to resolve duplicate content

 

Example-Website optimization guide provided by Bing (Microsoft MSN) webmaster:

• Provide a clean, keyword-rich URL structure
• Ensure that the content of the website is not buried by RichMedia media software (such as Adobe Flash Player, JavaScript, Ajax). If you use it, please ensure that the construction of the above software will not affect the crawler robot's search links.
• Create content with rich keywords and match the content with keywords frequently searched by users
• Website needs to generate new content from time to time
• Don't put the text you want to index in the picture. For example, if you want your company name or address to be indexed, make sure that the above information is not included in the company logo image, but a separate text.

The suggestions of the above two search engines are just a drop in the bucket among many search engines! In addition to the above optimization guidelines provided by major search engines, SEO/SEM experts have summarized some data on how to effectively improve rankings in search engines and help websites in the history of online search for nearly 15 years. Talk to customers to achieve better website positioning.

These efforts, search engines have adopted in the process of evolution. Although they do not publicly support these practices. However, in various search engine marketing conferences, search engine representatives will participate and participate in assisting roles in related blogs, forums, and online groups.