Article Search
Rajat Kumar (Tech Head)
August 01, 2017
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned. Robots are often used by search engines to categorize web sites.
Top Authors
-
Rajat Kumar
29 Posts | 28 Points
Popular Posts
- Difference between serialize() & FormData() methods in jQuery. (18)
- Difference between SMTP, IMAP and POP3 in mail & data transmission. (10)
- Basics of Cyber security to protect information & data from hackers (Session-1). (4)
- Concepts & techniques to find vulnerability and prevent the cyber-attacks (Session-2). (3)
- Complete list of PHP Supported Timezones for date and time functions. (2)
Top Topics
Popular Tags
google stadia beta organizatin dataInfiltrationgoogle stadia release date in india POP3 driver issueweb server relational modelDoS search engine crawlersgoogle stadia controller SEO cPanelweb designinggoogle ads (adwords) off-page seo techniquesgoogle stadia beta tester meta tags Rich structured data Database administrationgoogle ads steps for web designing open graph protocol open graph tagsgoogle stadia base