user_
The user-agent, which is the name of the crawler, for a specific group in the robots.txt file.
The object consists of a User-agent directive, and a value of the name of the user-agent. For example:
User-agent: *
Tip
You can customize the robots.txt file with the robots.txt.liquid template.
{
"directive": "User-agent",
"value": "*"
}
{
"directive": "User-agent",
"value": "*"
}Was this section helpful?