Robot problem solving and reflection
The source code html text has no actual content
View the topic description
robots agreement
View related knowledge
Directly request robots.txt to see what happens
Get the flag prompt
and request again to
get the flag.
Reflection
This question only applies the knowledge of url request, and the corresponding robots protocol is only mentioned,
but the purpose is to let us know about the robots protocol
robots note link: link .