Clear it in 3 steps
-
Start the environment
-
Investigate the target
-
Submit the flag
Websites often serve a robots.txt file for crawlers. In this challenge, check whether a file that robots.txt tells crawlers not to index can still be accessed directly.
Start by visiting /robots.txt
Check which path is listed under Disallow
Try accessing that path directly in your browser