WeChall - Training - WWW-Robots

Posted on In Views: Disqus:

Challenge

Understanding the robots.txt file.

Solution

访问目标网站的 /robots.txt，查看禁止爬取的路径，T0PS3CR3T 目录就是答案。

1
2
3
4
$ curl https://www.wechall.net/robots.txt
User-agent: *
Disallow: /challenge/training/www/robots/T0PS3CR3T
...

然后访问 https://www.wechall.net/challenge/training/www/robots/T0PS3CR3T/ 确认。

T0PS3CR3T