https://tools.kali.org/information-gathering/parsero Parsero is a free script written in Python which reads the Robots.txt file of a web server and looks at the Disallow entries. The Disallow entries tell the search engines what directories or files hosted on a web server mustn’t be indexed. For example, “Disallow: /portal/login” means that the content on www.example.com/portal/login it’s not allowed to be indexed by crawlers like Google, Bing, Yahoo… This is the way the administrator have to not share sensitive or private information with the search engines. Subscribe my channel https://www.youtube.com/channel/UC5hnY8XWWoPvnktQ651kr9w Official Trailer my Channel https://www.youtube.com/watch?v=qFJmVY-RnH4 Osint Kali Linux https://www.youtube.com/watch?v=NTF7WsHy4G4&list=PL0A5SH4w3NaIBKahXMaO29uToGLn3dARF My best Video Kali Linux https://www.youtube.com/watch?v=qFJmVY-RnH4&list=PL0A5SH4w3NaK9aIOSq5fpWsuHsq6pHkPX Tool Hacking Kali Linux https://www.youtube.com/watch?v=l9MkyuKW3_w&list=PL0A5SH4w3NaLTcClb6o_KU4zjqJM4Jt3d Powershell Windows10 https://www.youtube.com/watch?v=QcXbaXX6QHQ&list=PL0A5SH4w3NaKuiF7-CWHzxlaO-LL_s5rl ... https://www.youtube.com/watch?v=45LXUrBHp2w