Scanning web site subfolder
Can I search any website subfolder with C#?
For example
search -> www.domain.com/
find ->
www.domain.com/numbers/
www.domain.com/magazine/
if I can, I wanna try to find page
www.domain.com/index.php
www.domain.com/page.htm etc.
Some program such as Internet Download Manager can find all pages under folder
I wanna learn How can they do this?
Can u help me any idea , way or code?