Scanning web site subfolder

Question

Can I search any website subfolder with C#?
For example
search -> www.domain.com/

find ->
www.domain.com/numbers/
www.domain.com/magazine/

if I can, I wanna try to find page
www.domain.com/index.php
www.domain.com/page.htm etc.

Some program such as Internet Download Manager can find all pages under folder
I wanna learn How can they do this?
Can u help me any idea , way or code?

none · Answer

I dont know what Internet Download Manager can do... but i don`t think this is possible. U can create something like a http browser and can acces all file if the server it`s set to let you browse the files

Emre · Answer

I know this way but I want to learn If there isn't any adress or link in the page How can I search subfolder of website? For example Internet Download Manager find many folder some site which are underconstructor..

none · Answer

You can try to find all links from the first page and scan every page for every link...