Search files and folders from any website URL

Question

Hi I am trying to find the files from the URL path folders.that URL may be any website. for example Input: http://www.dreamincode.net/forums/topic/77912-ftp-download-delete/ First need to parse that URL and check it outis there any log files or document files are there within that folder. That URL is not belongs to local path it may third party site like www.google.com/ or something else. In our example that folder may forums  topic and 77912-ftp-download-delete *.log or *.doc like that.. that searching process going till end of the URL. I tried some of the way but i cant get the solutions. if any body knows the solution please update here.. Thanks in advance Regards sasi

Benjamin Kemner · Answer

Here is a ten minute code from mind and google. I hope this will help you :) private void button1_Click(object sender EventArgs e) { List result = new List (); Uri myUrl = new Uri(http://www.dreamincode.net/forums/topic/77912-ftp-download-delete/); string clean = myUrl.Scheme + :// + myUrl.GetComponents(UriComponents.HostAndPort UriFormat.UriEscaped); for (int i = 0; i { clean += myUrl.Segments[i]; result.AddRange(ParseSite(clean)); } // tada! result has all your files hope so ;) } private string[] ParseSite(string url){ List result = new List (); string data; // get the data with httprequest in data  not in mind now ;) MatchCollection m1 = Regex.Matches(data @( .*? ) RegexOptions.Singleline); foreach (Match m in m1) { string value = m.Groups[1].Value; Match m2 = Regex.Match(value @href=\(.*?)\ RegexOptions.Singleline); if (m2.Success){ string file = m2.Groups[1].Value; if (file.EndsWith(.doc) || file.EndsWith(.log) /* or whatever you want */ ) { // if file starts with http:// it absolue else you have to re-add the host here easy task ;) result.Add(m2.Groups[1].Value); } } } return result.ToArray(); }