0
Hi,
You can refer to these links
https://www.codeproject.com/Articles/1087859/Web-crawling-with-Csharp-part-one
https://stackoverflow.com/questions/10452749/simple-web-crawler-in-c-sharp
https://forums.asp.net/t/1810399.aspx?how+we+create+own+web+crawler+
Hope this helps