Can I gather customer information from web crawler?

Question

There is a need to automate the process where advisor will update the customer name in textbox and on a click of a button a search on all the possible websites on customer's demographic details can take place and a probable match thrown for ascertaining if it is a correct match or not. can we fetch the customer information from Social sites on web through my we application if YES the How please guide .

Praveen Kumar Sreeram · Answer

Hi Pankaj Please have a look at the below url. http://stackoverflow.com/questions/10452749/simple-web-crawler-in-c-sharp Hope it helps. Thanks Prawin

Pankaj Kadam · Answer

It would be very useful Praveen if u could give me some code snippet. or any example

Praveen Kumar Sreeram · Answer

Hi Pankay You can definitely crawl website using C# and other technologies. However nowadays social websites are very intelligent and note that if you make automatic requests (by developing some bots) they will track your IPAddress and block you. Anyways below is the process of crawling any website. Create HTTPRequest to the required Website or the URL of your customer. Retrieve the HTTPResponse if the request in the Step1 is successful. Parse the HTML to get the customer information. Please use Regex to make the HTML parsing run quick. Only get the required information from the retrieved HTML Play around the data that you have got. Ideally the above process is not suggested to get the data from popular websites. You need to identify if the concerned website provide APIs to get the data. Most of the popular Social Networking sites provide API to retrive the data. You would have to consume those API to get the data seamlessly. Hope It helps.