Problems with special letters like ÆØÅ from html code to listbox

Question

Hi

I'm trying to parse a HTML page where I ran into a small problem(well sort off big to me). F.eks lets take the letter Ø. In the HTML code the Ø = "Ø".

I thought I could replace the string like this:
line = line.Replace("Ø", "Ø").Replace("Ø", "Ø").Replace("Ø", "Ø");

But when I add the string to the listbox, it comes out like f.eks: "DD8;ivind".

How can I print out special characters like this in a correct way?

Also, I'm going to insert this data into a mysql database, how will this look like? When the Ø is "Ø", it becomes the same in the database as well.

If it is to any relevanse, I read the page like this:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.AllowAutoRedirect = true;
req.Method = "GET";
req.Timeout = 6000;
WebResponse res = req.GetResponse();
StreamReader sr = new StreamReader(res.GetResponseStream());
page = sr.ReadToEnd();
sr.Close();
res.Close();
parsePage();

and the web page content-type is:
<meta http-equiv="content-type" content="text/html; charset=iso-8859-1">

I hope there is an easy way to my problem.

Regards,

LordSvae

Torbjorn Svae · Answer

Might have used the wrong word. What I meant was that you get the value/content by collecting the name=summin thing. That I can't do on the IMDB site as all I can get that way is the title and year and still have to parse both that and the rest. And if they change the site the reader will fail and whatever I have next on my code won't happen so that way I won't get other trouble then changing the regular expression and it works again. At this point its easy but what would happen in the future I don't know. Just started learning this language. I do like your method and will keep it in the back of my head for something usefull in the future.

Sam Hobbs · Answer

I am familiar with IMDB. I use it often. I think that parsing the pages using the DOM would be completely appropriate. The IMDB might be a better site for me to use in my sample code but the IMDB is more likely to change in the future. The USA LOC has changed very little; at least the part I use in the sample. That however is not relevant to you except that scraping in the manner I suggest should work and would be appropriate. Of course there might be something I am missing about why you say it would not be appropriate. You might be able to compromise and do a little of both. You could get the HTML using the way you are doing that but then put the HTML into a document that you can parse using the DOM. I tried doing something such as that but did not get it working; I forget why not. I probably had difficulty figuring out how to get a HTML document form HTML without using the WebBrowser control.

Torbjorn Svae · Answer

I found the problem. I was replacing some symbols prior to that one above which then off course doesn't happen. When I fixed that and ran the replace it worked like a charm. Even in the mysql database. I do like your method tho but I don't think it applies to my case as I'm cathering data from IMDB and parsing the page. Your method would be a lot easier tho to be frank.

Sam Hobbs · Answer

I don't know what the data is that you are processing but I probably would not do it the way you are doing it. Look at my article about web site scraping if you want to know how I would do it.