The Problem with using Wiki API in C#

Question

Hello to everyone! Recently I've begun to wtite a small project in C# for creating ontologies from Wki. I'm using the Wiki api at the moment, but permanently got an error( Here is a code and an error:
System.Net.ServicePointManager.Expect100Continue = false;
HttpWebRequest myRequest =
(HttpWebRequest)WebRequest.Create("http://en.wikipedia.org/w/api.php?action=query&titles=Car");
using (HttpWebResponse response = (HttpWebResponse)myRequest.GetResponse())
{
string ResponseText;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
ResponseText = reader.ReadToEnd();
}
SearchString.Text = ResponseText;
}
and the error is
The remote server returned error 403.

Sam Hobbs · Answer

Add:
myRequest.UserAgent = "something";

Suthish Nair · Answer

Alex, might your ip is restricted to access the url. copy paste url to explorer and check the xml get returning: http://en.wikipedia.org/w/api.php?action=query&titles=Car

Answer

You first need to confirm if you have encountered a "No directory browsing" problem. You can see this if the URL ends in a slash '/' rather than the name of a specific Web page (e.g. .htm or .html). If this is your problem, then you have no option but to access individual Web pages for that Web site directly.

It is possible that there should be some content in the directory, but there is none there yet. For example if your ISP offers a 'Home Page' then you need to provide some content - usually HTML files - for the Home Page directory that your ISP assigns to you. Until the content is there, anyone trying to access your Home Page could encounter a 403 error. The solution is to upload the missing content - directly yourself or by providing it to your ISP. Once the content is in the directory, it also needs to be authorised for public access via the Internet. Your ISP should do this as a matter of course - if they do not, then they have missed a no-brainer step.

If the entire Web site is actually secured in some way (is not open at all to casual Internet users), then an 401 - Not authorized message could be expected. It is possible, but unlikely, that the Web server issues an 403 message instead.

Some Web servers may also issue an 403 error if they at one time hosted the site, but now no longer do so and can not or will not provide a redirection to a new URL. In this case it is not unusual for the 403 error to be returned instead of a more helpful error. So if you have recently changed any aspect of the Web site setup (e.g. switched ISPs), then a 403 message is a possibility. Obviously this message should disappear in time - typically within a week or two - as the Internet catches up with whatever change you have made.

If you think that the Web URL *should* be accessible to all and sundry on the Internet and you have not recently changed anything fundamental in the Web site setup, then an 403 message indicates a deeper problem. The first thing you can do is check the URL via a Web browser. This browser should be running on a computer to which you have never previously identified yourself in any way, and you should avoid authentication (passwords etc.) that you have used previously. Ideally all this should be done over a completely different Internet connection to any you have used before (e.g. a different ISP dial-up connection). In short, you are trying to get the same behaviour a total stranger would get if they surfed the Internet to the Web page URL.

If this type of browser check indicates no authority problems, then it is possible that the Web server (or surrounding systems) have been configured to disallow certain patterns of HTTP traffic. In other words, HTTP communication from a well-known Web browser is allowed, but automated communication from other systems is rejected with an 403 error code. This is unusual, but may indicate a very defensive security policy around the Web server.

Fixing 403 errors - CheckUpDown

The first question is whether the Web page for your URL is freely available to everyone on the Internet. If this is not the case, then you may need to provide two items 2. Web Site User ID and 3. Web Site Password for your CheckUpDown account - but only if the site uses HTTP Basic Authentication. The Web Master or other IT support people at the site will know what security and authentication is used.

If however the Web page is open to all comers and there have been no fundamental changes recently to how the Web site is hosted and accessed, then an 403 message should only appear if the Web server objects to some aspect of the access we are trying to get to the Web site. Because it indicates a fundamental authority problem, we can only resolve this by negotiation with the personnel responsible for security on and around the Web site. These discussions unfortunately may take some time, but can often be amicably resolved. You can assist by endorsing our service to the security personnel. Please contact us (email preferred) if you see persistent 403 errors, so that we can agree the best way to resolve them.