RSS Feed Project in .NET

Introduction

The RSS Feed project is aimed as demonstrating writing C# code to consume RSS feeds from the internet and putting the data from these RSS feeds into a database for you to use in your own applications.  In my sample we are concerned with getting a link, title and description for each of the items in the feed.  We will be consuming the RSS feeds provided by SlashDot.org and WiredNews.com.

Project Setup

To run the project you will need to create a Sql Server database and run the RssFeed.sql script.  RssFeed.sql can be found in the support directory in the zip file.  This will create the tables and stored procedures you will neeed.  It also populates the WebNewsSource with the data needed to use the RSS feeds from SlashDot.org and WiredNews.com.  You will also need to modify the connection stiring in the Rss Feed.exe.config to point to your database.   The Rss Feed.exe.config needs to be in the same directory as the exe that you are running.  When running the app from the IDE in debug mode the exe is in the bin\debug directory.

The Database

The database I designed for this application consists of two tables.  WebNewsSource holds the information we need to connect to a RSS feed and the field names of the data that we want to keep ( Link, title and description ).  The WebNewsItem table is the table that we will use to hold the data we retrive from the feeds.



Defining a RSS Feed

An RSS Feed is a specially formated XML document.  I used a database to hold the data that the application will need to know how to consume the RSS feed.  The data is kept in the WebNewsSource table.  The application will need to know the URL of the RSS feed, this data is kept in the RSSURL field.  The RSSSelect field tells the application how to get to the nodes in the RSS Feed that we are interested in.  The RSSTitle, RSSLink and RSSDescription fields tell the application which pieces of data we want to store in the Title, Link and ArticleDescriptions fields respectively in the WebNewsItem table.

The Code

The sample contains two classes.   Feed.cs is the class that will get the RSS feed as an XML document and go through it to get the data we are looking for.  RSSItem.cs will read the RSS Feed information from the database and hold it.  We need to use the System.Xml.XPath, System.Data, and System.Data.SqlClient namespaces

using System.Xml.XPath;
using System.Data;
using System.Data.SqlClient;

The GetFeed() method of the Feed class will get the RSS Feed and bring it back into a XPathDocument using a select that we defined in our database. It then loops through the nodes of the document passing each node to the ProcessNode method.

public void GetFeed()
{
try
{
// Bring back the Feed
XPathDocument doc = new XPathDocument(rssURL);
XPathNavigator nav = doc.CreateNavigator();
XPathNodeIterator iter =nav.Select( rssSelect) ;
// Loop through the nodes
while ( iter.MoveNext())
{
// Get the data we need from the node
ProcessNode ( iter.Current );
}
}
catch (Exception ex)
{
Console.WriteLine ( ex.Message.ToString());
}
}

The ProcessNode method gets the child nodes of the passed XPathNavigator and loops through them.  We are looking for three nodes (Link, Title and Description) as we defined them in the database.  We then make sure that the link and title are at least 10 characters long and pass the WebNewsSourceID, link, title and description to the InsertNews method.

void ProcessNode(XPathNavigator lstNav)
{
string title="";
string link ="";
string description="";
// Get the child nodes
XPathNodeIterator iterNews = lstNav.SelectDescendants( XPathNodeType.Element, false );
// Loop through the child nodes
while ( iterNews.MoveNext())
{
// Save the current Name
string rssName = iterNews.Current.Name ;
// Is this the title?
if ( rssName.ToUpper() == rssTitle.ToUpper() )
{ title = iterNews.Current.Value;}
// Is this the Link?
if ( rssName.ToUpper() == rssLink.ToUpper() )
{ link = iterNews.Current.Value;}
// Is this the Description?
if ( rssName.ToUpper() == rssDescription.ToUpper() )
{ description = iterNews.Current.Value;}
}
// Make sure the link and title are at least 10 characters long
if ( link.Length > 10 && title.Length > 10)
{
// Update the database
InsertNews( webNewsSourceID, link, title, description);
}
}

The InsertNews method reads the connection string from the config file.  Sets up a command to run the upWebNewsItem_Insert stored procedure and executes it.

void InsertNews( int webNewsSourceID, string link, string title, string description)
{
try
{
// Create Connection
SqlConnection objConnection =
new SqlConnection(System.Configuration.ConfigurationSettings.AppSettings["ConnectString"]);
// Set up the command
SqlCommand objCommand = new SqlCommand("upWebNewsItem_Insert",objConnection);
objCommand.CommandType = CommandType.StoredProcedure;
// Set up the parameters
objCommand.Parameters.Add("@WebNewsSourceID", SqlDbType.Int).Value = webNewsSourceID;
objCommand.Parameters.Add("@Link", SqlDbType.VarChar, 255).Value = link;
objCommand.Parameters.Add("@Title", SqlDbType.VarChar, 500).Value = title;
objCommand.Parameters.Add("@ArticleDescription", SqlDbType.VarChar, 1000).Value = description;
objCommand.Parameters.Add("@ID", SqlDbType.Int);
objCommand.Parameters["@ID"].Direction = ParameterDirection.Output;
// Execute the command
objConnection.Open();
objCommand.ExecuteNonQuery();
objConnection.Close();
}
catch (Exception ex)
{
Console.WriteLine( "ERROR:" + ex.Message.ToString() );
}
}