New to C# - Advice for speech based assistant project.

Question

Hello, thank you for taking the time to read this and providing me with any help you can.

I am new to programming but I need a hobby over the next year or two so I thought I would learn some programming. 

As a project I wanted to build a speech based assistant. I want to start simple and add features over time. From just a basic speech recognition and programed response, to a version something like whats found in this video..

http://www.youtube.com/watch?v=ORzXuNWfOQo

I have no false hope in thinking this will be easy but like I said I want to start at the very basic and build on it.

So far I have built a very small program that can recognize commands from a list like "Hello, how are you?" and have it respond with a preset response from a list "I'm fine thank you."

So does anyone that watches that video tell me whats going on there? Is something like that as a end goal way out of the realm of  probability?

I know adding AIP such as wolfram would be needed as well as other API.

So my starting question would be this. Other than learning the code, what are some things I should look at in order to make this happen?

Any other advice?

Thanks for any and all help.