built in function

Question

hi everyone

kindly do let me that is there any built-in function to select unique words or the words which come atleast one time in corpus?

actuly i wanna make vocab list so i need to select every word uniquely even if it occurs multiple time in a given corpus..

Nilesh Shah · Answer

I'm not sure if it will be a good idea to process big text collection using C# or any dot net code. Can't you leave it to SQLServer? You can use SQL Server Full text feature Create full text index on the table containing your data it will automatically create a search terms catalog which you can use Fol. SQL query can give you search terms sys.dm_fts_index_keywords_by_document ( DB_ID( 'database_name' ) OBJECT_ID( 'table_name' ) ) Also have a look at fol. links http://msdn.microsoft.com/en-us/library/cc280607.aspx http://technet.microsoft.com/en-us/library/cc280607.aspx