Out of Memory exception while converting byte array into string

Question

Hi I'm getting out of memory exception while converting byte array into a string. Initially I've stored content of a file(338 MB) into DataBase in varbinay format  after reading the content from DataBase I'm conveting that Byte array into string wher I'm getting the error . var val = cmd.ExecuteScalar();//Getting form DataBase System.Text.Encoding enc = System.Text.Encoding.ASCII; string myString = enc.GetString((byte[])val);//Converting into String Please someone tell me a solution. Thanks & Regards Ramana

Sam Hobbs · Answer

It was not long ago that the biggest computers did not have 16 MB of main memory. I am talking about computers that big companies such as Bank of America used. Also companies such as Boeing and Lockheed that built jumbo jets. Those computers cost millions of dollars. Young people these days don't appreciate how amazing it is to be able to process such a huge amount of data with computers that cost so little. 32-bit Windows is limited to 2 GB for an application even if virtual storage is used. A string requires that the memory for it be contiguous. It would be understandable if Windows has problems finding 676 MB of contiguous memory. Note that if Windows must use virtual storage (a disk drive) then that could become very innefficient. If paging is a possibility then consider designing your application to reduce paging.

Vulpes · Answer

It's just too big to deal with in one chunk. As strings are always unicode in .NET the ASCII byte array is going to convert into a string needing 676 MB of memory! Assuming you don't really need to have a string as large as that in memory at one time I'd copy the byte array in chunks to a much smaller array (say no larger than 10 MB) convert that to a string process it and then deal with the next chunk. In fact physical copying should be unnecessary as the GetString() method has an overload which takes as additional parameters the starting index in the byte array and the number of bytes to decode.