Problem with byte array with more size

Question

Hi all i want to upload file using web service.. For that i have written this code in client application. I have a problem with more no. of bytes. like for 1 gb(1073741823) I am not able to create the byte array with this size. i want to upload maximum 2 gb file. byte [] data = new byte [1073741823]; Getting following error at run time. Exception of type 'System.OutOfMemoryException' was thrown. System.IO. FileStream stream = new System.IO. FileStream (fileInfo.FullName System.IO. FileMode .Open System.IO. FileAccess .Read) BinaryReader br = new BinaryReader (stream); long numBytes = fileInfo.Length;//here fileinfo.length value is 1073741823 data = br.ReadBytes(( I am again getting the same exception error in the above line. How to solve this problem..how can this work till 2 gb size( 2147483647 bytes) Please help.. int )numBytes);

Sam Hobbs · Answer

Deepika everyone is telling you that you cannot use a huge amount of main memory. 2GB is huge. 2GB is the maximum amount of memory that 32-bit Windows can use for all applications which means that the real amount of memory that 32-bit Windows applications can use is less than 2GB. If you are using 32-bit Windows then it might work if you use a system with 64-bit Windows but the program should be modified to send the data in pieces.

Vulpes · Answer

If it's a web service which you've written yourself then this article may help you to upload a very large file in chunks: http://www.codeproject.com/KB/webservices/WebserviceFileUpload.aspx

Zoran Horvat · Answer

There is no way to do it other than to read the file content in smaller blocks one block at a time. There are not enough resources on your computer to do so. In addition 32-bit process cannot use more than 2GB of RAM (4GB with specific settings on 64-bit Windows). You can check it on this address: http://msdn.microsoft.com/en-us/library/aa366778%28v=vs.85%29.aspx Zoran

Deepika Chaudhary · Answer

System.IO. FileStream stream = new System.IO. FileStream (fileInfo.FullName System.IO. FileMode .Open System.IO. FileAccess .Read) BinaryReader br = new BinaryReader (stream); long numBytes = fileInfo.Length;//here fileinfo.length value is 1073741823 data = br.ReadBytes(( int )numBytes); I am again getting the same exception error in the above line. How to solve this problem..how can this work till 2 gb size( 2147483647 bytes) Please help.. the above code is working till 400 mb... is there any other solution bcoz i can't change the much code now..the application is already running..i just want to allow maximum 2 gb size.

Zoran Horvat · Answer

Try this: try { long size = 1; for (; ; ) { byte[] buffer = new byte[size]; Console.WriteLine(Allocated {0} bytes size); size *= 2; } } catch (System.Exception ex) { Console.WriteLine(ex.Message); } It will always fail with out of memory exception. You can't and you shouldn't allocate buffers that are more than say a megabyte in length. It is a design flaw if you try to do such thing instead of searching for a better solution. Zoran

Deepika Chaudhary · Answer

byte[] data = br.ReadBytes(( the maximum size can be 2 gb.. i want to send this data value to web service uploadmethod(byte[] data) this is working file till 400 mb. but after that i am getting error.. Exception of type 'System.OutOfMemoryException' was thrown. int )numBytes);//numBytes = 1073741823

Zoran Horvat · Answer

Buffer is allocated as byte[] and you can use it in successive calls to Web service in the loop. You can't have single buffer containing all data from such a large file because your computer and OS do not support such large buffers. There is off course a solution and that is to open socke connection to Web service and to mimic SOAP protocol yourself so that you write contents of the file iteratively into the socket. But that would be extremely complex solution for the problem. Zoran

Deepika Chaudhary · Answer

i need the value in byte..as i have a web service in which upload method i am using byte[] as paramenter..

Zoran Horvat · Answer

You can't allocate such a large block in memory because of memory size limitations and maximum memory assigned to a process limitations. Whenever working with files you should operate in smaller steps so that you can control amount of memory used at any time. Otherwise your application might crash simply because user has tried to execute it on a big file. Here is the code which iterates through file in smaller steps. You can control size of the buffer used. byte[] buffer = new byte[100000]; using (FileStream fs = new FileStream(tmp.txt FileMode.Open)) { int bytesRead = 0; do { bytesRead = fs.Read(buffer 0 buffer.Length); // Do something with the data } while (bytesRead > 0); } Zoran