I am currently having a problem with declaring or filling a large array with data because I get a dialog box saying "Out of memory", originating from CMemoryException.
I am trying to create an array or vector (tried both) with around 50000 elements of an object, where sizeof(MyObjectClass) returns around 37000 bytes.
If I try to just fill up a vector or a CArray element by element, then I get around to filling with somewhere near 16000 elements before getting the Out Of Memory exception. That should be close to 600MBs?
I have 8GB RAM on the machine and only 4GB are being used according to Windows Task Manager. So the amount of physical RAM should not impose a problem. I am running C++ MFC in Visual Studio 2010, 32-bit.
Also if I try to write
MyObjectClass* heaparray = new MyObjectClass[50000];
then I immediately get that very same Out of memory error, on that very row.
Any ideas? Thank You in advance!
UPDATE: I have also tried to simply create a TestStruct with the fields:
struct TestStruct
{
long long field1;
GUID field2;
GUID field3;
GUID field4;
TCHAR field5[256];
TCHAR field6[4];
TCHAR field7[258];
TCHAR field8[1026];
TCHAR field9[258];
TCHAR field10[16386];
TCHAR field11[258];
};
TestStruct* heapArr = new TestStruct[50000];
Still the same...I get a "Out of Memory" exception when executing the last line of code. Isn't one of the great things with the heap supposed to be possibility to be limited only by RAM (more or less) when handling big data. And yet...since it crashes already at 600MB of allocated space I cannot agree that that is very big data either...or should I? :/
This is a fun one. Both Vectors and arrays are stored contiguously in memory as stated here.
You are not only looking for 1850000000 bytes (1.72295 gigabytes) in memory, but one unbroken chunk of memory that big. That will be hard to find. If you switch to a different data structure that does not do contiguous storage (say a linked list) then you may be able to store that much.
Note: that will also make each object just a bit bigger.
What would be best would be to see if there is any way to just buffer the objects; load only the ones you will update and load the others on the fly when you need them. I have my doubts that you are doing cpu operations on more than one at a time. If you do it right (with threading most likely) you won't even suffer any slows from reading/writing them.
More information about what you are working on would be helpful. There may even be a way to just have an array filled with a type identifier, if your object has less than 2,147,483,647 (size of int) variations. You could store an array of integers that the class could be generated from (a toHash and fromHash that would be 50000 * 4 bytes = 195.312 kilobytes), that may work for you too. Again, it depends on what you are working on.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments