mem
Yesterday in the evening, I wasn't feeling like doing something intelligent, so I wrote this mega-program, which determines what's the maximum amount of memory you can allocate.
#include #include #include #define MEBIBYTE 1024*1024 int main(int argc, char **argv) { void *p, *q; const size_t step = 10 * MEBIBYTE; size_t size = step; p = q = 0; while ((q = realloc(p, size)) != NULL) { size += step; p = q; } printf("failed at %lld MiB (%lld)\n", size / (MEBIBYTE), size); printf("doing memset... :D\n"); memset(p, 0, size); free(p); exit(0); }
I ran it on my computer (Athlon64 X2, 4 GB RAM) which has Fedora Linux x64 installed ATM. Result:
[s@x2 tmp]$ ./testmaxmem failed at 83886080 MiB (87960930222080) doing memset... :D Killed [s@x2 tmp]$
80~ gibibytes. [Quake 3 voice] IMPRESSIVE! The comp started to make incresingly louder noise with the HDD and then the OS killed the process. The HDD noise is a bit strange, because I don't have neither a swap file, nor a swap partition. Also, why 80 gibibytes? It could've went all the way to allocating ~((u_int64_t) 0). Maybe it's because on 64 bit processors, not all of the address lines are actually used, but only something like 48? So the processor is simply unable to address the full 2^64 address space.
It would also be very interesting to see how all this looks from virtual memory's point of view. Due to my lack of knowledge, I can only make a very far approximation of what happens. It's written in malloc's manpage, that when there is an attempt to allocate memory chunk larger than a certain threshold (128kb by default), malloc takes this memory from a memory chunk that it allocates with an ANONYMOUS (there is no file associated) mmap(). Virtual memory is based around this pagetable, which is a map of the whole processor's address space, it contains the map of hard disk blocks -> memory locations, and flags which indicate wheter this page is in RAM or not. And I guess there is a special block address which indicates that this is not an HDD address? And when the program does memset(), the OS begins to actually map the memory, that is, it sets the present flag in the appropriate entry in the pagetable and looks for a free place in RAM, and writes this free place address to this entry too. So then it sees that there is no free space in RAM anymore, and it has no swap partition/file, so it cannot free some RAM by swapping some unused currently memory pages, which belong to other processes, to hard drive and the policy is to simply kill the process which is trying to access more memory? D: Okay, but what's with the segments? When the program dies -- what happens then? The mmap() made a segment for this process and had put it into Local Descriptor Table and OS simply deletes this entry? Ugh, enough speculating, I should read a book or something.
Some time ago, I also ran a similar program on WinXP (not 64) (on this same computer), the maximum allocatable amount of memory is 2~ gibibytes and memset() executes okay. It takes it several seconds (!) to finish.
I also ran it on my linux work computer, which has 1 Gb of RAM, result: I had to physically reboot the comp, because it started swapping so insanely that even the mouse froze :D
I wonder what happens on Vista64.