Linux VM
You know I have found that the Linux 2.4 memory scheduler needs some work. So for those of you that might not know, but care, all programs are given 4GB of memory, even if your machine doesn't have that much space. Instead of giving the memory to the program at the beginning, the OS will give it to the program as it uses. So what happens when there is no physical memory left? The kernel will take some chunk of memory and write it out to disk and then use that memory for anything else the program uses. When the evicted memory is used again, the kernel will kick out some other page and reload it from disk. The memory blocks that get kicks out depends on the algorithm that the kernel uses, but is generally the block that hasn't been touched in a while. This works just fine in general.
A second thing the OS does in the background is that to improve disk access, is that the kernel won't always go to the disk whenever you access the disk. Instead it tries to be intelligent and keep some blocks of disk in memory so that the OS doesn't have to read/write data to the disk, which is really slow. Also seems to be a reasonable thing to do.
Now let's combine these two concepts. Now supposed you have 2 GB of memory. Program A uses 1 GB of the memory. Program B uses 750MB of data. The kernel has about 250MB of the disk buffered in memory. Program B asks for 200 MB of memory. Now where does this memory come from:
1) Take the memory block that has not been accessed in the longest time
2) Write any modified data in the disk buffer and reclaim that memory for Program B
If you guessed 1, congratulations. You are a Linux kernel developer. This doesn't seem like a bad idea at face value, but really what this results in is that the kernel is constantly trying to load the memory that it told a program that it had. Say you take some memory from Program A. When A tries to access that memory, you have to discard some other memory and read A's memory from disk, which is slow. You just keep ending up doing that. I'm trying to figure out why program is running so slow and the kernel is doing this retarded trick, causing it to spend all of it's time reading data off of disk and write it back to disk. It turned out that some had turned on swap (the ability to write memory to disk) on these machines to debug another problem.
A second thing the OS does in the background is that to improve disk access, is that the kernel won't always go to the disk whenever you access the disk. Instead it tries to be intelligent and keep some blocks of disk in memory so that the OS doesn't have to read/write data to the disk, which is really slow. Also seems to be a reasonable thing to do.
Now let's combine these two concepts. Now supposed you have 2 GB of memory. Program A uses 1 GB of the memory. Program B uses 750MB of data. The kernel has about 250MB of the disk buffered in memory. Program B asks for 200 MB of memory. Now where does this memory come from:
1) Take the memory block that has not been accessed in the longest time
2) Write any modified data in the disk buffer and reclaim that memory for Program B
If you guessed 1, congratulations. You are a Linux kernel developer. This doesn't seem like a bad idea at face value, but really what this results in is that the kernel is constantly trying to load the memory that it told a program that it had. Say you take some memory from Program A. When A tries to access that memory, you have to discard some other memory and read A's memory from disk, which is slow. You just keep ending up doing that. I'm trying to figure out why program is running so slow and the kernel is doing this retarded trick, causing it to spend all of it's time reading data off of disk and write it back to disk. It turned out that some had turned on swap (the ability to write memory to disk) on these machines to debug another problem.