The Natural Progression of Storage Needs
We have 100 100Kb images we want to put on our website.
No Problem lets just throw them up there and serve them with apache
We have 1,000 100-1Mb images we want to put on our website.
No Problem that's only a Gb of data maximum. It may take a while to transfer and make sure things are consistent between redundant machines.
We have 10,000 100K-10Mb images we want to put on our website.
The average of those is fairly small so we can still upload them with the indexes. I think we'll be okay although lets go ahead and order bigger hard drives for our web servers.
We have 100,000 100K-100Mb image we want to put on our website.
Jeebus, we can't store these on the same server anymore. We'll have to get a redundant pair of storage machines with large disk and mirror the data. We can hack something together so the mirrors are in sync. I'll get DNS round robin going so we can load balance between the two.
We have 1,000,000+ 100K-100Mb images we want to put on our website.
Hmm, we'll need to distribute these across multiple servers now since we can't just stash them on one machine and do a simple mirror. Redundancy will be an issue and our hardware costs and maintenance costs will go up as we try to make these redundant. Also we'll need to start talking about backing this stuff up to tape and storing that off site since you're not storing all of this on your own machine anymore. We should also talk about skipping apache and going with a different client daemon depending on your throughput. I'll talk to F5 and see how much a BigIP will cost us.
No Problem lets just throw them up there and serve them with apache
We have 1,000 100-1Mb images we want to put on our website.
No Problem that's only a Gb of data maximum. It may take a while to transfer and make sure things are consistent between redundant machines.
We have 10,000 100K-10Mb images we want to put on our website.
The average of those is fairly small so we can still upload them with the indexes. I think we'll be okay although lets go ahead and order bigger hard drives for our web servers.
We have 100,000 100K-100Mb image we want to put on our website.
Jeebus, we can't store these on the same server anymore. We'll have to get a redundant pair of storage machines with large disk and mirror the data. We can hack something together so the mirrors are in sync. I'll get DNS round robin going so we can load balance between the two.
We have 1,000,000+ 100K-100Mb images we want to put on our website.
Hmm, we'll need to distribute these across multiple servers now since we can't just stash them on one machine and do a simple mirror. Redundancy will be an issue and our hardware costs and maintenance costs will go up as we try to make these redundant. Also we'll need to start talking about backing this stuff up to tape and storing that off site since you're not storing all of this on your own machine anymore. We should also talk about skipping apache and going with a different client daemon depending on your throughput. I'll talk to F5 and see how much a BigIP will cost us.