WTF is this "sleep" thing everyone keeps talking about?
Ok, so...
You know how you occasionally find a site that has a LOT of stuff you want to download?
Say, for example, the images on TheFunniest.info?
This is where we enter the realm of "windows kinda sucks sometimes".
Cause in order to download all of the files (let's say image files) on a given site, you'll either need a custom script, FTP access, a special utility, or a lot of clicking by hand. Basically, downloading all files of a given type on windows requires an awful lot of legwork.
I however, have discovered the wonder that is linux file gets.
Instead of having to click each item and save it individually or use a firefox plugin like DownThemAll, I can simply type:
wget -nd -np -r -A=jpg,gif,png,bmp http://thefunniest.info/
and get a big fat dump of all of the jpg, gif, png, and bmp image files on there, as long as I'm willing to devote asstons of download time and processor usage to actually retrieve said files.
Now, just doing that by itself will likely get you a copy of robots.txt instead of actual files, but there are some spiffy ways around such things that are "slightly impolite" when used sparingly. (It's punishable by fully precedented legal action for major search engines to use said 'spiffy ways' but still allowed for personal use as long as it's not overly abused and therefore traverses the gap between use and abuse and actually causes tangible harm to webmasters.)
So.
Anyone got any sites they want to grab small files off of so I can play around with some syntax and load testing? If it's less than 100 files and they're all under a megabyte in size I'd be interested in fucking around a bit. I could easily test this on a webserver of my choice, but I figure if anyone's in the process of doing a bunch of "right click -> save as" I could both help someone else AND myself at the same time.
In other news, I've been fucking around with some of the extremely convoluted databases at work (more than 15 years old as far as I've been told, and only got truly refactored once, when they upgraded from access to MSSQL somewhere in the last decade) and it's become apparent to me that in order to get anything other than a lecture pit understanding of this thing, I'm going to have to bring it home and follow it for a few weeks.
And now I should really consider going to bed.
It's 4am, and tomorrow I have to turn my laptop into a remote FTP server inside of someone's LAN in order to discover whether or not they have a outgoing permissions issue for when they FTP jobs to us (cause they can't right now) and solve that either via secondary server, special permissions, or whothefuckknowswhat before I head out to at least one, if not two, other locations, AND catch up on the week's worth of work I left in my cube when I went home tonight.
Yay for friday.
Begin Function "EverQuestCommandSyntax"
/auction WTB* several gigabyte RAM sticks for own home tower, Have 4GB DDR capacity, PST**
End Function
*Willing to Buy
**Please Send Tell
You know how you occasionally find a site that has a LOT of stuff you want to download?
Say, for example, the images on TheFunniest.info?
This is where we enter the realm of "windows kinda sucks sometimes".
Cause in order to download all of the files (let's say image files) on a given site, you'll either need a custom script, FTP access, a special utility, or a lot of clicking by hand. Basically, downloading all files of a given type on windows requires an awful lot of legwork.
I however, have discovered the wonder that is linux file gets.
Instead of having to click each item and save it individually or use a firefox plugin like DownThemAll, I can simply type:
wget -nd -np -r -A=jpg,gif,png,bmp http://thefunniest.info/
and get a big fat dump of all of the jpg, gif, png, and bmp image files on there, as long as I'm willing to devote asstons of download time and processor usage to actually retrieve said files.
Now, just doing that by itself will likely get you a copy of robots.txt instead of actual files, but there are some spiffy ways around such things that are "slightly impolite" when used sparingly. (It's punishable by fully precedented legal action for major search engines to use said 'spiffy ways' but still allowed for personal use as long as it's not overly abused and therefore traverses the gap between use and abuse and actually causes tangible harm to webmasters.)
So.
Anyone got any sites they want to grab small files off of so I can play around with some syntax and load testing? If it's less than 100 files and they're all under a megabyte in size I'd be interested in fucking around a bit. I could easily test this on a webserver of my choice, but I figure if anyone's in the process of doing a bunch of "right click -> save as" I could both help someone else AND myself at the same time.
In other news, I've been fucking around with some of the extremely convoluted databases at work (more than 15 years old as far as I've been told, and only got truly refactored once, when they upgraded from access to MSSQL somewhere in the last decade) and it's become apparent to me that in order to get anything other than a lecture pit understanding of this thing, I'm going to have to bring it home and follow it for a few weeks.
And now I should really consider going to bed.
It's 4am, and tomorrow I have to turn my laptop into a remote FTP server inside of someone's LAN in order to discover whether or not they have a outgoing permissions issue for when they FTP jobs to us (cause they can't right now) and solve that either via secondary server, special permissions, or whothefuckknowswhat before I head out to at least one, if not two, other locations, AND catch up on the week's worth of work I left in my cube when I went home tonight.
Yay for friday.
Begin Function "EverQuestCommandSyntax"
/auction WTB* several gigabyte RAM sticks for own home tower, Have 4GB DDR capacity, PST**
End Function
*Willing to Buy
**Please Send Tell