(no subject)
I just stumbled upon some odd code that I wrote a long time ago, and discovered that I had actually gotten it to a working state perviously. "Wow," thought I, "I never get things to a working state when nobody is paying me to code them!"
Then I got upset at how slow it was working. It basically does some VERY heavy crunching on strings, running through every possible permutation. I had already done some optimization, but mostly to the dictionary loading part. That used to be mega painful, but now it's okay.
> read dictionary /Users/daniel/words
Successfully opened /Users/daniel/words for dictionary reading.
Read 234885 entries in 2 seconds.
Note that it's not just reading the dictionary, it's also pre-formatting the entries so that they are more easily used in the upcoming steps.
When I first decided that this needed to be optimized I found a phrase that resulted in 276 items in the output. This took 206 seconds, or 1.33 results per second. This was a longer phrase than I was previously using with this program, but I fully intend to use this with very large phrases eventually, so this performance was not acceptable.
Worse, there was an artificial limitation in the code that was limiting my results. Without that restriction, the same test phrase would result in 3228 results. At the old speed that would have taken 2,427 seconds, or 40 minutes. This was entirely unacceptable.
It took me a few hours of mostly thinking, but I got it. This latest run resulted in 3228 results. The time it took? Three seconds. That's a full 1076 results per second, or 809 times as fast as the original.
It's going to be a good day today. Now let's see if I can solve this pesky little problem that threatens to make my life very uncomfortable soon if I don't get it solved... today seems to be the day to do it.
So that I'm not being overly mysterious for those who have the skills and knowledge to follow along at home: http://code.google.com/p/libarg/
Then I got upset at how slow it was working. It basically does some VERY heavy crunching on strings, running through every possible permutation. I had already done some optimization, but mostly to the dictionary loading part. That used to be mega painful, but now it's okay.
> read dictionary /Users/daniel/words
Successfully opened /Users/daniel/words for dictionary reading.
Read 234885 entries in 2 seconds.
Note that it's not just reading the dictionary, it's also pre-formatting the entries so that they are more easily used in the upcoming steps.
When I first decided that this needed to be optimized I found a phrase that resulted in 276 items in the output. This took 206 seconds, or 1.33 results per second. This was a longer phrase than I was previously using with this program, but I fully intend to use this with very large phrases eventually, so this performance was not acceptable.
Worse, there was an artificial limitation in the code that was limiting my results. Without that restriction, the same test phrase would result in 3228 results. At the old speed that would have taken 2,427 seconds, or 40 minutes. This was entirely unacceptable.
It took me a few hours of mostly thinking, but I got it. This latest run resulted in 3228 results. The time it took? Three seconds. That's a full 1076 results per second, or 809 times as fast as the original.
It's going to be a good day today. Now let's see if I can solve this pesky little problem that threatens to make my life very uncomfortable soon if I don't get it solved... today seems to be the day to do it.
So that I'm not being overly mysterious for those who have the skills and knowledge to follow along at home: http://code.google.com/p/libarg/