Last night, I pushed out a formal release of my S3 tools.

The big thing is that it's now a Perl module, what with an installer and namespace in CPAN and everything.

It has a Freshmeat page. You can grab the tarball here, or check it out of Mercurial here.

It should soon show up in CPAN as Net::Amazon::S3::Tools.

link.

It's not much of an SLA, but it's there. It reaches only to Amazon's network edge, and you have to actually ask for it if you deserve a reimb.

I know that someone has already written a distributed multimountable filesystem for S3. But it's commercial and closed source ( Read more... )

Things are always harder and take longer.

I have s3getacl working and documented.

$ ./s3getacl example
# bucket: example
# owner: 5a1568e09392dad4b4ceb54f29f0a64d651a531350d6f720fbd2367eed995f08
OWNER:FULL_CONTROL
a00490decea9d0ad76e5ef8b450b3efa63861adccfb9197cfb42f837eb222df2:WRITE
USERS:READ
WORLD:READ
$ ./s3getacl example/thingee
# bucket: ( Read more... )

So, there was a command line toolset for manipulating S3, called the Hanzoarchives hanzo-s3-tools. They were kind of iffy in the installation and documentation, but they worked, more or less. However, their handling of S3 ACLs was pretty poor, very not useful. And as the last straw, it seems to have gone offline and vanished ( Read more... )

Brian has written a library that implements "XML Row Storage", and has added it to his HTTP Storage Engine.

I've got a couple of hours to add it to my S3 Storage Engine.

I've already stumbled over the fact that my engine as it was won't build against a very recent MySQL 5.1 source tree. Type byte is now uchar, and the call parameters of ( Read more... )

Yesterday morning, I had an initial face-to-face meeting with a prospective client. We met at the Victrola Cafe. He's putting together a pretty neat Web2.0 custom app. My first task for him will be writing a little process that monitors a particular public database and writes changes into S3, to start capturing a historical timeline. There is a ( Read more... )

Immediately after my presentation about my MySQL S3 storage engine, I was interviewed by Eric Lai, a reporter from ComputerWorld. The resulting article is here, and was abstracted by the conference here.

Also, Sheeri Kritzer recorded me. Here is the audio and the videoWhen I get the larger form of the video, I'll push it into an S3 bucket/item ( Read more... )

My talk was right after the PBXT storage engine talk. PBXT is darncool, it looks to be a leading contender for fast localstore nearly transactional data with lots of blobs and varchars. I was taking lots of notes, especially his "gotchas for storage engine developers" and his "streaming blobs" ideas ( Read more... )

I've described this idea to a few people, but I figured I would post it here.

I've had an idea for using Amazon AWS S3 to distribute MySQL cluster replication data.

The existing architecture for MySQL clustering is as follows:

1. The master has N slaves
2. The master copies each binlog replication to each slave. If there are 7 slaves, then the master ( Read more... )