Thrift for Remote App Services

[code_martial]

We've just started moving some of the baseline application stuff to Python services running on independent app servers (no, not like J2EE app servers). One of the key aspects to consider in such a setup is the scalability of the bridge between the webserver and app server

Comments

[anomalizer]

The suckage is with the lack of quality server implementation in public domain.

[code_martial]

You have to write your own server for each service. The only thing that can be standardised may be the generation of code that says, "your code here". The real world difference it makes is just 2-3 days' worth of effort.

The suckage really is the IDL and its half-assed support for type safety which brings the worst of both worlds together. I expect Avro to be more suitable for this. Sadly, though, their first RPC implementation would be in Java but I hope others will be quick to catch up.

[anomalizer]

The hadoop project despite being open source seems to be following the principles of something developed by a large corporation. In spirit it always seems to say "For best results use with other Hadoop projects and do it all on Java".

[anonymous]

I'm not sure what a Python POD is? (a couple searches led me to believe pod is more akin to perl's DBI, which doesn't make sense in context)

Assuming it's a way of serializing python stuff; why not just use gziped pickled output and don't stuff it in json?

If you were serializing to json structures, that could make sense, but if you're stuffing a python serialized format into json, the json 'envelope' doesn't really add anything other than processing time.

Also, I've found in php that the platform specific serialize/deserialize is faster than xml or json. I would presume that's true in Python as well, so if you're communicating between two Python layers, there's no reason to speak JSON.

[code_martial]

POD is Plain Old Datatype (scalars, lists, dicts). JSON was brought in for cross-language data transfer between PHP and Python. PHP serialisation is marginally faster than JSON but the latter is much more compact.