RPC for Twisted Python servers

Lesson learned: if you need an RPC protocol for Python/Twisted, don’t use AMP, use HTTP (with urllib if necessary for synchronous calls). AMP’s 65k message limit is really annoying to work within, and if you strace it you’ll see it doing tons of 1-byte reads/writes. HTTP overhead sucks, but I’d rather have that than the limitations of AMP.

edit: after a few more months of experience, this turns out to be wrong. AMP is actually not too bad in terms of the socket calls it makes, and you can work around the 65k message limit (at the cost of extra round-trips) by using a chunked result protocol. urllib, on the other hand, is pretty awful for synchronous HTTP. It is the source of those 1-byte reads/writes, because it doesn’t have a buffering mechanism between the socket and the Python file interface it is trying to implement.

Leave a Reply

Your email address will not be published. Required fields are marked *