Comments | bramcohen: print() experiencing deja vu

bramcohen

print() experiencing deja vu

Oct 15, 2009 12:07

I have a very multithreaded Python3 app I'm running which I have a whole bunch of calls to print() in. I don't generally do multithreading, but this is spinning up a whole bunch of servers to check if they're working right. I have another test which is non-multithreaded and reproducible, and will of course have another test of actually running ( Read more... )

Comments 4

cypherpunk95 October 15 2009, 21:00:03 UTC

One possible cause: if you do the print(), the actual string may get buffered before it's output. If you launch a thread at that point, both threads may believe it's their job to output what's in the buffer.

I've definitely seen this happen with fork, but I'm not positive it should happen with threads.

Have you seen a claim that sys.stdout is thread-safe? peterschuller October 15 2009, 21:31:12 UTC

Check out pythonrun.c. On cursory examination I don't see anything special going on with stdout; it's just an io object like others. Why would there be an expectation that this be thread-safe? Especially given that it's defined to be using stdio style buffering. Is there a claim that IO objects in python 3 should be thread-safe (seems strange)?

sys.stdout in Python 3.0 (based on runtime inspection) is a TextIOWrapper. I don't see any synchronization going on in io.py, which is what I would expect.

As far as I can tell this behavior is expected.

(I didn't find where print() itself is defined, though any thread-safety added there would not jive well with other uses of stdout anyway.)

And on the topic of a work-around peterschuller October 15 2009, 21:32:59 UTC

You might use the logging API which is thread-safe (though I don't remember whether this is defined to be the case or just is).

Re: And on the topic of a work-around peterschuller October 15 2009, 21:35:40 UTC

(Or make sys.stdout thread-safe by wrapping it before loading any other modules that have a chance to de-ref sys.stdout. Not sure whether there is a standard thread-safe wrapper in py3 or whether you'd have to implement it. Also not sure whether the print() function can be counted on to deref sys.stdout - I would presume so.)