'print' statement ignores 'sys.stdout'

[alanjds]

Replacing sys.stdout have no effect over print statement.

This test script outputs AssertionError: 0 chars printed, instead of 3, after printing 'foo' in the stdout.

import StringIO
import sys

sio = StringIO()
stdout = sys.stdout
sys.stdout = sio

print 'foo'

sys.stdout = stdout

chars = sio.tell()
assert chars == 3, '%s chars printed, instead of 3' % chars

[trotterdylan]

Effectively, the sys module does this:

from __go__.grumpy import Stdout

stdout = Stdout

The Print() function in the Go runtime writes directly to grumpy.Stdout. It does not get it from the sys module. So things work fine until you re-bind sys.stdout, as you've found.

There are a couple ways to fix this:

1. Write the sys module in Go and have the grumpy package import it at package init time so that the Print() function can access the sys module and write to the stdout attribute instead of a global Go variable. This is close to how CPython works. However, I think it would require adding some special casing into the import system, which could get ugly.

2. Make the sys module an instance of some class where stdout is a property whose setter updates the value of Stdout in the grumpy runtime package. Because there's no way to assign to Go package variables from Python at the moment, this would require adding a SetStdout() function to the grumpy runtime package, which feels a little hacky.

In short, there's no great way to fix it at the moment, however, I think some avenues will open up once Grumpy is self-hosted.

[alanjds]

What about implementing the print statement as a pure-python function, and hook the golang Print function to the python one?

I will need to reimplement the print stuff very soon anyway...

[trotterdylan]

If, for example, you define print() in __builtins__.py then you have the problem of accessing the __builtins__ Go package from within the grumpy runtime package, because they each would depend on each other.

[alanjds]

I am thinking about defining some _print_statement.py and hooking Print() to use it. Ideally, it should import only sys.stdout I did not yet took a look there to see what the Print() imports Il 23 apr 2017 1:18 PM, "Dylan Trotter" <notifications@github.com> ha scritto:

…

If, for example, you define print() in __builtins__.py then you have the problem of accessing the *builtins* Go package from within the grumpy runtime package, because they each would depend on each other. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#290 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJf5ywht3Zc_SoCZu71nbK2xoVoBm7Dks5ry3m_gaJpZM4NFOm8> .

[trotterdylan]

I think the fundamental issue you're doing to run into is circular dependencies between the grumpy runtime and the packages generated from the Python module you write.

…

On Sun, Apr 23, 2017 at 11:57 AM Alan Justino da Silva < ***@***.***> wrote: I am thinking about defining some _print_statement.py and hooking Print() to use it. Ideally, it should import only sys.stdout I did not yet took a look there to see what the Print() imports Il 23 apr 2017 1:18 PM, "Dylan Trotter" ***@***.***> ha scritto: > If, for example, you define print() in __builtins__.py then you have the > problem of accessing the *builtins* Go package from within the grumpy > runtime package, because they each would depend on each other. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#290 (comment)>, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/AAJf5ywht3Zc_SoCZu71nbK2xoVoBm7Dks5ry3m_gaJpZM4NFOm8 > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#290 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHLexHc0dg-_mbK7ycf_BujVrHN6Psdks5ry58EgaJpZM4NFOm8> .

[alanjds]

I just found the Print() function, and it should not rely on 'file' arg being a File anyway, otherwise "print" will never accept a duck-typed file-like, in my opinion.

For now, my plan is to factor it out from builtins, even if it is always loaded

Will draw some sketches on dead-trees to help with the circular dependencies. Maybe lazy-importing sys.Stdout, with a local cache to not penalize the most used case. I like it not, however.

[trotterdylan]

+1 to taking any object instead of a file. Looking forward to seeing what you come up with.

…

On Sun, Apr 23, 2017 at 3:20 PM Alan Justino da Silva < ***@***.***> wrote: I just found the Print() function <https://github.com/google/grumpy/blob/91ffc9fd18ee51cce79d2930e6850e284c3cd721/runtime/core.go#L1261>, and it should not rely on 'file' arg being a File anyway, otherwise "print" will never accept a duck-typed file-like, in my opinion. For now, my plan is to factor it out from builtins, even if it is always loaded Will draw some sketches on dead-trees to help with the circular dependencies. Maybe lazy-importing sys.Stdout, with a local cache to not penalize the most used case. I like it not, however. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#290 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHLe2DGzZa32iGJWegwGVKaV1agVe62ks5ry86YgaJpZM4NFOm8> .

[cclauss]

Are there a test cases for Python scripts that start with from future import print_function? That import makes the printer statement, the >> and the trailing comma all invalid. It also enables the end function parameter.

[alanjds]

@cclauss Yeah, but is a flag already handled by the compiler. I remember to saw some ifs in the compiler code about this. Print from "print statement" and from "print function" are different code paths, per file compiled, IIUC.

[alanjds]

@trotterdylan: Thinking in Print() check for 'sys' in SysModules and, if found, use 'sys.stdout'. If not found, fallback to Stdout as the 'sys' was not imported and sys.stdout is sure to be Stdout anyway.

[trotterdylan]

Although that solution should work (if sys was not imported, then there's no way for someone to override it from Python), it's not very satisfying IMO. It's pretty easy to imagine a scenario where this logic would diverge from CPython (probably involving messing around with sys.modules). Although these scenarios are far fetched, I'm more worried that this implementation divergence from CPython will cause other, more subtle problems. I think the right approach is to make the sys module "built in" in a similar way to CPython.

A variation on option 1. above is to do exactly what CPython does and define the sys module within the runtime itself: https://github.com/python/cpython/blob/2.7/Python/sysmodule.c

This would still require some changes to the import system to special case sys, but it's probably the best option we have available in the short term.

[corona10]

@trotterdylan I definitely agree with your idea. If we need modules of python which are implemented by C-level API. At that case, We should implement it by Go also.

[alanjds]

I do not know if I like this path.

I remember of news of some conference (could not find the reference, sorry) where people of different Python "runtimes" including CPython agreed that the best would be to write the stdlib in pure-python if possible, falling back to C-API only for serious speed needs or low-level access needs.

In the long term, do you expect to be easier to maintain a pure-python sys or a Go sys?
For now, the pure-python code have 73 LOC. The CPython one have 1801 LOC.
And I do not see a huge speed improvement on this path.

Another point is that, as we are borrowing PyPy and Ouroboros code, they could take it back and share the maintenance burden with us, but only if our modules are useful to him.

For now, my view of the thing is to implement whatever possible in pure-python (will be transpiled to Go anyway...), not worrying with implementation divergence from CPython. The result could be simpler and easier to maintain. And more useful to other pals too (PyPy for exemple).

[alanjds]

Be clear that it is my philosophical perspective, more aligned with PyPy "view". But a view like "lets transpile CPython code to Go, whatever it is a good code or not" will sure work too, and will easily emulate the goods and bads there. Including maintenance ones.

[alanjds]

Please confirm your spirits. I can wait or can give a shot, even if to change it later. just confirm.