Remove Python2 target.

[kaby76]

This PR fixes #4325.

This change removes the Python2 target from Antlr.

Github Actions removed support of testing of Python 2.x on June 20, 2023. Consequently, every PR (#4330, #4329, #4323, #4321, #4319, ...) fails because the workflow cannot install Python 2.x. In addition, official support from python.org for 2.x stopped several years ago. Keeping code around that we cannot develop or test is illogical. Old versions of Antlr4 exist for people to use, but going forward, it has to stop.

The instructions for the Python target have been updated. (NB: although the instructions specifically say to "not mix code and documentation updates in the sample pull request", it does not make sense to do that for this major PR.) References to Python2 are removed. I also took the liberty to update the content of the instructions itself, which were not very detailed. Minimum Python requirements were added. A functioning code example for a minimal parser driver was added. Functioning examples for Antlr tree traversal using a visitor and a listener, which evaluate an expression of an expression grammar, were added.

The instructions for the Python target had contained a section on target-agnostic grammars. This information is unrelated to Python target itself, but too important to delete. I moved the content to a new file, and rewrote the text in a more formal style.

[ericvergnaud]

typo ?

[kaby76]

Simplier is better. Actions are usually in a grammar to spport predicates. No need to single out actions.

[ericvergnaud]

'being' vs 'begin' is, I believe, a typo

[ericvergnaud]

I believe we're missing an announcement in the top-level read-me ?
Such as:
Gents, as of version 4.14, we are dropping support for Python 2. We love the Python community, but Python 2 support was officially halted in Jan 2020. More recently, GiHub also dropped support for Python 2, which has made it impossible for us to maintain a consistent level of quality across targets (we use GitHub for our CI). Long live Python 3!

[kaby76]

(Changing to "Draft" to resolve issues.)

[kaby76]

I believe we're missing an announcement in the top-level read-me

In /README.md?

There is a mention of the targets that are currently available here, a list of author/contributor with target name and specifically for Python here, and a link on the text "ANTLR code generation targets" here. Note, the title used in the link is not the same as the title of the document itself, which is "Runtime Libraries and Code Generation Targets". Standard industry and academic documentation should have the titles agree, letter for letter!

[ericvergnaud]

Yes, in /README.md
It's an announcement for this release, that we will remove in the following release.
The intent is that developers still addict to Python 2 would find this immediately when horrified by the lack of 4.14 version in PyPI and coming to antlr4 GH to file a bug.
@parrt will have to opine on this proposed communication channel

[kaby76]

• Agreed on the change to /index.md. I will add.
• Looks like "index.md" is linked from other websites to this repo, for example, https://www.antlr.org/about.html. I'll back out the file renaming ("README.md" => "index.md") because I won't be able to coordinate the change. I could use symbolic links, but people might get confused with that, and I am not sure it's supported on all OSes still.
• Is there a Markdown document for releases? I see this webpage in Github, but it is not in the repo's files per se.

[kaby76]

• The OSes support symbolic links, but Github displays the contents of index.md as a link, not the contents of "README.md". I will try a hard link next.

[kaby76]

• git does not handle hard links. The file is forked upon cloning in Linux. "index.md" has to be restored. Very annoying.

[ericvergnaud]

@parrt blessed

[kaby76]

Wordsmithed the heck out of the two .md files. Done.

[parrt]

Changed next release to 4.14.0, not 4.13.1. Thanks for the hard work cleaning this up @kaby76 !

[agatti]

This leaves people using Jython with the last available version being partially broken (see PR #4313). While CPython 2.7 is effectively dead, Jython is still quite alive (and currently stuck at 2.7, sadly).

Would it be possible to still have 4.13.1 as the last available version for Jython users (along with a Python wheel package for the Python2 runtime this time) and then get rid of the Python2 target?

[kaby76]

How do you test a 4.13.1 release? Or just release it untested?

[agatti]

If needed, I can take a look at how much work it would take to run the test suite under Jython.

I haven't looked at it yet, but I guess that besides an added maven dependency for https://mvnrepository.com/artifact/org.python/jython-standalone/2.7.3, script-wise it should be a matter of replacing python2 with a java invocation to run org.python.util.jython instead.

[ericvergnaud]

@agatti how about forking 4.13.0, apply the missing patch and publish an antlr4-jython-runtime ? Then you can gradually evolve it to follow python upgrades ?

[agatti]

@ericvergnaud This can work just for that release. Generated Python3 code has type annotations and it cannot be used as such.

If it is acceptable to modify the Python generator to use comment-based typing instead of the usual inline typing declarations then this may be an option.

[ericvergnaud]

@agatti sorry but we won't be maintaining a python 2 runtime going forward. People in need of antlr4 parsers generated for Python 2 will need to stick to antlr4 4.13.0 anyway.

[agatti]

@ericvergnaud I may have expressed myself the wrong way, apologies for that. My point was that if the Python3 generator can no longer output code like:

# This class defines a complete listener for a parse tree produced by CParser.
class CListener(ParseTreeListener):

    # Enter a parse tree produced by CParser#primaryExpression.
    def enterPrimaryExpression(self, ctx:CParser.PrimaryExpressionContext):
        pass
# ...

and would output code like this instead:

# This class defines a complete listener for a parse tree produced by CParser.
class CListener(ParseTreeListener):

    # Enter a parse tree produced by CParser#primaryExpression.
    def enterPrimaryExpression(self, ctx):
        # type: (CParser.PrimaryExpressionContext) -> None
        pass
# ...

it would still be PEP-0484 compliant and somebody else (I guess that would be me) would fork the runtime for Jython and bring it forward when needed. No Python2 runtime would have to be kept alive on your end.

[agatti]

Ok, there may be a couple more ImportError guards to be added when importing TextIO/StringIO but at a quick glance there shouldn't be much more work needed on the generator side.

[ericvergnaud]

Your proposed approach would prevent the Python 3 generator from using any Python 3 feature not available in Python 2.
Type hints is one of them, but there are others such as enums. Not sure that's acceptable...
Your best shot with Jython is to use an existing Python 3 -> Jython transformer, or to create one (using antlr and the existing Python 3 grammar, I suspect this would not be a massive task). That's until Jython ship their Python 3 version.
Another approach would be to switch to GraalPy, see https://www.graalvm.org/latest/reference-manual/python/Jython/
In theory, that would let you use the generated Python 3 files and Python 3 runtime.

[agatti]

@ericvergnaud Fair enough. I'll weigh my options and see what to do then. Thank you for your time.