COMMITS
/ Python/Python-tokenize.c December 11, 2023
P
gh-112943: Correctly compute end offsets for multiline tokens in the tokenize module (#112949)
Pablo Galindo Salgado committed
November 2, 2023
S
Remove unnecessary includes (GH-111633)
Serhiy Storchaka committed
October 11, 2023
L
gh-104169: Refactor tokenizer into lexer and wrappers (#110684)
Lysandros Nikolaou committed
July 26, 2023
P
gh-107015: Remove async_hacks from the tokenizer (#107018)
Pablo Galindo Salgado committed
June 9, 2023
P
gh-105564: Don't include artificial newlines in the line attribute of tokens (#105565)
Pablo Galindo Salgado committed
June 7, 2023
K
gh-105390: Add explicit type cast (#105466)
Kirill Podoprigora committed
P
gh-105435: Fix spurious NEWLINE token if file ends with comment without a newline (#105442)
Pablo Galindo Salgado committed
P
gh-105390: Correctly raise TokenError instead of SyntaxError for tokenize errors (#105399)
Pablo Galindo Salgado committed
June 6, 2023
P
gh-105259: Ensure we don't show newline characters for trailing NEWLINE tokens (#105364)
Pablo Galindo Salgado committed
May 30, 2023
L
gh-105042: Disable unmatched parens syntax error in python tokenize (#105061)
Lysandros Nikolaou committed
P
gh-105069: Add a readline-like callable to the tokenizer to consume input iteratively (#105070)
Pablo Galindo Salgado committed
May 28, 2023
M
gh-105017: Include CRLF lines in strings and column numbers (#105030)
Marta Gómez Macías committed
May 26, 2023
P
gh-104976: Ensure trailing dedent tokens are emitted as the previous tokenizer (#104980)
Pablo Galindo Salgado committed
P
gh-104972: Ensure that line attributes in tokens in the tokenize module are correct (#104975)
Pablo Galindo Salgado committed
May 24, 2023
P
gh-104825: Remove implicit newline in the line attribute in tokens emitted in the tokenize module (#104846)
Pablo Galindo Salgado committed
May 22, 2023
M
gh-104741: Add line number attribute to indentation error exception (#104743)
Marta Gómez Macías committed
M
gh-102856: Tokenize performance improvement (#104731)
Marta Gómez Macías committed
May 21, 2023
M
gh-102856: Python tokenizer implementation for PEP 701 (#104323)
Marta Gómez Macías committed
May 5, 2023
E
gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)
Eric Snow committed
April 19, 2023
P
gh-102856: Initial implementation of PEP 701 (#102855)
Pablo Galindo Salgado committed
October 6, 2022
L
gh-97973: Return all necessary information from the tokenizer (GH-97984)
Lysandros Nikolaou committed
August 11, 2022
February 11, 2022
P
bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081)
Petr Viktorin committed
October 13, 2021
V
bpo-45434: Mark the PyTokenizer C API as private (GH-28924)
Victor Stinner committed
October 3, 2021
S
Remove trailing spaces. (GH-28706)
Serhiy Storchaka committed
August 25, 2021
P
Format the Python-tokenize module and fix exit path (GH-27935)
Pablo Galindo Salgado committed
August 24, 2021
P
Add tests for the C tokenizer and expose it as a private module (GH-27924)
Pablo Galindo Salgado committed