Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle python 3.6 f-strings without error #2622

Merged
merged 17 commits into from
Feb 21, 2017
Merged

Handle python 3.6 f-strings without error #2622

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
fc03182
Handle python 3.6 f-strings without error
Dec 30, 2016
31e0252
Fix to support older versions of typed_ast
Dec 30, 2016
6c4aa78
Fix f-string test
Dec 30, 2016
2112398
Add new f-string test to check expression type checking
Dec 30, 2016
025c948
Fix test flags
Dec 30, 2016
e8cb282
Split f-string tests and fix ast visit of JoinedStr
Dec 30, 2016
d6ba5fd
Use casts to fix typechecking of visitor method
Dec 30, 2016
25043d4
Remove casts and fix return type of visit_JoinedStr
Dec 31, 2016
8f17a4c
Add new f-string test case
Jan 1, 2017
4847e27
Add call to __str__() before concatenating strings in visit_JoinedStr
Jan 1, 2017
5534c1a
Fix tests by adding missing primitives
Jan 2, 2017
5567215
Add missing line numbers in newly constructed nodes
Jan 2, 2017
929c5ef
Add more tests on f-strings
Jan 3, 2017
2c73695
Merge remote-tracking branch 'upstream/master' into handle-f-string-w…
Feb 15, 2017
47d592a
fix: Rename asst35 to ast3
Feb 15, 2017
0867026
fix: Move f-string tests to check-newsyntax.test (executed only for >…
Feb 15, 2017
8a17249
Minor cleanings
Feb 17, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions mypy/fastparse.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -769,6 +769,22 @@ def visit_Str(self, n: ast35.Str) -> Union[UnicodeExpr, StrExpr]:
else:
return UnicodeExpr(n.s)

# Only available with typed_ast >= 0.6.2
if hasattr(ast35, 'JoinedStr'):
# JoinedStr(expr* values)
@with_line
def visit_JoinedStr(self, n: ast35.JoinedStr) -> StrExpr:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're also going to have to add JoinedStr and FormattedValue to the stubs for typed_ast.ast35 in typeshed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes thanks for the tip

Copy link
Contributor Author

@achauve achauve Dec 31, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done python/typeshed#803

result_string_expression = StrExpr('')
for value in n.values:
value_as_string_expr = cast(StrExpr, self.visit(value))
result_string_expression = cast(StrExpr, OpExpr('+', result_string_expression, value_as_string_expr))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if you could use asserts instead of casts, e.g.

value_as_string_expr = self.visit(value)
assert isinstance(value_as_string_expr, StrExpr)
r = OpExpr('+', result_string_expression, value_as_string_expr)
assert isinstance(r, StrExpr)
result_string_expression = r

That way if the type is ever not what you expect it'll fail right there rather than mysteriously failing at some later point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used cast because otherwise I thought it would systematically fail without a call to str(...). Am I missing something?

I thought another typesafe solution would be to replace casts by something like (untested) CallExpr(NameExpr('str'), OpExpr('+', result_string_expression, value_as_string_expr))? but it's rather complex compared to untyped casts, and at the end every expression in a f-string is indeed passed to a str() call?

return result_string_expression

# FormattedValue(expr value)
@with_line
def visit_FormattedValue(self, n: ast35.FormattedValue) -> Expression:
return self.visit(n.value)

# Bytes(bytes s)
@with_line
def visit_Bytes(self, n: ast35.Bytes) -> Union[BytesExpr, StrExpr]:
Expand Down
24 changes: 24 additions & 0 deletions test-data/unit/check-expressions.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1157,6 +1157,30 @@ b'%c' % (123)
[case testUnicodeInterpolation_python2]
u'%s' % (u'abc',)


-- F-String
-- --------


[case testFStringBasics]
# flags: --fast-parser --python-version 3.6
f'foobar'
f'{"foobar"}'
f'foo{"bar"}'
a: str
a = f'foobar'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need # flags: --fast-parser here (see how other tests use it) else you get a SyntaxError.

Can you also add tests showing that the expressions in {} are actually type-checked?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PS. To quickly run just this test, try: pytest -n0 -k testFStringParseOk

Copy link
Contributor Author

@achauve achauve Dec 30, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tip, better than python runtests.py testcheck that I was using.

I've just added another test to check type checking of expressions but I can't get it working.
If I put the content in a test_f_string.py file and run mypy --fast-parser --python-version 3.6 --show-traceback test_f_strings.py it runs without any error.

However running the test with pytest -n0 -k testFStringParseOk gives me:

___________________________________________________ testFStringTypecheckExpression ____________________________________________________
data: /Users/achauve/dev/github.com/achauve/mypy/test-data/unit/check-expressions.test:1169:
../../mypy/test/testcheck.py:113: in run_case
    self.run_case_once(testcase)
../../mypy/test/testcheck.py:190: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
../../mypy/test/helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (/Users/achauve/dev/github.com/achauve/mypy/test-data/unit/check-expressions.test, line 1169)
-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
Expected:
Actual:
  main:3: error: Variable annotation syntax is only suppoted in Python 3.6, use type comment instead (diff)

It doesn't use python 3.6 syntax. How can I specify it in the tests?

a = f'{"foobar"}'

[case testFStringExpressions]
# flags: --fast-parser --python-version 3.6
f'{1 + ""}'
f' {1 + ""}'
[out]
main:2: error: Unsupported operand types for + ("int" and "str")
main:3: error: Unsupported operand types for + ("int" and "str")
main:3: error: Unsupported operand types for + ("str" and "int")


-- Lambdas
-- -------

Expand Down