PEP 572 (Assignment Expressions in python 3.8)

PEP 572 - Assignment Expressions

• annotation
• Justification
• Syntax and semantics
• Specification changes during implementation
• Examples
• Rejected Alternatives
• Frequent objections
• Style Recommendations
• Thanks
• Appendix A: Tim Peters Conclusions
• Appendix B: A Rough Code Interpreter for Generators
• Appendix C: No Changes in Scope Semantics
• References
• Copyright
• My part

annotation

Justification

The Importance of Real Code

match = re.match(data)
group = match.group(1) if match else None

group = re.match(data).group(1) if re.match(data) else None

match1 = pattern1.match(data)
match2 = pattern2.match(data)
if match1:
    result = match1.group(1)
elif match2:
    result = match2.group(2)
else:
    result = None

match1 = pattern1.match(data)
if match1:
    result = match1.group(1)
else:
    match2 = pattern2.match(data)
    if match2:
        result = match2.group(2)
    else:
        result = None

Syntax and semantics

# Handle a matched regex
if (match := pattern.search(data)) is not None:
    # Do something with match

# A loop that can't be trivially rewritten using 2-arg iter()
while chunk := file.read(8192):
   process(chunk)

# Reuse a value that's expensive to compute
[y := f(x), y**2, y**3]

# Share a subexpression between a comprehension filter clause and its output
filtered_data = [y for x in data if (y := f(x)) is not None]

Exceptional Cases

• Assignment expressions not enclosed in parentheses are prohibited at the “upper” level:
  
  

  y := f(x)  # 
  (y := f(x))  # ,   

  
  This rule will make it easier for the programmer to choose between an assignment operator and an assignment expression - there will be no syntactic situation in which both options are equivalent.
• . :
  
  

  y0 = y1 := f(x)  # 
  y0 = (y1 := f(x))  # ,   

  
  . :
  
  

  foo(x = y := f(x))  # 
  foo(x=(y := f(x)))  # ,     

  
  , .
• . :
  
  

  def foo(answer = p := 42):  # 
      ...
  def foo(answer=(p := 42)):  # Valid, though not great style
      ...

  
  , (. , «» ).
• , . :
  
  

  def foo(answer: p := 42 = 5):  # 
      ...
  def foo(answer: (p := 42) = 5):  # ,  
      ...

  
  : , "=" ":=" .
• -. :
  
  

  (lambda: x := 1) # 
  lambda: (x := 1) # ,  
  (x := lambda: 1) # 
  lambda line: (m := re.match(pattern, line)) and m.group(1) # Valid

  
  - , ":=". . , , () , .
• f- . :
  
  

  >>> f'{(x:=10)}'  # ,  
  '10'
  >>> x = 10
  >>> f'{x:=10}'    # ,  ,  '=10'
  '        10'

  
  , , f-, . f- ":" . , f- . , .

if any((comment := line).startswith('#') for line in lines):
    print("First comment:", comment)
else:
    print("There are no comments")

if all((nonblank := line).strip() == '' for line in lines):
    print("All lines are blank")
else:
    print("First non-blank line:", nonblank)

# Compute partial sums in a list comprehension
total = 0
partial_sums = [total := total + v for v in values]
print("Total:", total)

[[(j := j) for i in range(5)] for j in range(5)] # 
[i := 0 for i, j in stuff]                       # 
[i+1 for i in (i := stuff)]                      # 

[False and (i := 0) for i, j in stuff]     # 
[i for i, j in stuff if True or (j := 1)]  # 

# [.  . - ""   
# ,       
# ,    ,   ]

[i+1 for i in (j := stuff)]                    # 
[i+1 for i in range(2) for j in (k := stuff)]  # 
[i+1 for i in [j for j in (k := stuff)]]       # 
[i+1 for i in (lambda: (j := stuff))()]        # 

class Example:
    [(j := i) for i in range(5)]  # 

Relative Priority: =

x := 0 # 

(x := 0) #  

x = y := 0 # 

x = (y := 0) #  

len(lines := f.readlines()) # 

foo(x := 3, cat='vector') # 

foo(cat=category := 'vector') # 

foo(cat=(category := 'vector')) #  

# Valid
if any(len(longline := line) >= 100 for line in lines):
    print("Extremely long line:", longline)

Change the order of calculations.

• In dictionary generators {X: Y for ...}, Y is currently evaluated before X. We suggest changing this so that X is calculated before Y. (In a classic dict such as {X: Y}, as well as in dict ((X, Y) for ...) this has already been implemented. Therefore, dictionary generators must comply with this mechanism)

Differences between assignment expressions and assignment instructions.

• Cascading assignment is not supported directly
  
  

  x = y = z = 0  # Equivalent: (z := (y := (x := 0)))

• Separate "targets", except for the simple variable name NAME, are not supported:
  
  

  # No equivalent
  a[i] = x
  self.rest = []

• Functionality and priority “around” commas differs:
  
  

  x = 1, 2  # Sets x to (1, 2)
  (x := 1, 2)  # Sets x to 1

• Unpacking and packing values ​​do not have “pure” equivalence or are not supported at all
  
  

  # Equivalent needs extra parentheses
  loc = x, y  # Use (loc := (x, y))
  info = name, phone, *rest  # Use (info := (name, phone, *rest))
  
  # No equivalent
  px, py, pz = position
  name, phone, email, *other_info = contact

• Inline type annotations are not supported:
  
  

  # Closest equivalent is "p: Optional[int]" as a separate declaration
  p: Optional[int] = None

• There is no shortened form of operations:
  
  

  total += tax  # Equivalent: (total := total + tax)

Specification changes during implementation

• To ensure consistency with other similar exceptions, and not to introduce a new name that may not be convenient for end users, the originally proposed subclass of TargetScopeError for SyntaxError has been removed and reduced to the usual SyntaxError. [3]
• Due to limitations in parsing the CPython character table, the reference implementation of the assignment expression raises a SyntaxError for all uses within iterators. Previously, this exception occurred only if the name of the variable being created coincided with that already used in the iterative expression. This can be revised if there are sufficiently convincing examples, but the additional complexity seems inappropriate for purely “hypothetical” use cases.

Examples

Python Standard Library Examples

site.py

• Current Code:
  

  env_base = os.environ.get("PYTHONUSERBASE", None)
  if env_base:
      return env_base

• Improved code:
  

  if env_base := os.environ.get("PYTHONUSERBASE", None):
      return env_base

_pydecimal.py

• Current Code:
  

  if self._is_special:
      ans = self._check_nans(context=context)
      if ans:
          return ans

• Improved code:
  

  if self._is_special and (ans := self._check_nans(context=context)):
      return ans

copy.py

• Current Code:
  

  reductor = dispatch_table.get(cls)
  if reductor:
      rv = reductor(x)
  else:
      reductor = getattr(x, "__reduce_ex__", None)
      if reductor:
          rv = reductor(4)
      else:
          reductor = getattr(x, "__reduce__", None)
          if reductor:
              rv = reductor()
          else:
              raise Error(
                  "un(deep)copyable object of type %s" % cls)

• Improved code:
  
  

  if reductor := dispatch_table.get(cls):
      rv = reductor(x)
  elif reductor := getattr(x, "__reduce_ex__", None):
      rv = reductor(4)
  elif reductor := getattr(x, "__reduce__", None):
      rv = reductor()
  else:
      raise Error("un(deep)copyable object of type %s" % cls)

datetime.py

• Current Code:
  
  

  s = _format_time(self._hour, self._minute,
                   self._second, self._microsecond,
                   timespec)
  tz = self._tzstr()
  if tz:
      s += tz
  return s

• Improved code:
  
  

  s = _format_time(self._hour, self._minute,
                   self._second, self._microsecond,
                   timespec)
  if tz := self._tzstr():
      s += tz
  return s

sysconfig.py

• Current Code:
  
  

  while True:
      line = fp.readline()
      if not line:
          break
      m = define_rx.match(line)
      if m:
          n, v = m.group(1, 2)
          try:
              v = int(v)
          except ValueError:
              pass
          vars[n] = v
      else:
          m = undef_rx.match(line)
          if m:
              vars[m.group(1)] = 0

• Improved code:
  
  

  while line := fp.readline():
      if m := define_rx.match(line):
          n, v = m.group(1, 2)
          try:
              v = int(v)
          except ValueError:
              pass
          vars[n] = v
      elif m := undef_rx.match(line):
          vars[m.group(1)] = 0

Simplify List Generators

results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]

stuff = [[y := f(x), x/y] for x in range(5)]

Capture Values ​​in Conditions

# Loop-and-a-half
while (command := input("> ")) != "quit":
    print("You entered:", command)

# Capturing regular expression match objects
# See, for instance, Lib/pydoc.py, which uses a multiline spelling
# of this effect
if match := re.search(pat, text):
    print("Found:", match.group(0))
# The same syntax chains nicely into 'elif' statements, unlike the
# equivalent using assignment statements.
elif match := re.search(otherpat, text):
    print("Alternate found:", match.group(0))
elif match := re.search(third, text):
    print("Fallback found:", match.group(0))

# Reading socket data until an empty string is returned
while data := sock.recv(8192):
    print("Received data:", data)

Fork

if pid := os.fork():
    # Parent code
else:
    # Child code

Rejected Alternatives

Changing the scope for generators

Alternative spellings

1. EXPR as NAME:
   
   

   stuff = [[f(x) as y, x/y] for x in range(5)]

   
   EXPR as NAME import, except with, (, ).
   
   ( , «with EXPR as VAR» EXPR VAR, EXPR.__enter__() VAR.)
   
   , ":=" :
   
   • , if f(x) as y , ​​ if f x blah-blah, if f(x) and y.
   • , as , , :
     • import foo as bar
     • except Exc as var
     • with ctxmgr() as var
     
     , as if while , as « » .
   • «»
     • NAME = EXPR
     • if NAME := EXPR
     
     .
2. EXPR -> NAME
   
   

   stuff = [[f(x) -> y, x/y] for x in range(5)]

   
   , R Haskell, . ( , - y < — f (x) Python, - .) «as» , import, except with, . Python ( ), ":=" ( Algol-58) .
3. «»
   
   

   stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as"
   stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":="

   
   . Python, , .
4. where: :
   
   

   value = x**2 + 2*x where:
       x = spam(1, 4, 7, q)

   
   ( , «»). , «» ( with:). . PEP 3150, ( given: ).
5. TARGET from EXPR:
   
   

   stuff = [[y from f(x), x/y] for x in range(5)]

   
   This syntax is less likely to conflict with others than as (unless you count the raise Exc from Exc constructs), but otherwise be comparable to them. Instead of a parallel with with expr as target: (which may be useful, but it can also be confusing), this option has no parallels with anything at all, but it is surprisingly better remembered.

Special cases in conditional statements

if re.search(pat, text) as match:
    print("Found:", match.group(0))

Special cases in generators

1. where, let, or given:
   
   

   stuff = [(y, x/y) where y = f(x) for x in range(5)]
   stuff = [(y, x/y) let y = f(x) for x in range(5)]
   stuff = [(y, x/y) given y = f(x) for x in range(5)]

   
   This method results in a subexpression between the for loop and the main expression. It also introduces an additional language keyword, which can create conflicts. Of the three options, where is the cleanest and most readable, but potential conflicts still exist (for example, SQLAlchemy and numpy have their where methods, as well as tkinter.dnd.Icon in the standard library).
2. with NAME = EXPR:
   
   

   stuff = [(y, x/y) with y = f(x) for x in range(5)]

   
   , , with. . , «» for. C, , . : « «with NAME = EXPR:» , ?»
3. with EXPR as NAME:
   
   

   stuff = [(y, x/y) with f(x) as y for x in range(5)]

   
   , as, . , for. with

Lower operator priority

pos = -1
while pos := buffer.find(search_term, pos + 1) >= 0:
    ...

You give commas on the right

(point := (x, y))
(point := x, y)

foo (x: = 1, y)
foo (x: = (1, y))

Always requiring brackets

# Top level in if
if match := pattern.match(line):
    return match.group(1)

# Short call
len(lines := f.readlines())

Frequent objections

Why not just turn the assignment statements into expressions?

Why bother with assignment expressions if assignment instructions exist ?

Why not use local scope and prevent namespace pollution?

Style Recommendations

1. If you can use both assignment options, then give preference to operators. They most clearly express your intentions.
2. If the use of assignment expressions leads to ambiguity in the execution order, then rewrite the code using the classical operator.

Thanks

Appendix A: Tim Peters Conclusions

i = j = count = nerrors = 0

i = j = 0
count = 0
nerrors = 0

mylast = mylast[1]
yield mylast[0]

yield (mylast := mylast[1])[0]

while True:
    old = total
    total += term
    if old == total:
        return total
    term *= mx2 / (i*(i+1))
    i += 2

while total != (total := total + term):
    term *= mx2 / (i*(i+1))
    i += 2
return total

result = solution(xs, n)
if result:
    # use result

if result := solution(xs, n):
    # use result

reductor = dispatch_table.get(cls)
if reductor:
    rv = reductor(x)
else:
    reductor = getattr(x, "__reduce_ex__", None)
    if reductor:
        rv = reductor(4)
    else:
        reductor = getattr(x, "__reduce__", None)
        if reductor:
            rv = reductor()
        else:
            raise Error("un(shallow)copyable object of type %s" % cls)

if reductor := dispatch_table.get(cls):
    rv = reductor(x)
elif reductor := getattr(x, "__reduce_ex__", None):
    rv = reductor(4)
elif reductor := getattr(x, "__reduce__", None):
    rv = reductor()
else:
    raise Error("un(shallow)copyable object of type %s" % cls)

diff = x - x_base
if diff:
    g = gcd(diff, n)
    if g > 1:
        return g

if (diff := x - x_base) and (g := gcd(diff, n)) > 1:
    return g

Numerical example

while a > (d := x // a**(n-1)):
    a = ((n-1)*a + d) // n
return a

while True:
    d = x // a**(n-1)
    if a <= d:
        break
    a = ((n-1)*a + d) // n
return a

Appendix B: A Rough Code Interpreter for Generators

• Source code (EXPR most often uses the VAR variable):
  
  

  def f():
      a = [EXPR for VAR in ITERABLE]

• The converted code (let's not worry about name conflicts):
  
  

  def f():
      def genexpr(iterator):
          for VAR in iterator:
              yield EXPR
      a = list(genexpr(iter(ITERABLE)))

• Source:
  
  

  def f():
      a = [TARGET := EXPR for VAR in ITERABLE]
  

• Converted Code:
  
  

  def f():
      if False:
          TARGET = None  # Dead code to ensure TARGET is a local variable
      def genexpr(iterator):
          nonlocal TARGET
          for VAR in iterator:
              TARGET = EXPR
              yield TARGET
      a = list(genexpr(iter(ITERABLE)))

• Source:
  
  

  def f():
      global TARGET
      a = [TARGET := EXPR for VAR in ITERABLE]
  

• Converted Code:
  
  

  def f():
      global TARGET
      def genexpr(iterator):
          global TARGET
          for VAR in iterator:
              TARGET = EXPR
              yield TARGET
      a = list(genexpr(iter(ITERABLE)))

• Source:
  
  

  def g():
      TARGET = ...
      def f():
          nonlocal TARGET
          a = [TARGET := EXPR for VAR in ITERABLE]
  

• Converted Code:
  
  

  def g():
      TARGET = ...
      def f():
          nonlocal TARGET
          def genexpr(iterator):
              nonlocal TARGET
              for VAR in iterator:
                  TARGET = EXPR
                  yield TARGET
          a = list(genexpr(iter(ITERABLE)))

• Source:
  
  

  def f():
      a = [[TARGET := i for i in range(3)] for j in range(2)]
      # I.e., a = [[0, 1, 2], [0, 1, 2]]
      print(TARGET)  # prints 2
  

• Converted Code:
  
  

  def f():
      if False:
          TARGET = None
      def outer_genexpr(outer_iterator):
          nonlocal TARGET
          def inner_generator(inner_iterator):
              nonlocal TARGET
              for i in inner_iterator:
                  TARGET = i
                  yield i
          for j in outer_iterator:
              yield list(inner_generator(range(3)))
      a = list(outer_genexpr(range(2)))
      print(TARGET)

Appendix C: No Changes in Scope Semantics

a = 42
def f():
    # `a` is local to `f`, but remains unbound
    # until the caller executes this genexp:
    yield ((a := i) for i in range(3))
    yield lambda: a + 100
    print("done")
    try:
        print(f"`a` is bound to {a}")
        assert False
    except UnboundLocalError:
        print("`a` is not yet bound")

>>> results = list(f()) # [genexp, lambda]
done
`a` is not yet bound
# The execution frame for f no longer exists in CPython,
# but f's locals live so long as they can still be referenced.
>>> list(map(type, results))
[<class 'generator'>, <class 'function'>]
>>> list(results[0])
[0, 1, 2]
>>> results[1]()
102
>>> a
42

References

1. Proof of concept implementation
2. Discussion of the semantics of assignment expressions (VPN is tight but loaded)
3. Discussion of TargetScopeError in PEP 572 (loaded similarly to the previous one)

Copyright

My part

• So that people don’t try to remove semantic duality, in many “classical” places where one could use both “=” and “: =” there are restrictions, therefore the operator:: = should be often enclosed in brackets. These cases will have to be reviewed in the section describing the basic use.
• The priority of assignment expressions is slightly higher than that of a comma. Due to this, tuples are not formed during assignment. It also makes it possible to use the: = operator when passing arguments to a function.
• , , , . . lambda , «» .
• : ,
• , .
• / .
• , .