let and const declarations shouldn't hoist

[waldemar]

function f() {
  let const x = 2;
  {
    let y = x;
    if (false)
      let const x = 3;
    return y;
  }
}

Following the comments in #240, I hadn't realized until now that let reintroduced the idea of hoisting that it was meant to banish. This seems problematic and counterintuitive, leading to forward references to things that haven't been defined yet and might not ever get defined. I can't tell what the above program is meant to return.

[brendan]

The answer is undefined, of course :-P.

The intention of let declarations is to provide a three-letter upgrade to var decls, no change in hoisting-ness (but block scope, so hoisting more locally).

If let is the new var, and let declarations hoist to top of enclosing block, then the only win is block scope. The hoisting oddness remains, which is propagation of evil in many eyes. On the other hand, your example is "easy" to understand if you replace let with var, let const with var (just for hoisting analysis), and blocks with immediately applied function expressions whose return values propagate as needed.

I'm not defending hoisting, just recapitulating the argument that has won so far: var x hoists to its scope boundary, so should let x.

I have to agree that this is problematic, at least in the following sense. Firefox 2 shipped with the bug that let at top level translates to var. Here's the hard case:

<script>
alert(x);
// 60,000 lines of code here:
. . .
let x = 42;
</script>

Without another compiler pass, we were not able to revise the early bytecode selection made when x was seen.

Ok, you say, this is "just a SpiderMonkey? bug", fixable. But it reflects the counterintuitiveness you cite, I think. Who would know, with 60,000 lines of code in the way, that that early x refers to the very late let binding? But I'm now arguing the other way, since our position was that "anyone who knows var hoists would know, that's who!"

Cc'ing more folks, including Dave who was in on early let proposal editing.

/be

[brendan]

If let declarations should not hoist, then let blocks become superfluous, as #240 proposed; let expressions still have use-cases unmet by let declarations.

/be

[chrispi]

Why would let blocks become superfluous? With a let block, you have a place where the scope ends (not so with let decls), and you can shadow outer variables within the block... otherwise, people are back to using the define-and-immediately-call-a-closure-trick.

[brendan]

The lack of hoisting means let (x = x) {...} can be rewritten { let x = x; ... } with no extra block, and no parentheses. Either way, you have to close the block to say where the scope ends.

/be

[lth]

First point: I think most of the world would be surprised if the example code even compiles. Unless I am mistaken, "let" directives (of any flavor) can only appear at the top level of an explicit block, ie, inside "{" ... "}". This means that there is never any question about where a name is first defined and whether it may be initialized at all; every let directive dominates the end of the block.

Second point: The scope of the name introduced by a let directive needs to be the entire block (IMO), but I would consider it very desirable in strict mode to flag use-before-initialization, and even lightweight implementations could do this in standard mode if they wanted, it's a read barrier if they don't want to do the analysis. But the analysis is trivial (by my first point).

[waldemar]

lth: Why do you say that the scope of the name introduced by a let directive must retroactively include statements before the let? This would be quite surprising to anyone familiar with C++ or Java (they do not retroactively include statements in the scope), and experience shows that these languages made the correct decision here.

[brendan]

Replying to lth:

First point: I think most of the world would be surprised if the example code even compiles.

SpiderMonkey? (Firefox 2, 3):

js> function f(x){if (x) let y = 42; return y}
js> dis(f)
main:
00000:  enterblock depth 0 {y: 0}
00003:  getarg 0
00006:  ifeq 15 (9)
00009:  int8 42
00011:  setlocal 0
00014:  pop
00015:  getlocal 0
00018:  return
00019:  leaveblock 1
00022:  stop
Source notes:
  0:     6 [   6] if
  1:    11 [   5] decl     offset 2
js> f
function f(x) {
    if (x)
        let y = 42;
    return y;
}

ES4 RI rejects (diagnostic could be better):

>> function f(x){if (x) let y = 42; return y}
**ERROR** ParseError: unknown token in letExpression (near <no filename>:1:22-1.24)

This is a relief, but it may need to be specified better. In particular, var as an unbraced declaration controlled by if works just fine in ES1-3, in particular in SpiderMonkey?:

js> function f(x){if (x) var y = 42; return y}
js> dis(f)
main:
00000:  getarg 0
00003:  ifeq 12 (9)
00006:  int8 42
00008:  setvar 0
00011:  pop
00012:  getvar 0
00015:  return
00016:  stop
Source notes:
  0:     3 [   3] if
  1:     8 [   5] decl     offset 0
js> f
function f(x) {
    if (x) {
        var y = 42;
    }
    return y;
}

(Note how the decompiler must avoid bracing the let-declaration-as-then-clause, but need not and does not in the var case!)

The RI is not backward-compatible at the moment:

>> function g(x){if (x) var y = 42; return y}
**ERROR** ParseError: expecting 'identifier' before 'var' (near <no filename>:1:22-1.24)

Unless I am mistaken, "let" directives (of any flavor) can only appear at the top level of an explicit block, ie, inside "{" ... "}". This means that there is never any question about where a name is first defined and whether it may be initialized at all; every let directive dominates the end of the block.

This is a relief to hear. We missed it when implementing JS1.7 based on the let proposal last year, because it is different from the var case -- so "let is the new var (except that a let declaration must occur as a direct child of a block)."

Second point: The scope of the name introduced by a let directive needs to be the entire block (IMO),

This is a bone of contention. We agreed to this, and I'm still in favor because var hoists and let is the new var, but that rule is not quite as simple as we thought, as this discussion reveals.

but I would consider it very desirable in strict mode to flag use-before-initialization, and even lightweight implementations could do this in standard mode if they wanted, it's a read barrier if they don't want to do the analysis. But the analysis is trivial (by my first point).

Strict mode is optional to implementations, but normatively specified, so we need to specify more. IIRC Pascal raised the question of more analysis for strict mode with Cormac recently. This should go on the agenda for the weekly and face-to-face meetings.

/be

[jeffdyer]

Based on private discussion between Lars, Waldemar, Pascal, Brendan and Jeff, the following was agreed upon:

• let bindings hoisted the the nearest block scope; a new block scope is not introduced at the let declaration
• forward references to let vars are errors (see unresolved items below)

Unresolved are:

• whether or not forward reference errors are runtime or compile time errors
• are forward references in closures special, to allow mutual recursion

[lth]

Specifically, are forward references from closures to later defined let variables in the enclosing scope legal or illegal? Here are some of Waldemar's examples:

{
   let x = f()                    // OK with run-time error here
   let function f() { return w }  // compile-time error here
   let w = 20
}

Lars thinks that the reference to f should never fail because function initialization should be hoisted to the top of the block (by analogy with how "function" works in ES3) and w should be a run-time error at most, if w has not been initialized when f is called.

[lth]

The issue about

  if (x)
    let y = 10

being illegal is written up in #280.

[waldemar]

Replying to lth:

{{{ { let x = f() // OK with run-time error here let function f() { return w } // compile-time error here let w = 20 } }}}

In this example the forward-reference on the first line to f would be ok if we wanted to allow mutually recursive locally defined functions. The early reference to w should be a compile-time error -- other than functions, such forward references lead to trouble and will make things either unnecessarily complicated or impossible if we want to make types more dynamic in the future.

[ibukanov]

Replying to jeffdyer:

* let bindings hoisted the the nearest block scope; a new block scope is not introduced at the let declaration

What about let statements at the top-level in a script? If they would not add a new block scope, would they overwrite the var declarations? I.e. in a browser context should the alert show 1 or 2 in the example below?

<script>var x = 1;</script>
<script>let x = 2;</script>
<script>alert(x);</script>

[ibukanov]

Replying to jeffdyer:

Unresolved are: * whether or not forward reference errors are runtime or compile time errors * are forward references in closures special, to allow mutual recursion

Here is an example of how mutual recursion looks:

  let function f(n) { return n == 0 ? x : g(n); }
  let function g(n) { return f(n - 1); }
  let x = 1;

If no forward references from closures are allowed, the example will look like:

  let f, g, x;
  f = function(n) { return n == 0 ? x : g(n); }
  g = function(n) { return f(n - 1); }
  x = 1;

IMO it does not look bad with the explicit forward declaration serving as a hint about cross-references between f and g.

[ibukanov]

To clarify: is the following is an example of a forward reference from a closure to the let-declared name f?

let f = function(n) { n == 0 ? 1 : n * f(); }
...

I.e. should the semantic of

let a = b; ...

be equivalent to

let a; a = b; ...

or to that of a let block:

let (a = b) { ... }

?

[brendan]

Replying to ibukanov:

Replying to jeffdyer:

* let bindings hoisted the the nearest block scope; a new block scope is not introduced at the let declaration

What about let statements at the top-level in a script?

There is an implicit block around top-level code, as around function body, class init, and package init code.

/be

[David-Sarah Hopwood]

Changing version to ES-Harmony. I think there is general concensus that let (and const) shouldn't hoist, but when use-before-initialisation errors should be detected is still unresolved.

[brendan]

The consensus is clear, but it seems to me declarations do hoist to start of nearest block (not to start of program or function body as with var). It's an error (details to be specified, as you note) to use the binding before it has been initialized by control flow reaching its declaration site and evaluating the initializer, but one can't use an outer binding in this dead zone.

So "hoisting" still seems accurate -- just to start of nearest block. What other term might be used? Shadowing? Dead-zoning? :-)

/be

[brendan]

"foreshadowing" ;-)

/be