SolvedPowerShell Do not require { ... } around variable names for null-conditional member access

Follow-up from #3240.

Null-conditional member access (?.) was implemented and will be available as an experimental feature in 7.0

However, in the interest of backward compatibility the implementation currently requires that the variable name be enclosed in {...}, because ? is technically a legal character in a variable name whose use surprisingly doesn't require {...}.

That is, if you want to ignore an attempt to call$obj.Method() if $obj is $null (or undefined), you would expect the following, analogous to C#:

# Does NOT work as intended.
# 'obj?' as a whole is interpreted as the variable name.
$obj?.Method()

Instead, you must currently use:

# !! Must enclose 'obj' in {...} to signal that '?' is a syntactic element as part of '?.'
${obj}?.Method()

Update: Null-coalescing - $var??='default and $a??$b - and the ternary operator - $var?1:0 - are similarly affected (though there the use of whitespace before the ? resolves the issue).

This requirement is (a) unexpected and (b) cumbersome:

  • New users will not expect the need for {...} and won't necessarily know that it is what is required when the following fails (possibly undetected, unless Set-StrictMode -Version 2 or higher is in effect): $o = [pscustomobject] @{ one = 1 }; $o?.one

  • Even once users do know about the need for {...}:

    • They will forget to use it on occasion, because of the counter-intuitive need for it.
    • When they do remember, this seemingly artificial requirement will be an ongoing source of frustration, especially since {...} is hard to type.

Use of {...} shouldn't be necessary, and while not requiring it amounts to a breaking change, it arguably falls into bucket 3: Unlikely Grey Area.


Why this change should be considered acceptable:

Note: @rjmholt discusses why changing PowerShell's syntax is highly problematic in general in this comment, but for the reasons presented below I think it is worth making an exception here.

?. is a new syntactic feature; by definition scripts that use it cannot (meaningfully) run on older versions - unless you specifically add conditionals that provide legacy-version-compatible code paths, but that seems hardly worth it - you would then just stick with the legacy features.

Yes, the interpretation of $var?.foo in old scripts that used $var? as a variable name would break, but:

  • ? should never have been allowed as part of an identifier without enclosing it in {...}.

  • Since being able to do so is unexpected and probably even unknown to many, such variable names are exceedingly rare in the wild, speaking from personal experience, but @mburszley's analysis provides more tangible evidence.

    • Even Ruby only allows ? (and !) at the end of identifiers, and there only of method identifiers, and I suspect that most Ruby users are aware that they should not assume that other languages support the same thing.

So, pragmatically speaking:

  • The vast majority of existing code will not be affected - a token such as $var?.foo will simply not be encountered.

  • If you write $var?.foo with the new semantics, then yes, running that on older versions could result in different behavior (rather than breaking in an obvious manner), depending on what strict mode is in effect - but you should always enforce the minimum version required to run your code as intended anyway (#requires -Version, module-manifest keys).

All in all, to me this a clear case of a bucket 3 change: a technically breaking change that breaks very little existing code while offering real benefits (or, conversely, avoiding perennial headaches due to unexpected behavior).

48 Answers

✔️Accepted Answer

@rjmholt, while I think we can all appreciate how tricky a change like this is in principle, in this particular case it doesn't matter in practice, and I don't think a PSSA rule is even needed:

  • @ThomasNieto's analysis has shown that - if we believe the corpus to be representative of all PowerShell code out there - that virtually no one uses variables with names that end in ?.

  • If I were to guess, most people wouldn't even think to use such variable names, because they wouldn't expect them to work (me included), and probably don't even notice the exception that the automatic $? variable constitutes (whose name is equally an exception in POSIX-compatible shells such as bash).

    • In fact, looking closer at @ThomasNieto's analysis reveals that, among the 8 files listed:
      • 5 contain only false positives, namely variables uses inside strings such as "Are you sure you want to kill $vParam?" - in fact, these uses are broken, and reveal that the script authors did not expect ? to be part of the variable name.
      • 2 of those files are no longer publicly available.
      • Only 1 of them is a bona fide use of ?-ending variables - as Boolean variables (e.g., $key1?), which, as an aside, are therefore not combined with the member-access operator, . /cc @stevenayers.

Based on the above, this strikes me as a textbook Bucket 3: Unlikely Grey Area change, which is therefore acceptable - and I think the benefits of this change have been argued persuasively.

Documenting the change in behavior (with a rationale) should suffice.

Of course:

  1. this is predicated on (a) the analysis being correct and (b) the publicly available corpus being representative.

  2. it is starkly at odds with the committee's claim that "there was quite a bit of usage of variable names ending with the question mark" (which, of course, would only be a problem if such variable names aren't enclosed in {...}).

Proceeding on the assumption that 1. is true (we haven't heard anything to support 2.), we have two options:

  • Rip off the band-aid and simply disallow ? in variable names going forward, unless enclosed in {...} (with the obvious exception of $?) - this will break the vanishingly small proportion of scripts that currently rely on that.

    • That is, something like $key1? that is neither followed by . nor another ? ($key1??'else') nor a ternary expression ($key1?1:0) would cause a parser error.
      • As the null-coalescing and ternary operator examples show, they too would benefit from this change - currently, you either need a space - $key1 ?1:0 / $key1 ??'else' - or {...} - ${key1}?1:0 / ${key1}??'else'
    • Inside expandable strings, ? would then no longer be considered part of a (non-{...}-enclosed) variable name, so we would actually fix existing scripts that have mistakenly used strings such as "Are you sure you want to kill $vParam?". 😁
  • If we really feel we need to avoid breaking the vanishingly small proportion of existing scripts, we could consider the logic proposed by @ExE-Boss, but I don't think we want to introduce this conceptual complexity - let alone implementation complexity (which I can't really speak to).

Other Answers:

If the consensus is to force ${...} syntax for this, support for OptionalFeatures would be appreciated here, so that folks like myself who will use this syntax (I'll be using it a lot) can opt out of the noise.

Since we're paraphrasing here:
"Logic clearly dictates that the needs of the many outweigh the needs of the few." - Spock

So, running the math, there are several concrete reasons not to do this, and only one vague reason to do this (it might be easier, for some users).

@StartAutomating: There are multiple reasons, and they are concrete, not vague. Also, one of the goals of this is code simplicity. Having to wrap variable names with {} works directly against that goal.

I'm not a serious PowerShell developer; I came here in search of a solution to a bug and happened to stumble upon this issue. For command line scripting, I mostly use bash. I'm probably in the demographic that Microsoft is trying to convince to switch to PowerShell Core: cross-platform developer in need of a quick scripting language that's less of a pain than bash.

I've often found many of the decisions related to PowerShell's syntax unintuitive to people who aren't primarily PowerShell developers. The last time I used PowerShell for something serious, I was appalled by the lack of a null coalescing operator, or even a ternary operator I could use instead--which resulted in me posting this popular PowerShell answer on Stack Overflow. That was in 2013, and it's easily one of the top three reasons I've mostly avoided PowerShell ever since. The whole point of a scripting language is that I want something quick and dirty with a lot of syntax sugar, not verbose if-else statements.

@StartAutomating said:

Deeply geeky folks aside, I'd expect that ${obj}?.Method() or $obj?.Method() will get little uptick. There's a decade of guidance out on the web of how to manage nulls out there, and I don't see that many people switching from if ($obj) { $obj.Method() } just to get on the v7 bandwagon

I'm just n=1, but as a non-PowerShell developer, I strongly disagree. The new ? syntax is something I would find out about pretty quickly even as someone who touches PowerShell maybe once every few months, and it would definitely increase the odds of me using PowerShell in the future in place of alternatives. If I then see that some crazy {} syntax is required for backwards compatibility, I'm going to roll my eyes and go back to whatever I was already using.

I don't understand the emphasis on backwards compatibility here. PowerShell Core already broke most of my existing scripts because they used parts of .NET that aren't in .NET Core. (I'm not complaining about that--.NET Core is great, but if you're going to break backwards compatibility, now's your chance.)

I'm with @mklement0 and @KirkMunro on this. Personally I find the ${...}? syntax unnatural and unnecessary complex. We have had breaking changes already and in this case it is small price to pay in exchange for clarity.

If the requirement for {..} is removed then the script will still parse but the semantics will be different, which in my opinion is very dangerous.

Aren't you also worried about folks using ?. the way they do in C/C++/C# and getting the wrong result with no indication they did anything wrong? For example:

PS> $res = "My length is 15"
PS> $res?.Length
0

Sure, strict-mode will find this but I don't know that a lot of folks use strict mode. So, to prevent a very small percentage of scripts from breaking, it seems we're setting up a lot of folks for failure using ?.. I'm not crazy about the trade-off. That said, I definitely understand the desire to avoid breaking folks even if it's a very small number of folks.

Maybe PowerShell needs an equivalent of Enable-ExperimentalFeature call it Enable-PSFeature, for features that are no longer in the experimental state but are not ready to be turned on by default. For scripts that execute with code like this $?.GetType() you could start emitting warnings that in the future this will break and that you should use ${?}.GetType() instead. Individual scripts could enable the feature with a #requires -version 7.1 -feature PSNullConditionalOperators. Then maybe whenever PS 8 hits, the feature could be enabled by default.

More Issues: