[SUGGESTION] for loop is ambiguous with lambda syntax

Preface

Cpp2 has the following control structures:

if CONDITION { ... }
else if CONDITION { ... }
...
else { ... }

while CONDITION next STEP { ... }

do { ... }
while CONDITION next STEP;

for ITEMS do: (ITEM) = { ... }

While I always thought that for loop gets a lambda, and because of that I started to think that Cpp2 should have user-defined language constructs. But NO, hopefully the statement block of for loop is not a lambda.

All control structures of Cpp2 have statement blocks, therefore we don't need to capture anything inside the statement block of for loop.

How to reproduce?

It's an example:

#include <iostream>
#include <vector>
#include <string>

main: () = {
    items: std::vector<int> = (1, 2, 3);
    r: = 10;

    for items do: (item) = {
        r = 0;
        std::cout << item << "\n";
    }
}

A typical programmer would think that : (item) = { ... } is a lambda because of its similarity with lambda syntax, but hopefully it's not. Cpp2 generates the following code:

auto main() -> int{
    std::vector<int> items {1, 2, 3}; 
    auto r {10}; 

    for ( auto const& cpp2_range = items;  auto const& item : cpp2_range ) {
        r = 0;
        std::cout << item << "\n";
    }
}

Suggestion Detail

I didn't know I should create a bug report or a suggestion for ambiguous problem of for loop. By the way I created a suggestion.

I suggest to change the syntax of for loop from:

for ITEMS do: (ITEM) = {
    ...
}

to something consistent with other control structures such as one of the following syntax:

for ITEM keyword ITEMS {
    ...
}

for ITEMS keyword ITEM {

}

In which:

keyword for the first option can be in, of, iter or etc.
keyword for the second option can be next, each, loop or etc.

Any other syntax which is similar to other control structures, is good.

Why do I suggest this change?

Because:

The statement block of for loop syntax : (item) = { ... } is ambiguous with lambda syntax. A typical programmer expects to capture variables inside it.
The statement block of for loop syntax : (item) = { ... } is inconsistent with other control structures which their statement block are simply { ... }.

Your Questions

Will your feature suggestion eliminate X% of security vulnerabilities of a given kind in current C++ code?

No.

Will your feature suggestion automate or eliminate X% of current C++ guidance literature?

Yes absolutely. It would make control structures to be consistent with each other in addition to simplification of guidance:

{ ... } will be for the statement block of control structures, therefore variables don't have to be captured.
But : (args) = { ... } will be for lambdas, therefore variables have to be captured. for loop breaks this rule.
Expressions in control structures (e.g. the expression after next keyword in while loop) won't be evaluate at first evidence, and they can be evaluated multiple times.
But expressions in other places (e.g. for variable initialization or function arguments) will be evaluated one time at first evidence.

Considered alternatives

The other alternative solution is to make the statement block of for loop to be actually a lambda, which is weird and inconsistent with other control structures.

The syntax is supposed to make the language more regular, as a for loop's body is parametrized on the element being iterated.

I did find it a bit weird, but only because of the association with function expressions and the knowledge they're lowered to Cpp1 lambda expressions.

Otherwise, I think it's a win. The function syntax is regular. At namespace scope and type scope, they declare functions. At statement scope, they're currently banned. As expressions, they work and permit captures. And finally, they work as the for loop's parametrized statements' body (no scope introduced, and I suppose captures not permitted).

Otherwise, I think it's a win. The function syntax is regular. At namespace scope and type scope, they declare functions. At statement scope, they're currently banned. As expressions, they work and permit captures. And finally, they work as the for loop's parametrized statements' body (no scope introduced, and I suppose captures not permitted).

IMO that's why it complicates the language. It just feels like another static keyword in Cpp1 which has many meanings in different places.

Consider the upcoming template for. In Cpp2, that's a noop. cppfront would just have to permit <T> before :(item), and emit template before for in that case.

EDIT: It's https://wg21.link/P1306, for....

That's nice, but { ... } doesn't cause trouble for for... in generic programming:

func: <T: type...> (args: T...) = {
    for... arg: T in args {
        //: statements...
    }
}

or this one:

func: <T: type...> (args: T...) = {
    for... args next arg: T {
        //: statements...
    }
}

or any other syntax without : (arg) = { ... }.

Sorry, that was a bad example. "Expansion statements", proposed for Cpp1, operates on values. So it'd probably be a matter of permitting for... in Cpp2, too.

Thanks for the feedback. The status quo is by design, because the body of a range-for loop (not a do or while loop) is already conceptually a function invoked once per element in the range.

Note that capture is already not allowed in all functions. In particular, named functions already don't allow capture. Perhaps it helps if you view the body of a for loop as being a function named do?

Although it's not possible to have nested functions in Cpp2, but yes, it helps if I view it as a function named do.

BTW, Cpp2 will have local functions. They're pretty much there, I just ban them as a 'temporary alpha limitation' only because I haven't been motivated to prioritize the task of emitting them as a lambda and making sure they can capture. It shouldn't be hard, but I've wanted to drive other things like meta functions first. Also, they have a pretty easy workaround (as in Cpp1 today) which is in the error message you get if you try to write one:

(temporary alpha limitation) local functions like 'f: (/*params*/) = {/*body*/}' are not currently supported - write a local variable initialized with an unnamed function like 'f := :(/*params*/) = {/*body*/};' instead (add '=' and ';')

That said, I have said the same about other features and then ended up prioritizing and implementing them anyway already because they came up in issues... squeaky wheels do get grease, and programmers asking for a feature and showing concrete use cases is new information that justifies revisiting priority. So if you want cppfront to prioritize implementing nested functions, get friends to open issues and show use cases. 😁

Or... propose implementation in PR. It might save some Herb time (hopefully). I can help in reviewing the code.

@JohelEGP

Thank you. I think I have to write a conclusion for future readers if they come up to this issue.

I'm reading again this comment from @JohelEGP:

Otherwise, I think it's a win. The function syntax is regular. At namespace scope and type scope, they declare functions. At statement scope, they're currently banned. As expressions, they work and permit captures. And finally, they work as the for loop's parametrized statements' body (no scope introduced, and I suppose captures not permitted).

And I put it next to this informative comment from @hsutter:

BTW, Cpp2 will have local functions. They're pretty much there, I just ban them as a 'temporary alpha limitation' only because I haven't been motivated to prioritize the task of emitting them as a lambda and making sure they can capture. It shouldn't be hard, but I've wanted to drive other things like meta functions first. ...

So I should give up to separate functions and lambdas for learning Cpp2, because they are implementation detail. To simplify the learning progress I should keep in mind that there is only functions (either named or unnamed). Thus I should change my comment from:

Yes absolutely. It would make control structures to be consistent with each other in addition to simplification of guidance:

{ ... } will be for the statement block of control structures, therefore variables don't have to be captured.

But : (args) = { ... } will be for lambdas, therefore variables have to be captured. for loop breaks this rule.

Expressions in control structures (e.g. the expression after next keyword in while loop) won't be evaluate at first evidence, and they can be evaluated multiple times.

But expressions in other places (e.g. for variable initialization or function arguments) will be evaluated one time at first evidence.

... to this one:

Conclusion

Cpp2 have functions and local functions (either named or unnamed). { ... } is a statement block. : (args) = { ... } is a function. The following rules apply to them:

Expressions (e.g. for variable initialization, function arguments and ...) will be evaluated one time at first evidence in their place, because they have to be evaluated before they can be passed to a function. Also to access local variables which are from outer function scope, we have to capture them inside statement blocks (e.g. either { ... } or : (args) = { ... }):

...
variable: = 0;
forrr(items, : (item) = {
    ...
    // We have to capture `variable`.
    x: = variable$ + 10;
    ...
});


count: = 0;
// `count < 10` will be evaluated only one time at first evidence in their place.
// `count++` will be evaluated only one time at first evidence in their place.
// So this doesn't do the same thing that `while` does.
whileee(count < 10, count++, : () = {
    ...
});
...

But in control structures, expressions (e.g. the expression after next keyword in while loop) won't be evaluate at first evidence, and they can be evaluated multiple times (they are somehow macros), because of that, to access local variables from outer function scope, we don't have to capture them inside statement blocks (e.g. either { ... } or : (args) = { ... }):

...
variable: = 0;
for items do: (item) = {
    ...
    // We don't have to capture `variable`.
    x: = variable + 10;
    ...
}

count: = 0;
// `count < 10` will be evaluated multiple times.
// `count++` won't be evaluated at first time but it will be evaluated later multiple times.
while count < 10 next count++ {
    ...
}
...

In this way Cpp2 language is consistent with simple rules. Thanks for your guidance from those comments.

@filipsajdak

Or... propose implementation in PR. It might save some Herb time (hopefully). I can help in reviewing the code.

@filipsajdak, You're right, it's the best way to suggest features in Cpp2. But I hope I could... I don't have any experience or study in transpilers, compilers and etc... I'm going to read the source code to try my luck to understand it.

@msadeqhe

@msadeqhe I did not have much experience with it as well. I can help.

Preface lost in an accidentally deleted comment. Recovered:

So, I'd say, in the happy path:
The Cpp2 range-based for statement
label? 'for' expression next-clause? 'do' unnamed-declaration
is equivalent to (after lowering to Cpp1)
{
  label?
  for ( parameter-declaration : expression ) {
    { compound-statement }
    next-clause?
  }
}
where parameter-declaration and compound-statement
are the immediate productions that make up the unnamed-declaration.

That said, I'd prefer a description that doesn't rely on Cpp1. I'd like to think that you don't have to learn Cpp1 to learn Cpp2, before learning Cpp1, and such. Much like we say about teaching C++ and C.

Will this let parameters require/allows setting argument passing type?

Yes, that's the idea.

So would "next" become a keyword for standard for loops then?

Do you mean something like this? This works now:

main: (args) = {
    local := 10;
    for args next local++ do: (arg) = {
        x := local + 10;
        std::cout << x << "\n";
    }
}

Invoked with test.exe a b this prints:

20
21
22

@JohelEGP

Also, over on #382, @JohelEGP mentioned this paper proposing do expressions.

Which makes me think, hmm, could using do alleviate my concern about introducing a special "let" keyword for parameterized blocks... i.e., what if:
// alternative 'for' syntax
for items do (item) {
    use( item, local_var );
    // ...
}

// a block with parameters: what if it also starts with 'do'?
do (item := f()) { // an 'in' (read-only) parameter (initializer is required)
    use( item, local_var );
    // ...
}

If we consider it with if/while/... then it's for variable initialization inside the statement block. I think it's different from do (params) { ... } in for loop which passes arguments to the statement block. Therefore choosing let keyword seems more natural for its purpose. Both let (inits) and do (params) can coexist (be used together) in one control structure:

let (item: = f()) {
    user (item, local_var);
    // ...
}

let (item: = f()) if item.is_valid() {
    // ...
}

let (local: = 10) for args next local++ do (arg) {
    // ...
}

let (local: = 10) {
    // ...
} while local > 0;

let (i: = 0) while i < 10 next i++ {
    // ...
}

Interestingly let-while-next is similar to for (init; condition; step) { ... } in Cpp1. Also let-while is complementary to do-while. Maybe let (inits) is a little more readable and it can be used in reflections in comparison to { inits; ... }. In a nutshell:

do (params) { ... } for passing arguments to parameterized blocks.
let (inits) LANGUAGE-CONSTRUCT { ... } for variable initializations in control structures and parameterized blocks.

Both of them are parameterized statement blocks, and they can be used together, e.g. let (inits) for args do (arg) { ... }.

Maybe a version with an explicit block will be better:
(it := blocks.begin()) {
  while it != blocks.end() next it++ {
    // ...
  }
}
Exactly that example was something that made me hesitate, because it's not much different from the already-legal:
{
  it := blocks.begin();
  while it != blocks.end() next it++ {
    // ...
  }
}
and that made me hesitate about the value of the feature.

That's right, and let is going to be just a syntax sugar (perhaps more readable which is opinion-based, BTW it's a separation of semantic which means it is used to construct while loop):

let (it := blocks.begin()) while it != blocks.end() next it++ {
    // ...
}

Thanks everyone for all the comments, and especially for pointing out the key things that prompted me to finally bring back and implement the function/block unification I've always wanted, including block/statement scope parameter lists, and removing : and = from the for loop syntax.

Now that I've implemented block/statement scope parameter lists and tried using them in a ~dozen test examples, I do find them to be as semantically useful as I had hoped, which is shown in this new test case where it feels nice to be able to directly declare intended side effects of a statement/loop/block of code. For now I'm limiting it to in and inout parameters, but some notes:

move is that one that would make the most sense, to express a statement/block that consumes its move parameter's initializer, but I haven't got around to making that work yet;
out is of questionable value... it would express a statement/block that can construct the parameter's initializer, but (a) the variable (local or parameter) that it would initialize is already in scope and therefore fully covered by the existing initialization guarantee logic, so I'm not sure of the benefit of supporting this extra indirection if it merely amounts to renaming (which could be a negative thing, if it's really just code obfuscation), and (b) I suspect that would be a bunch more work in sema and I don't want to be distracted with that right now
forward would express a statement/block that forwards the initializer, but to make sense that initializer would have to be some function call expression, not a local variable (which we can already see the definition of, so forwarding isn't needed, similarly to the reason why I don't think the out flavor is probably all that useful)

Re migration: The for loop change is well justified and is just removing two characters, but it's also the first significant syntax breaking change. Because I showed that syntax in a talk/documentation, I've added the first compiler migration diagnostic so that people who try the old syntax should see a helpful message that the syntax has changed and what to write instead. Also, I need to update the example(s) on Godbolt [update: fortunately it uses while and so isn't broken]. There are costs to breaking changes even when there are no production users yet... 😁

Feedback always welcome, especially bugs (e.g., does something not work) and usability (e.g., does the migration message fire in most cases). Thanks again.

@ntrel

Thanks. I like how the following is similar to for (init; condition; step) ...:

(i: = 0) while i < 10 next i++ {
    std::cout << i << "\n";
}

EDIT

Thanks @ntrel about the issue with my example. This is working now:

(copy i: = 0) while i < 10 next i++ {
    std::cout << i << "\n";
}

(i: = 0) while i < 10 next i++ {

That needs inout i for mutability, though it doesn't work yet: #393.

(inout i := 0) while i < 10 next i++ {

I think it would read a bit nicer if inout was spelled mut.

Work around a C++20 compiler's range-for scoped variable bug/ICE

Minor `enum` and `union` tidying

Add statement (including block) scope in/inout parameters, and apply …

Navigation Menu

Repository navigation

[SUGGESTION] `for` loop is ambiguous with lambda syntax #386

Preface

How to reproduce?

Suggestion Detail

Why do I suggest this change?

Your Questions

Considered alternatives

Conclusion

14 remaining items

EDIT

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[SUGGESTION] for loop is ambiguous with lambda syntax #386

Description

Preface

How to reproduce?

Suggestion Detail

Why do I suggest this change?

Your Questions

Considered alternatives

Activity

JohelEGP commented on Apr 20, 2023

msadeqhe commented on Apr 20, 2023

JohelEGP commented on Apr 20, 2023

msadeqhe commented on Apr 20, 2023

JohelEGP commented on Apr 20, 2023

hsutter commented on Apr 20, 2023

msadeqhe commented on Apr 20, 2023

hsutter commented on Apr 20, 2023

filipsajdak commented on Apr 20, 2023

msadeqhe commented on Apr 21, 2023

Conclusion

msadeqhe commented on Apr 21, 2023

filipsajdak commented on Apr 21, 2023

JohelEGP commented on Apr 21, 2023

14 remaining items

hsutter commented on Apr 21, 2023

msadeqhe commented on Apr 21, 2023

msadeqhe commented on Apr 21, 2023

msadeqhe commented on Apr 21, 2023

msadeqhe commented on Apr 22, 2023

msadeqhe commented on Apr 22, 2023

msadeqhe commented on Apr 22, 2023

hsutter commented on Apr 23, 2023

msadeqhe commented on Apr 24, 2023

EDIT

ntrel commented on Apr 24, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

[SUGGESTION] `for` loop is ambiguous with lambda syntax #386