Skip to content

Add generators support - #177

Merged
php-pulls merged 74 commits into
php:masterfrom
nikic:addGeneratorsSupport
Sep 1, 2012
Merged

php-pulls merged 74 commits into
php:masterfrom
nikic:addGeneratorsSupport

Conversation

@nikic

@nikic nikic commented Aug 25, 2012

Copy link
Copy Markdown
Member

PR for generators as outlined in https://wiki.php.net/rfc/generators.

(For the diff)

nikic added 30 commits May 15, 2012 18:30
Generator functions have to specify the * (asterix) modifier after the
function keyword. If they do so the ZEND_ACC_GENERATOR flag is added to
the fn_flags.
The block for the foreach separator was nested unnecessary. This commit
simply removes that nesting.
The yield statement can only be used in generator functions, which are
marked with an asterix.
The execution of generator functions will be suspended right after the
arguments were RECVed. This will be done in zend_do_suspend_if_generator.
If the function is a generator this opcode will be invoked right after
receiving the function arguments.

The current implementation is just a dummy.
The Generator class now uses a zend_generator struct, so it'll be able to
store additional info.

This commit also ensures that Generator cannot be directly instantiated
and extended. The error tests are now in a separate folder from the
(yet-to-come) functional tests.
Right now generator functions simply immediately return a new Generator
object (no suspension yet).
Generators need to switch the execute_data very often. If the execute_data
is allocated on the VM stack this operation would require to always copy
the structure (which is quite large). That's why the execution context is
allocated on the heap instead (only for generators obviously).
This is just some initial code, which is still quite broken (and needs to be
moved so it can be reused.)
This simply adds dummy rewind/valid/current/key/next methods to Generator.
Before one could only call it with cwd=Zend.
This adds another function execute_ex(), which accepts a zend_execute_data
struct to run (contrary to execute(), which accepts a zend_op_array from
which it initialized the execute_data).

This needs a bit more cleanup.
The generator zval is put into the return_value_ptr_ptr.
For generators ZEND_RETURN directly calls ZEND_VM_RETURN(), thus passing
execution back to the caller (zend_generator_resume).

This commit also adds a check that only return; is used in generators and
not return $value;.
The test implements an xrange() function (the generator version of range()).
If the generator is closed before it has finished running, it may happen
that some FREE or SWITCH_FREE opcodes haven't been executed and memory is
leaked.

This fixes it by walking the brk_cont_array and manually freeing the
variables.
This happens primarily when the generator is invoked from some internal
place like a dynamic function call.
This fixes several issues. In particular it makes method generators work
properly and also allows generators using a symbol table.
To keep things clean two new functions are introduced:

zend_clean_and_cache_symbol_table(HashTable *symbol_table)
zend_free_compiled_variables(zval ***CVs, int num)
Yield now is an expression and the return value is the value passed to
$generator->send(). By default (i.e. if ->next() is called) the value is
NULL.

Unlike in Python ->send() can be run without priming the generator with a
->next() call first.
If the generator is used as a coroutine it often doesn't make sense to yield
anything. In this case one can simply receive values using

    $value = yield;

The yield here will simply yield NULL.
Calling $generator->close() is equivalent to executing a return statement
at the current position in the generator.
nikic added 5 commits August 24, 2012 13:29
Conflicts:
	Zend/zend_vm_def.h
	Zend/zend_vm_execute.h
Generators don't have a return value, so it doesn't make sense to have
a shared implementation here.
 * Trying to resume a generator while it is already running now throws a
   fatal error.
 * Trying to use yield in finally while the generator is being force-closed
   (by GC) throws a fatal error.
 * Rewinding after the first yield now throws an Exception
@laruence

Copy link
Copy Markdown
Member

Now I get the diff, thanks

nikic added 2 commits August 25, 2012 19:03
I decided to leave out yield delegation for an initial proposal, so remove
the stubs for it too.
Conflicts:
	Zend/zend_language_parser.y
	Zend/zend_vm_execute.skl

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why delete this test?

nikic added 3 commits August 29, 2012 20:31
If you try to traverse an already closed generator an exception will now be
thrown.

Furthermore this changes the error for traversing a by-val generator by-ref
from an E_ERROR to an Exception.
@travisbot

Copy link
Copy Markdown

This pull request fails (merged dbc7809 into 35951d4).

@php-pulls
php-pulls merged commit dbc7809 into php:master Sep 1, 2012
@KendallHopkins

Copy link
Copy Markdown

@nikic I'm getting segfaults with this code https://gist.github.com/3588986

@smalyshev

Copy link
Copy Markdown
Contributor

@KendallHopkins please submit a bug report to bugs.php.net so it could be tracked properly.

@KendallHopkins

Copy link
Copy Markdown

Comment thread Zend/zend_vm_def.h

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Through? 😄

heyJordanParker added a commit to heyJordanParker/php-sharp that referenced this pull request Oct 13, 2026
fix: a file that imports a class named like an effect still writes the effect in uses
heyJordanParker added a commit to heyJordanParker/php-sharp that referenced this pull request Oct 13, 2026
php#177 lets a file that imports or declares a class named like a standard
effect still write the effect in `uses`, and names a standard effect in
messages by the name the file's import gives it. This branch keeps that rule
and reads it from the one effect source instead of STANDARD_EFFECTS:

- the binder's `uses` resolves through standard_class, the same match every
  default name of the `Sharp` namespace goes through
- sharp_effect_name names any effect of the `Sharp` namespace by its import's
  name or its own, and any other effect as a class
- the uses check lives in the analyzer, so php#177's semantics row moves there as
  a_uses_takes_a_standard_effect_in_a_file_that_declares_a_class_of_its_name
- php#177's message row declares App.Effects.Payments as a foreign class, since a
  `uses` now takes a project class only when it is foreign

crates/
├── analyzer/src/utils/names.rs   <- sharp_effect_name reads the `Sharp` namespace, not a list
├── analyzer/src/effects/check.rs <- an implementation's refusal names effects as `uses` writes them
├── analyzer/tests/sharp.rs       <- php#177's rows, with the declared-class form moved from semantics
├── names/src/internal/walker.rs  <- `uses` resolves through standard_class
└── semantics/                    <- check_uses and its rows stay removed
heyJordanParker added a commit to heyJordanParker/php-sharp that referenced this pull request Oct 13, 2026
Master brings #114: the engine runs section 12's collection methods
and index places, and `sharp/mago/` moves to mago-sharp 3cc39cc. It
updates the five engine fixtures that the slice's Mago refused, so
`mago compile` compiles all 111 again. 3cc39cc also brings mago-sharp
php#175 and php#177, which git merged into the slice's Mago without a
conflict.

Six files conflicted, and each keeps both sides:
- `ext/sharp/sharp_arginfo.h` is regenerated from the merged
  `sharp.stub.php`, which holds master's `insert`, `removeAt`, `put` and
  `clear` and the slice's `\Sharp\Internal` functions.
- `index_read.inc` keeps master's two `count` calls and the slice's
  three `standing` calls.
- `enum_keys.phpt` and its two JIT twins keep the slice's `TypeError`:
  the Map's check refuses the key "missing" where plain PHP hands the
  Map in, before master's loop would convert it. Master's new `nest`
  line moves `byName` to line 67. `enum_keys.inc`'s comment that plain
  PHP collections are not checked is no longer true, and now says where
  the key fails.
- `opcodes.phpt` keeps master's 16 op arrays for `EnumKeys` and the
  slice's check lines, with one for each of master's `grouped` and
  `groupNames`.

Zend/tests/sharp/
├── enum_keys.inc                   <- the comment says where the key fails
├── enum_keys.phpt, enum_keys_jit_function.phpt, enum_keys_jit_tracing.phpt
│                                   <- the slice's TypeError, at line 67
├── index_read.inc                  <- both sides' calls
└── opcodes.phpt                    <- master's count, the slice's checks
ext/sharp/sharp_arginfo.h           <- regenerated from the merged stub
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants