Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Latest
* [#440](https://github.com/cleverage/process-bundle/issues/440) Documentation: `json_stream_reader` and `file_splitter` open their path with `SplFileObject`, any PHP stream wrapper is accepted (`http://`, `php://filter`...); the path must not be built from untrusted input (SSRF, reading local files), validate it before the task.

## Fixes
* [#455](https://github.com/cleverage/process-bundle/issues/455) `cleverage:process:execute -o <file> -t json-stream`: the invalid UTF-8 sequences of the output are replaced, as on STDOUT (a `JsonException` was thrown, also by the buffer of `process_launcher`); an output that cannot be encoded as JSON is reported on the error output. Update documentation, add tests.
* [#457](https://github.com/cleverage/process-bundle/issues/457) `mapping` / `callback`: the deprecations of a literal key of an array destination (once per target property) and of `additional_parameters` are triggered once, instead of for each item or each resolution of the options. Add tests.
* [#453](https://github.com/cleverage/process-bundle/issues/453) `trim`: the ranges of single-byte characters (`a..z`) of a `charlist` with a multibyte character are expanded before `mb_trim()`, which does not support them (the range was lost, its `.` removed). Update documentation, add tests.
* [#451](https://github.com/cleverage/process-bundle/issues/451) `process_launcher`: when the process fails or is stopped, the JSON buffers of the terminated sub-processes not output yet are removed too, as documented (they were left in the temporary directory). Add tests.
Expand Down
2 changes: 1 addition & 1 deletion docs/01-quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ Process 'project_prefix.process_name' executed successfully
| `--input-from-stdin` | | Read the input value from STDIN instead (e.g. `cat file.json \| bin/console cleverage:process:execute ...`) |
| `--context=<key>:<value>` | `-c` | Contextual value, can be repeated. The key must only contain word characters (`\w+`: letters, digits, `_`). The value is parsed as YAML, so `-c limit:10` gives an integer and `-c name:"'foo'"` forces a string (also needed for dates: `-c date:2024-01-01` gives an integer timestamp) |
| `--output=<path>` | `-o` | Where to dump the value returned by the task defined by the process `end_point`: `-` (default) for STDOUT, or a file path |
| `--output-format=<format>` | `-t` | Format of the dumped output: `dump` (Symfony VarDumper, STDOUT only) or `json-stream` (written to STDOUT, invalid UTF-8 sequences replaced, or into the `--output` file if the value is an array: otherwise, or when the value cannot be encoded as JSON for STDOUT, a warning is written on the error output). Nothing is dumped when this option is omitted. Without `end_point`, the output is `null`. The format and the process codes are checked before any process is executed |
| `--output-format=<format>` | `-t` | Format of the dumped output: `dump` (Symfony VarDumper, STDOUT only) or `json-stream` (written to STDOUT, or into the `--output` file if the value is an array, invalid UTF-8 sequences replaced: otherwise, or when the value cannot be encoded as JSON, a warning is written on the error output). Nothing is dumped when this option is omitted. Without `end_point`, the output is `null`. The format and the process codes are checked before any process is executed |

Example:

Expand Down
12 changes: 10 additions & 2 deletions src/Command/ExecuteProcessCommand.php
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,16 @@ protected function handleOutputData(mixed $data, InputInterface $input, OutputIn

return;
}
$outputFile = new JsonStreamFile($input->getOption('output'), 'wb');
$outputFile->writeLine($data);
// As on STDOUT, the invalid UTF-8 sequences are replaced (e.g. a value read from a Latin-1 file); the buffer of
// the process_launcher task is written here too
$outputFile = new JsonStreamFile($input->getOption('output'), 'wb', null, [\JSON_THROW_ON_ERROR, \JSON_INVALID_UTF8_SUBSTITUTE]);
try {
$outputFile->writeLine($data);
} catch (\JsonException $exception) {
$this->getErrorOutput($output)->writeln(\sprintf("<comment>Output not stored in '%s': it cannot be encoded as JSON (%s)</comment>", $input->getOption('output'), $exception->getMessage()));

return;
}
if ($output->isVerbose()) {
$output->writeln(\sprintf("Output stored in '%s'", $input->getOption('output')));
}
Expand Down
25 changes: 25 additions & 0 deletions tests/Command/ExecuteProcessCommandTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,31 @@ public function testAnArrayOutputIsStoredInTheFile(): void
self::assertSame('{"foo":"bar"}', trim((string) file_get_contents($this->outputFile)));
}

/**
* As on STDOUT: a JsonException was thrown (e.g. for the buffer of the process_launcher task).
*/
public function testInvalidUtf8IsReplacedInTheFile(): void
{
$tester = $this->createTester($this->once(), ['name' => "Caf\xe9"]);

self::assertSame(0, $tester->execute(['processCodes' => ['test.process'], '--output' => $this->outputFile, '--output-format' => 'json-stream']));
self::assertSame('{"name":"Caf\ufffd"}', trim((string) file_get_contents($this->outputFile)));
}

public function testAnOutputThatCannotBeEncodedAsJsonIsReportedInsteadOfBeingStored(): void
{
$tester = $this->createTester($this->once(), ['value' => \NAN]);

self::assertSame(0, $tester->execute(
['processCodes' => ['test.process'], '--output' => $this->outputFile, '--output-format' => 'json-stream'],
['capture_stderr_separately' => true],
));
self::assertStringContainsString(
"Output not stored in '{$this->outputFile}': it cannot be encoded as JSON (Inf and NaN cannot be JSON encoded)",
$tester->getErrorOutput()
);
}

public function testANonArrayOutputIsReportedInsteadOfBeingSilentlyLost(): void
{
$tester = $this->createTester($this->once(), 'scalar');
Expand Down
Loading