Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ Latest

## Fixes
* [#388](https://github.com/cleverage/process-bundle/issues/388) The temporary files of `file_splitter`, `csv_splitter` and of the JSON buffer of `process_launcher` (#343) are created with the `0077` umask: they are no longer readable by other users before their permissions are set to `0600`. Add tests.
* [#390](https://github.com/cleverage/process-bundle/issues/390) `cleverage:process:execute` with `--output-format=json-stream` and STDOUT (#353): the invalid UTF-8 sequences of the output are replaced, and an output that still cannot be encoded as JSON (e.g. `NAN`) is reported on the error output instead of throwing a `JsonException` that made the command fail after the execution and skipped the next processes. `UPGRADE.md` (v5.5): the `dump` format without `symfony/var-dumper` and the unknown formats make the command fail before any execution. Update documentation, add tests.

v5.6
-----
Expand Down
2 changes: 2 additions & 0 deletions UPGRADE.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ process because of an error, let the error fail it (error strategy `stop`).
- With `--output-format`, the output is printed to STDOUT whatever the verbosity (it required `-vv`): a script reading
STDOUT now receives it.
- A non-array output that cannot be stored as JSON stream is reported on the error output.
- `--output-format=dump` without `symfony/var-dumper`, or an unknown format, makes the command fail before any process
is executed: it was ignored for STDOUT without `-vv`. Install `symfony/var-dumper` or fix the option.

### Tasks and transformers

Expand Down
4 changes: 2 additions & 2 deletions docs/01-quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Some features rely on optional packages, suggested by the bundle:
| `ext-intl` (PHP extension) | the [SlugifyTransformer](reference/transformers/slugify_transformer.md) |
| `symfony/process` | the [CommandRunnerTask](reference/tasks/command_runner_task.md) and the [ProcessLauncherTask](reference/tasks/process_launcher_task.md) |
| `symfony/stopwatch` | the [StopwatchTask](reference/tasks/stopwatch_task.md) |
| `symfony/var-dumper` | the dumps of the [DebugTask](reference/tasks/debug_task.md), of the `debug` transformer and of `cleverage:process:execute --output-format=dump` (nothing is dumped without it) |
| `symfony/var-dumper` | the dumps of the [DebugTask](reference/tasks/debug_task.md), of the `debug` transformer and of `cleverage:process:execute --output-format=dump` (required by this format: the command fails without it) |
| `symfony/monolog-bundle` | the `cleverage_process`, `cleverage_process_task` and `cleverage_process_transformer` log channels, with the process and task data in the `extra` of the records (see [logging](03-custom_tasks.md#logging)); without it, the default Symfony logger is used |

Without `symfony/process`, `symfony/stopwatch` or `ext-intl`, the container still compiles, and a process using one of
Expand Down Expand Up @@ -267,7 +267,7 @@ Process 'project_prefix.process_name' executed successfully
| `--input-from-stdin` | | Read the input value from STDIN instead (e.g. `cat file.json \| bin/console cleverage:process:execute ...`) |
| `--context=<key>:<value>` | `-c` | Contextual value, can be repeated. The key must only contain word characters (`\w+`: letters, digits, `_`). The value is parsed as YAML, so `-c limit:10` gives an integer and `-c name:"'foo'"` forces a string (also needed for dates: `-c date:2024-01-01` gives an integer timestamp) |
| `--output=<path>` | `-o` | Where to dump the value returned by the task defined by the process `end_point`: `-` (default) for STDOUT, or a file path |
| `--output-format=<format>` | `-t` | Format of the dumped output: `dump` (Symfony VarDumper, STDOUT only) or `json-stream` (written to STDOUT, or into the `--output` file if the value is an array: otherwise a warning is written on the error output). Nothing is dumped when this option is omitted. Without `end_point`, the output is `null`. The format and the process codes are checked before any process is executed |
| `--output-format=<format>` | `-t` | Format of the dumped output: `dump` (Symfony VarDumper, STDOUT only) or `json-stream` (written to STDOUT, invalid UTF-8 sequences replaced, or into the `--output` file if the value is an array: otherwise, or when the value cannot be encoded as JSON for STDOUT, a warning is written on the error output). Nothing is dumped when this option is omitted. Without `end_point`, the output is `null`. The format and the process codes are checked before any process is executed |

Example:

Expand Down
17 changes: 14 additions & 3 deletions src/Command/ExecuteProcessCommand.php
Original file line number Diff line number Diff line change
Expand Up @@ -171,16 +171,22 @@ protected function handleOutputData(mixed $data, InputInterface $input, OutputIn
if (self::OUTPUT_FORMAT_DUMP === $input->getOption('output-format')) {
VarDumper::dump($data);
} else {
$output->writeln(json_encode($data, \JSON_THROW_ON_ERROR));
// Invalid UTF-8 replaced: the output of a process reading a non UTF-8 file can still be printed
$json = json_encode($data, \JSON_INVALID_UTF8_SUBSTITUTE);
if (false === $json) {
// Reported, as before 5.5 without -vv: the next processes are executed
$this->getErrorOutput($output)->writeln(\sprintf('<comment>Output not printed: it cannot be encoded as JSON (%s)</comment>', json_last_error_msg()));
} else {
$output->writeln($json);
}
}

return;
}

// JsonStreamFile::writeLine() only takes an array
if (!\is_array($data)) {
$errorOutput = $output instanceof ConsoleOutputInterface ? $output->getErrorOutput() : $output;
$errorOutput->writeln(\sprintf("<comment>Output not stored in '%s': the JSON stream format requires an array, %s given</comment>", $input->getOption('output'), get_debug_type($data)));
$this->getErrorOutput($output)->writeln(\sprintf("<comment>Output not stored in '%s': the JSON stream format requires an array, %s given</comment>", $input->getOption('output'), get_debug_type($data)));

return;
}
Expand All @@ -190,4 +196,9 @@ protected function handleOutputData(mixed $data, InputInterface $input, OutputIn
$output->writeln(\sprintf("Output stored in '%s'", $input->getOption('output')));
}
}

private function getErrorOutput(OutputInterface $output): OutputInterface
{
return $output instanceof ConsoleOutputInterface ? $output->getErrorOutput() : $output;
}
}
20 changes: 20 additions & 0 deletions tests/Command/ExecuteProcessCommandTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,26 @@ public function testTheJsonOutputIsPrintedWithoutVerbosity(): void
self::assertStringContainsString('{"foo":"bar"}', $tester->getDisplay());
}

public function testInvalidUtf8IsReplacedInTheJsonOutput(): void
{
$tester = $this->createTester($this->once(), ['name' => "Caf\xe9"]);

self::assertSame(0, $tester->execute(['processCodes' => ['test.process'], '--output-format' => 'json-stream']));
self::assertStringContainsString('{"name":"Caf\ufffd"}', $tester->getDisplay());
}

public function testAnOutputThatCannotBeEncodedAsJsonIsReportedAndTheNextProcessesAreExecuted(): void
{
$tester = $this->createTester($this->exactly(2), ['value' => \NAN]);

self::assertSame(0, $tester->execute(
['processCodes' => ['test.process', 'test.process'], '--output-format' => 'json-stream'],
['capture_stderr_separately' => true],
));
self::assertSame(2, substr_count($tester->getErrorOutput(), 'Output not printed: it cannot be encoded as JSON (Inf and NaN cannot be JSON encoded)'));
self::assertSame(2, substr_count($tester->getDisplay(), "Process 'test.process' executed successfully"));
}

public function testAnArrayOutputIsStoredInTheFile(): void
{
$tester = $this->createTester($this->once(), ['foo' => 'bar']);
Expand Down
Loading