Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ Latest
## Changes
* [#197](https://github.com/cleverage/process-bundle/issues/197) Add missing tests: every Task and Transformer is now covered by unit tests. Remove the obsolete `tests.old` directory.
* [#143](https://github.com/cleverage/process-bundle/issues/143) Improve PHPStan configuration: remove all `ignoreErrors` and `@phpstan-ignore` comments (report unmatched ignored errors again), add missing iterable value types and generic types in PHPDoc, remove unreachable code in AbstractIterableOutputTask and InputAggregatorTask.
* [#143](https://github.com/cleverage/process-bundle/issues/143) Improve PHPStan level from 6 to 7, and fix the level 8 errors that do not require a signature change (the remaining ones, due to nullable return types such as `AbstractConfigurableTask::getOptions(): ?array`, will be fixed in v6.0). Default to a `ContextualOptionResolver` in ProcessState and to a property accessor in ConditionTrait when none is set.

## Fixes
* [#143](https://github.com/cleverage/process-bundle/issues/143) Fix InputIteratorTask: an `\IteratorAggregate` input whose `getIterator()` does not return an `\Iterator` (e.g. another `\IteratorAggregate`) is iterated instead of failing with a `TypeError`. Update documentation, add tests.
* [#143](https://github.com/cleverage/process-bundle/issues/143) Fix level 8 edge cases failing with a `TypeError` or an `Error`: `CsvResource` rejects a blank header line and checks `ftell()`, XpathEvaluatorTransformer throws an explicit `\UnexpectedValueException` on an invalid query (with the libxml messages), a namespace node to unwrap or a node without document, SlugifyTransformer on a value that cannot be transliterated, `ProcessState::getContext()` returns `[]` until the context is defined, GenericTransformer and TransformerTrait throw a `\LogicException` when not initialized, IterableBatchTask and JsonStreamReaderTask can be flushed / iterated before their first execution, RegistryCompilerPass checks its configuration, SubprocessInstance checks the PHP executable. Update documentation, add tests.

v5.1
-----
Expand Down
3 changes: 2 additions & 1 deletion docs/reference/tasks/csv_reader_task.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ Notes
-----

* Each line must contain exactly as many columns as there are headers, otherwise an `\UnexpectedValueException` is thrown.
* A UTF-8 BOM is removed from the first header when headers are read from the file.
* A UTF-8 BOM is removed from the first header when headers are read from the file. If the first line is blank, an
`\UnexpectedValueException` is thrown.
* `csv_file` and `csv_line` are added to the error context of the process.
* The file is closed once fully read: if the task is executed again (e.g. for a new input), the file is read again
from its beginning.
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/transformers/slugify_transformer.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,5 @@ Notes
An invalid `transliterator` identifier (rejected by `\Transliterator::create()`), or an option that is not a string,
raises an `InvalidOptionsException` when the options are resolved, i.e. when the transformer is configured, not on the
first transformed value.

A value that cannot be transliterated (e.g. invalid UTF-8) throws an `\UnexpectedValueException`.
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ Transformer reference
Accepted inputs
---------------

`\DOMNode` (including `\DOMDocument`). Any other value throws an `\UnexpectedValueException`.
`\DOMNode` (including `\DOMDocument`). Any other value, a node that does not belong to a document or an invalid xpath
query throws an `\UnexpectedValueException`.

Possible outputs
----------------
Expand Down
2 changes: 1 addition & 1 deletion phpstan.neon
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
parameters:
level: 6
level: 7
paths:
- src
- tests
Expand Down
2 changes: 1 addition & 1 deletion src/Command/ListProcessCommand.php
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,6 @@ protected function padMessage(string $message, int $length = 80): string

protected function filterOutTags(string $string): string
{
return preg_replace('/<[^<>]*>/', '', $string);
return preg_replace('/<[^<>]*>/', '', $string) ?? $string;
}
}
23 changes: 16 additions & 7 deletions src/Command/ProcessHelpCommand.php
Original file line number Diff line number Diff line change
Expand Up @@ -107,12 +107,13 @@ protected function execute(InputInterface $input, OutputInterface $output): int
$totalBranches = \count($taskList);
for ($i = 0; $i < $totalBranches; ++$i) {
// Find the best task to display
$nextTaskCode = $this->findBestNextTask($branches, $remainingTasks, $process);
// A numeric task code may be returned as an integer array key
$nextTaskCode = (string) $this->findBestNextTask($branches, $remainingTasks, $process);

$this->resolveBranchOutput($branches, $nextTaskCode, $process, $output);

// Remove the task from the remaining list
$remainingTasks = array_filter($remainingTasks, static fn ($task): bool => $task !== $nextTaskCode);
$remainingTasks = array_values(array_filter($remainingTasks, static fn ($task): bool => $task !== $nextTaskCode));
}

$branches = array_filter($branches);
Expand All @@ -127,8 +128,8 @@ protected function execute(InputInterface $input, OutputInterface $output): int
/**
* Try to find a best candidate for next display.
*
* @param array<string|null> $branches
* @param list<string> $taskList
* @param array<int, string|null> $branches
* @param list<string> $taskList
*/
protected function findBestNextTask(
array $branches,
Expand Down Expand Up @@ -227,7 +228,7 @@ protected function getTaskChildrenCount(TaskConfiguration $task): int
/**
* Merge needed branches, display a task node, split following needed branches.
*
* @param array<string|null> $branches
* @param array<int, string|null> $branches
*/
protected function resolveBranchOutput(
array &$branches,
Expand Down Expand Up @@ -293,7 +294,12 @@ protected function resolveBranchOutput(

$origin = array_shift($branchesToMerge);
$final = $gapFrom;
$branches[$origin] = $taskCode;
if (null === $origin) {
// No previous branch found (an error has been displayed): open a new branch
$branches[] = $taskCode;
} else {
$branches[$origin] = $taskCode;
}
}

// Merge branches
Expand Down Expand Up @@ -377,6 +383,9 @@ static function ($taskCode, $i) use ($gapBranches, $origin, $final, $branches):
$this->writeBranches($output, $branches);
array_shift($nextTasks);
$origin = array_search($taskCode, $branches, true);
if (false === $origin) {
throw new \UnexpectedValueException("Task '{$taskCode}' is not in any branch");
}
$expandBranches = [];
foreach ($nextTasks as $nextTask) {
$index = array_search(null, $branches, true);
Expand Down Expand Up @@ -448,7 +457,7 @@ static function ($branchTask, $i) use ($origin, $branches, $gapBranches, $final)
}

/**
* @param array<string|null> $branches
* @param array<int, string|null> $branches
* @param string|iterable<string> $comment
*/
protected function writeBranches(
Expand Down
4 changes: 4 additions & 0 deletions src/DependencyInjection/Compiler/RegistryCompilerPass.php
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ public function __construct(
*/
public function process(ContainerBuilder $container): void
{
if (\in_array(null, [$this->registry, $this->tag, $this->method], true)) {
throw new \LogicException('The registry, tag and method of the RegistryCompilerPass must be defined');
}

if (!$container->has($this->registry)) {
return;
}
Expand Down
17 changes: 14 additions & 3 deletions src/Filesystem/CsvResource.php
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,8 @@ public function isEndOfFile(): bool
/**
* Warning, this function will return exactly the same value as the fgetcsv() function.
*
* @param int<0, max>|null $length
*
* @return list<string|null>|false
*/
public function readRaw(?int $length = null): array|false
Expand All @@ -167,6 +169,8 @@ public function readRaw(?int $length = null): array|false
}

/**
* @param int<0, max>|null $length
*
* @return array<string, string|null>|null
*/
public function readLine(?int $length = null): ?array
Expand Down Expand Up @@ -259,7 +263,12 @@ public function tell(): int
{
$this->assertOpened();

return ftell($this->handler);
$position = ftell($this->handler);
if (false === $position) {
throw new \RuntimeException("Unable to get the position in {$this->getResourceName()}");
}

return $position;
}

public function seek(int $offset): int
Expand Down Expand Up @@ -313,12 +322,14 @@ protected function parseHeaders(?array $headers = null): array
// If headers are not passed in the constructor but file is readable, try to read headers from file
if (null === $headers) {
$autoHeaders = $this->readRaw();
if (false === $autoHeaders || [] === $autoHeaders) {
// A blank line is read as [null]
if (false === $autoHeaders || [null] === $autoHeaders) {
throw new \UnexpectedValueException("Unable to read headers for {$this->getResourceName()}");
}
$autoHeaders = array_map(strval(...), $autoHeaders);
// Remove BOM if any
$bom = pack('H*', 'EFBBBF');
$autoHeaders[0] = preg_replace("/^{$bom}/", '', (string) $autoHeaders[0]);
$autoHeaders[0] = (string) preg_replace("/^{$bom}/", '', $autoHeaders[0]);

return $autoHeaders;
}
Expand Down
2 changes: 2 additions & 0 deletions src/Filesystem/FileStreamInterface.php
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ public function getLineNumber(): int;
public function isEndOfFile(): bool;

/**
* @param int<0, max>|null $length
*
* @return array<mixed>|null
*/
public function readLine(?int $length = null): ?array;
Expand Down
2 changes: 2 additions & 0 deletions src/Filesystem/JsonStreamFile.php
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ public function isEndOfFile(): bool
/**
* Return an array containing current data and moving the file pointer.
*
* @param int<0, max>|null $length
*
* @return array<mixed>|null
*
* @throws \UnexpectedValueException if the line decodes to a scalar value
Expand Down
5 changes: 3 additions & 2 deletions src/Model/ProcessState.php
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ public function isResolved(): bool
*/
public function getContext(): array
{
return $this->context;
return $this->context ?? [];
}

/**
Expand All @@ -303,9 +303,10 @@ public function getContextualizedOptions(): ?array
if (!$this->contextualizedOptions) {
$options = $this->getTaskConfiguration()
->getOptions();
$this->contextualOptionResolver ??= new ContextualOptionResolver();
$this->contextualizedOptions = $this->contextualOptionResolver->contextualizeOptions(
$options,
$this->context
$this->getContext()
);
}

Expand Down
7 changes: 5 additions & 2 deletions src/Model/SubprocessInstance.php
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,14 @@ public function __construct(
*/
public function buildProcess(): static
{
$pathFinder = new PhpExecutableFinder();
$phpBinary = (new PhpExecutableFinder())->find();
if (false === $phpBinary) {
throw new \RuntimeException('Unable to find the PHP executable');
}

$arguments = [
'nohup',
$pathFinder->find(),
$phpBinary,
$this->consolePath,
'--env='.$this->environment,
'cleverage:process:execute',
Expand Down
10 changes: 7 additions & 3 deletions src/Task/AbstractIterableOutputTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,14 @@ abstract class AbstractIterableOutputTask extends AbstractConfigurableTask imple
public function execute(ProcessState $state): void
{
$this->handleIteratorFromInput($state);
$iterator = $this->iterator;
if (!$iterator instanceof \Iterator) {
throw new \LogicException('No iterator initialized');
}

if ($this->iterator->valid()) {
$state->addErrorContextValue('iterator_key', $this->iterator->key());
$state->setOutput($this->iterator->current());
if ($iterator->valid()) {
$state->addErrorContextValue('iterator_key', $iterator->key());
$state->setOutput($iterator->current());
} else {
$state->setSkipped(true);
$this->iterator = null;
Expand Down
12 changes: 8 additions & 4 deletions src/Task/File/Csv/CsvReaderTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,17 @@ public function execute(ProcessState $state): void
if (!$this->csv instanceof CsvFile) {
$this->initFile($state);
}
$lineNumber = $this->csv->getLineNumber();
$output = $this->csv->readLine();
$csv = $this->csv;
if (!$csv instanceof CsvFile) {
throw new \LogicException('No CSV File initialized');
}
$lineNumber = $csv->getLineNumber();
$output = $csv->readLine();

if (null === $output) {
if ($this->getOption($state, 'log_empty_lines')) {
$logContext = [
'csv_file' => $this->csv->getFilePath(),
'csv_file' => $csv->getFilePath(),
'csv_line' => $lineNumber,
];
$this->logger->warning("Empty line detected at line: {$lineNumber}", $logContext);
Expand All @@ -55,7 +59,7 @@ public function execute(ProcessState $state): void
$state->setSkipped(true);
}

$state->addErrorContextValue('csv_file', $this->csv->getFilePath());
$state->addErrorContextValue('csv_file', $csv->getFilePath());
$state->addErrorContextValue('csv_line', $lineNumber);
$state->setOutput($output);
}
Expand Down
3 changes: 2 additions & 1 deletion src/Task/File/Csv/CsvWriterTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ protected function getHeaders(ProcessState $state, array $options): ?array
{
$headers = $options['headers'];
if (null === $headers) {
return array_keys($state->getInput());
// Keys of a list are integers: the CSV output is the same with their string value
return array_map(strval(...), array_keys($state->getInput()));
}

return $headers;
Expand Down
25 changes: 21 additions & 4 deletions src/Task/File/FolderBrowserTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,9 @@ public function execute(ProcessState $state): void
$this->files = $finder->in($options['folder_path'])->sortByName()->getIterator();
$this->files->rewind();
}
$files = $this->getFilesIterator();

if (!$this->files->valid()) {
if (!$files->valid()) {
$this->logger->log($options['empty_log_level'], "No item found in path {$options['folder_path']}");
$state->setSkipped(true);
$state->setErrorOutput($options['folder_path']);
Expand All @@ -61,7 +62,7 @@ public function execute(ProcessState $state): void
return;
}
/** @var SplFileInfo $fileInfo */
$fileInfo = $this->files->current();
$fileInfo = $files->current();
$filePath = $fileInfo->getPathname();
$state->addErrorContextValue('current_file_path', $filePath);
$state->setOutput($filePath);
Expand All @@ -77,10 +78,11 @@ public function next(ProcessState $state): bool
if (!$this->files) {
return false;
}
$this->files->next();
$files = $this->getFilesIterator();
$files->next();
$state->removeErrorContext('current_file_path');

if (!$this->files->valid()) {
if (!$files->valid()) {
// Reset the iterator to allow the following iteration
$this->files = null;

Expand Down Expand Up @@ -126,4 +128,19 @@ static function (Options $options, $value) {
]
);
}

/**
* @return \Iterator<SplFileInfo>
*/
private function getFilesIterator(): \Iterator
{
if (\is_array($this->files)) {
$this->files = new \ArrayIterator($this->files);
}
if (!$this->files instanceof \Iterator) {
throw new \LogicException('No folder browsed');
}

return $this->files;
}
}
3 changes: 3 additions & 0 deletions src/Task/File/JsonStream/JsonStreamReaderTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ public function execute(ProcessState $state): void

public function next(ProcessState $state): bool
{
if (!$this->file instanceof JsonStreamFile) {
return false;
}
$eof = $this->file->isEndOfFile();
if ($eof) {
$this->file = null;
Expand Down
4 changes: 3 additions & 1 deletion src/Task/IterableBatchTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ public function initialize(ProcessState $state): void
public function flush(ProcessState $state): void
{
$this->flushMode = true;
$this->outputQueue ??= new \SplQueue();
if ($this->outputQueue->isEmpty()) {
$state->setSkipped(true);
} else {
Expand All @@ -58,14 +59,15 @@ public function flush(ProcessState $state): void
public function execute(ProcessState $state): void
{
$batchCount = $this->getOption($state, 'batch_count');
$this->outputQueue ??= new \SplQueue();

// Register new input
if (!$this->flushMode) {
$this->outputQueue->enqueue($this->processInput($state));
}

// Detect flushing
if (null !== $batchCount && ($this->outputQueue instanceof \SplQueue ? \count($this->outputQueue) : 0) >= $batchCount) {
if (null !== $batchCount && \count($this->outputQueue) >= $batchCount) {
$this->flushMode = true;
}

Expand Down
4 changes: 2 additions & 2 deletions src/Task/Process/ProcessLauncherTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -222,8 +222,8 @@ function (Options $options, $value) {
$resolver->setAllowedTypes('process_options', ['array']);
$resolver->setNormalizer(
'process_options',
static function (Options $options, $value): array {
if (!empty($value)) {
static function (Options $options, array $value): array {
if ([] !== $value) {
// Todo deprecation trigger
throw new \InvalidArgumentException('Deprecated option, please contact support for help');
}
Expand Down
Loading
Loading