Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Latest
* [#392](https://github.com/cleverage/process-bundle/issues/392) Documentation: reset of the process configurations between two messages of a worker (`kernel.reset`, #355), permissions of the JSON buffer of `process_launcher`, `ProcessManager` subclasses in `UPGRADE.md` (v5.5), broken link in the CHANGELOG.

## Fixes
* [#408](https://github.com/cleverage/process-bundle/issues/408) Regular expressions of `preg_match`, `preg_filter`, `slugify` (`replace`) and of the conditions (`match_regexp`, `not_match_regexp` of `filter`, `column_aggregator`, `array_filter`, `unset`): an invalid pattern is rejected when the options are resolved (`InvalidOptionsException`), instead of a PHP warning with the first value then "no match"; a PCRE error while matching (invalid UTF-8 with the `u` modifier, backtracking limit) throws an `UnexpectedValueException`, instead of being taken for "no match" (both `match_regexp` and `not_match_regexp` excluded the input). Update documentation, add tests.
* [#410](https://github.com/cleverage/process-bundle/issues/410) `xpath_evaluator`: the queries are evaluated with `\DOMXPath::evaluate()`, an expression that does not return nodes (`count(...)`, `string(...)`, a comparison) returns its value (`float`, `string`, `bool`) instead of `null`; new `namespaces` option (prefix => URI) to query the elements of a namespace, including the default namespace of a document (documented). Documentation: the `mb_*` functions in the examples of `callback`, `generic_transformers` and `array_map`. Add tests.
* [#402](https://github.com/cleverage/process-bundle/issues/402) `date_parser`: a date or a time out of range (`2024-02-31`, `25:61`), parsed by `\DateTime::createFromFormat()` with a warning and overflowing to the next month or day, throws an `UnexpectedValueException`, as documented for a value that cannot be parsed. A scalar (e.g. an `int` timestamp with the `U` format) or `\Stringable` input is cast to string, an array or another object throws an `UnexpectedValueException` instead of a `TypeError`. Update documentation, add tests.
* [#398](https://github.com/cleverage/process-bundle/issues/398) Transformers edge cases. `trim`: a `charlist` with multibyte characters is applied with `mb_trim()` (it produced invalid UTF-8). `convert_value`: with `keep_missing` and `auto_cast`, the input itself is returned, as documented, not its value cast to string. `multi_replace`: the numeric keys of `replace_mapping` no longer throw a `TypeError`, and a replacement that is not a scalar is rejected with the configuration. Conditions (`filter`, `column_aggregator`, `array_filter`, `unset`): the list form of `empty` / `not_empty` (`not_empty: ['[email]']`), described by the documentation, and the numeric property paths no longer throw a `TypeError`. `hash`: an array or a non-`Stringable` object throws an `UnexpectedValueException` (an array was hashed as the string `Array`). The empty `delimiter` of `explode`, an invalid `format` of `sprintf` and an unknown `class` of `instantiate` are rejected with the configuration, not with the first value. Update documentation, add tests.
Expand Down
5 changes: 3 additions & 2 deletions docs/reference/traits/condition_trait.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@ Provide a configurable set of matching rules, used by tasks and transformers to
| `match_regexp` | `array` | | `[]` | `path => pattern`: the value at `path`, cast to string, must match the regular expression |
| `not_match_regexp` | `array` | | `[]` | `path => pattern`: the value at `path`, cast to string, must not match the regular expression |

An invalid regular expression makes both `match_regexp` and `not_match_regexp` fail. In debug mode, the PHP warning
of `preg_match()` is converted into an exception by the Symfony error handler: the task fails.
An invalid regular expression is rejected with an `InvalidOptionsException` when the options are resolved. A PCRE
error while matching a value (e.g. invalid UTF-8 with the `u` modifier, backtracking limit reached) throws an
`UnexpectedValueException` with the PCRE error message, instead of being taken for "no match".

## Usage

Expand Down
6 changes: 5 additions & 1 deletion docs/reference/transformers/preg_filter_transformer.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,11 @@ Any value that can be cast to string.
Possible outputs
----------------

`string|null`: the replaced string, or `null` if no pattern matches (or on regex error).
`string|null`: the replaced string, or `null` if no pattern matches.

An invalid regular expression is rejected with an `InvalidOptionsException` when the options are resolved. A PCRE
error while matching (e.g. invalid UTF-8 with the `u` modifier, backtracking limit reached) throws an
`UnexpectedValueException` with the PCRE error message, instead of being taken for "no match".

Options
-------
Expand Down
4 changes: 4 additions & 0 deletions docs/reference/transformers/preg_match_transformer.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ Possible outputs
`preg_match`)
* `null` if the input is `null` or an empty string

An invalid regular expression is rejected with an `InvalidOptionsException` when the options are resolved. A PCRE
error while matching (e.g. invalid UTF-8 with the `u` modifier, backtracking limit reached) throws an
`UnexpectedValueException` with the PCRE error message, instead of being taken for "no match".

Options
-------

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/transformers/slugify_transformer.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Options
| Code | Type | Required | Default | Description |
|------------------|----------|:--------:|------------------------------------------|-----------------------------------------------------------------------------------------|
| `transliterator` | `string` | | `'NFD; [:Nonspacing Mark:] Remove; NFC'` | Transliterator identifier, passed to `\Transliterator::create()` (see Notes) |
| `replace` | `string` | | `'/[^a-z0-9]+/'` | Regular expression of the characters to replace (applied on the lowercased string) |
| `replace` | `string` | | `'/[^a-z0-9]+/'` | Regular expression of the characters to replace (applied on the lowercased string), checked with the configuration |
| `separator` | `string` | | `'_'` | Replacement string, also trimmed from both ends of the result |
| `strip_tags` | `bool` | | `true` | Strip the HTML tags with `strip_tags()`, which removes everything after a `<` that does not open a tag (`'Tom<Jerry'` becomes `'tom'`, `'<3 love'` becomes `''`): set it to `false` for a text that is not HTML |

Expand Down
18 changes: 18 additions & 0 deletions src/Transformer/ConditionTrait.php
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

namespace CleverAge\ProcessBundle\Transformer;

use Symfony\Component\OptionsResolver\Exception\InvalidOptionsException;
use Symfony\Component\OptionsResolver\Options;
use Symfony\Component\OptionsResolver\OptionsResolver;
use Symfony\Component\PropertyAccess\PropertyAccess;
Expand Down Expand Up @@ -107,6 +108,19 @@ protected function configureConditionOptions(OptionsResolver $resolver): void
$resolver->setAllowedTypes('empty', 'array');
$resolver->setAllowedTypes('not_match_regexp', 'array');
$resolver->setAllowedTypes('match_regexp', 'array');
// Rejected with the configuration: an invalid pattern made both match_regexp and not_match_regexp fail
foreach (['match_regexp', 'not_match_regexp'] as $option) {
$resolver->setNormalizer($option, static function (Options $options, array $value) use ($option): array {
foreach ($value as $pattern) {
if (!\is_string($pattern)) {
throw new InvalidOptionsException(\sprintf('The patterns of the "%s" option must be strings, %s given.', $option, get_debug_type($pattern)));
}
Regex::validate($pattern, $option);
}

return $value;
});
}
// The list form of empty / not_empty (e.g. not_empty: ['[email]']): the values are the property paths
foreach (['empty', 'not_empty'] as $option) {
$resolver->setNormalizer($option, static function (Options $options, array $value): array {
Expand Down Expand Up @@ -146,6 +160,10 @@ protected function checkValue(

if ($regexpMode) {
$pregMatch = preg_match($value, (string) $currentValue);
// A PCRE error (e.g. invalid UTF-8 with the "u" modifier) made both match_regexp and not_match_regexp fail
if (false === $pregMatch) {
Regex::checkLastError($value);
}

if ($shouldMatch && (false === $pregMatch || 0 === $pregMatch)) {
return false;
Expand Down
21 changes: 20 additions & 1 deletion src/Transformer/PregFilterTransformer.php
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,13 @@ public function transform(mixed $value, array $options = []): array|string|null
$pattern = $options['pattern'];
$replacement = $options['replacement'];

return preg_filter($pattern, $replacement, (string) $value);
$result = preg_filter($pattern, $replacement, (string) $value);
// null is also "no match": a PCRE error (e.g. invalid UTF-8 with the "u" modifier) is not
if (null === $result) {
Regex::checkLastError(\is_array($pattern) ? implode('", "', $pattern) : $pattern);
}

return $result;
}

/**
Expand All @@ -49,6 +55,19 @@ public function configureOptions(OptionsResolver $resolver): void
$resolver->setRequired(['pattern', 'replacement']);
$resolver->setAllowedTypes('pattern', ['string', 'array']);
$resolver->setAllowedTypes('replacement', ['string', 'array']);
$resolver->setNormalizer(
'pattern',
static function (Options $options, string|array $value): string|array {
foreach ((array) $value as $pattern) {
if (!\is_string($pattern)) {
throw new InvalidOptionsException(\sprintf('The patterns of the "pattern" option must be strings, %s given.', get_debug_type($pattern)));
}
Regex::validate($pattern);
}

return $value;
}
);
$resolver->setNormalizer(
'replacement',
static function (Options $options, string|array $value): string|array {
Expand Down
61 changes: 61 additions & 0 deletions src/Transformer/Regex.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
<?php

declare(strict_types=1);

/*
* This file is part of the CleverAge/ProcessBundle package.
*
* Copyright (c) Clever-Age
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/

namespace CleverAge\ProcessBundle\Transformer;

use CleverAge\ProcessBundle\Exception\UnexpectedValueException;
use Symfony\Component\OptionsResolver\Exception\InvalidOptionsException;

/**
* Regular expressions of the options: an invalid pattern is rejected with the configuration, and a PCRE error (e.g.
* invalid UTF-8 with the "u" modifier, backtracking limit) is not taken for "no match".
*
* @internal
*/
final class Regex
{
/**
* @throws InvalidOptionsException if the pattern cannot be compiled
*/
public static function validate(string $pattern, string $option = 'pattern'): string
{
$error = null;
set_error_handler(static function (int $type, string $message) use (&$error): bool {
$error = $message;

return true;
});
try {
$valid = false !== preg_match($pattern, '');
} finally {
restore_error_handler();
}
if (!$valid) {
$message = null !== $error ? preg_replace('/^preg_match\(\): /', '', $error) : preg_last_error_msg();

throw new InvalidOptionsException(\sprintf('Invalid regular expression "%s" in the "%s" option: %s', $pattern, $option, $message));
}

return $pattern;
}

/**
* @throws UnexpectedValueException if the last PCRE function has failed
*/
public static function checkLastError(string $pattern): void
{
if (\PREG_NO_ERROR !== preg_last_error()) {
throw new UnexpectedValueException(\sprintf('The regular expression "%s" has failed: %s', $pattern, preg_last_error_msg()));
}
}
}
11 changes: 9 additions & 2 deletions src/Transformer/String/PregMatchTransformer.php
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
namespace CleverAge\ProcessBundle\Transformer\String;

use CleverAge\ProcessBundle\Transformer\ConfigurableTransformerInterface;
use CleverAge\ProcessBundle\Transformer\Regex;
use Symfony\Component\OptionsResolver\Options;
use Symfony\Component\OptionsResolver\OptionsResolver;

/**
Expand All @@ -35,9 +37,13 @@ public function transform(mixed $value, array $options = []): ?array
}

if ($options['mode_all']) {
preg_match_all($options['pattern'], (string) $value, $matches, $options['flags'], $options['offset']);
$result = preg_match_all($options['pattern'], (string) $value, $matches, $options['flags'], $options['offset']);
} else {
preg_match($options['pattern'], (string) $value, $matches, $options['flags'], $options['offset']);
$result = preg_match($options['pattern'], (string) $value, $matches, $options['flags'], $options['offset']);
}
// A PCRE error (e.g. invalid UTF-8 with the "u" modifier) is not "no match"
if (false === $result) {
Regex::checkLastError($options['pattern']);
}

return $matches;
Expand All @@ -55,6 +61,7 @@ public function configureOptions(OptionsResolver $resolver): void
{
$resolver->setRequired(['pattern']);
$resolver->setAllowedTypes('pattern', ['string']);
$resolver->setNormalizer('pattern', static fn (Options $options, string $pattern): string => Regex::validate($pattern));
$resolver->setDefault('flags', 0);
$resolver->setAllowedTypes('flags', ['int']);
$resolver->setDefault('offset', 0);
Expand Down
21 changes: 13 additions & 8 deletions src/Transformer/String/SlugifyTransformer.php
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

use CleverAge\ProcessBundle\Exception\UnexpectedValueException;
use CleverAge\ProcessBundle\Transformer\ConfigurableTransformerInterface;
use CleverAge\ProcessBundle\Transformer\Regex;
use Symfony\Component\OptionsResolver\Exception\InvalidOptionsException;
use Symfony\Component\OptionsResolver\Options;
use Symfony\Component\OptionsResolver\OptionsResolver;
Expand Down Expand Up @@ -47,15 +48,18 @@ public function transform(mixed $value, array $options = []): string
$string = strip_tags($string);
}

return trim(
(string) preg_replace(
$options['replace'],
(string) $options['separator'],
// The characters left by the transliterator may be multibyte
mb_strtolower(trim($string))
),
$options['separator']
$slug = preg_replace(
$options['replace'],
(string) $options['separator'],
// The characters left by the transliterator may be multibyte
mb_strtolower(trim($string))
);
if (null === $slug) {
// A PCRE error gave an empty slug
Regex::checkLastError($options['replace']);
}

return trim((string) $slug, $options['separator']);
}

/**
Expand All @@ -81,6 +85,7 @@ public function configureOptions(OptionsResolver $resolver): void
$resolver->setAllowedTypes('transliterator', ['string']);
$resolver->setAllowedTypes('replace', ['string']);
$resolver->setAllowedTypes('separator', ['string']);
$resolver->setNormalizer('replace', static fn (Options $options, string $pattern): string => Regex::validate($pattern, 'replace'));

$resolver->setNormalizer(
'transliterator',
Expand Down
45 changes: 45 additions & 0 deletions tests/Task/FilterTaskTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
#[\PHPUnit\Framework\Attributes\UsesClass(ContextualOptionResolver::class)]
#[\PHPUnit\Framework\Attributes\UsesClass(ProcessHistory::class)]
#[\PHPUnit\Framework\Attributes\UsesClass(ProcessState::class)]
#[\PHPUnit\Framework\Attributes\UsesClass(\CleverAge\ProcessBundle\Transformer\Regex::class)]
class FilterTaskTest extends TestCase
{
/**
Expand Down Expand Up @@ -200,4 +201,48 @@ public function testListFormOfEmptyAndNotEmpty(): void
$this->assertTrue($this->execute(['not_empty' => ['[email]']], ['email' => ''])->isSkipped());
$this->assertTrue($this->execute(['empty' => ['[deleted_at]']], ['deleted_at' => 'yesterday'])->isSkipped());
}

/**
* @return iterable<string, array{array<string, mixed>}>
*/
public static function provideInvalidRegexpOptions(): iterable
{
yield 'match_regexp' => [['match_regexp' => ['[name]' => '/(a/']]];
yield 'not_match_regexp' => [['not_match_regexp' => ['[name]' => '/(a/']]];
yield 'not a string' => [['match_regexp' => ['[name]' => 1]]];
}

/**
* An invalid pattern made both match_regexp and not_match_regexp fail: every input was filtered out.
*
* @param array<string, mixed> $options
*/
#[DataProvider('provideInvalidRegexpOptions')]
public function testInvalidRegexpIsRejected(array $options): void
{
$this->expectException(InvalidOptionsException::class);

$this->execute($options, ['name' => 'a']);
}

/**
* @return iterable<string, array{string}>
*/
public static function provideRegexpOptions(): iterable
{
yield 'match_regexp' => ['match_regexp'];
yield 'not_match_regexp' => ['not_match_regexp'];
}

/**
* A PCRE error made both match_regexp and not_match_regexp fail: the input was filtered out.
*/
#[DataProvider('provideRegexpOptions')]
public function testPcreErrorThrows(string $option): void
{
$this->expectException(\CleverAge\ProcessBundle\Exception\UnexpectedValueException::class);
$this->expectExceptionMessage('The regular expression "/a/u" has failed: Malformed UTF-8 characters');

$this->execute([$option => ['[name]' => '/a/u']], ['name' => "a\xff"]);
}
}
1 change: 1 addition & 0 deletions tests/Transformer/Array/ArrayFilterTransformerTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
use Symfony\Component\PropertyAccess\PropertyAccess;

#[\PHPUnit\Framework\Attributes\CoversClass(ArrayFilterTransformer::class)]
#[\PHPUnit\Framework\Attributes\UsesClass(\CleverAge\ProcessBundle\Transformer\Regex::class)]
class ArrayFilterTransformerTest extends TestCase
{
public function testWholeValueConditionOnScalars(): void
Expand Down
38 changes: 38 additions & 0 deletions tests/Transformer/PregFilterTransformerTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#[\PHPUnit\Framework\Attributes\CoversMethod(PregFilterTransformer::class, 'transform')]
#[\PHPUnit\Framework\Attributes\CoversMethod(PregFilterTransformer::class, 'configureOptions')]
#[\PHPUnit\Framework\Attributes\CoversMethod(PregFilterTransformer::class, 'getCode')]
#[\PHPUnit\Framework\Attributes\UsesClass(\CleverAge\ProcessBundle\Transformer\Regex::class)]
class PregFilterTransformerTest extends TestCase
{
public function testTransformWithStringReplacement(): void
Expand Down Expand Up @@ -86,4 +87,41 @@ private function resolveOptions(PregFilterTransformer $transformer, array $optio

return $resolver->resolve($options);
}

/**
* @return iterable<string, array{string|array<mixed>}>
*/
public static function provideInvalidPatterns(): iterable
{
yield 'string' => ['/(a/'];
yield 'array' => [['/a/', '/(a/']];
yield 'not a string' => [['/a/', 1]];
}

/**
* Rejected with the configuration (a PHP warning with the first value, then null as for "no match").
*
* @param string|array<mixed> $pattern
*/
#[\PHPUnit\Framework\Attributes\DataProvider('provideInvalidPatterns')]
public function testInvalidPatternIsRejected(string|array $pattern): void
{
$this->expectException(InvalidOptionsException::class);

$this->resolveOptions(new PregFilterTransformer(), ['pattern' => $pattern, 'replacement' => 'b']);
}

/**
* A PCRE error was taken for "no match": null.
*/
public function testPcreErrorThrows(): void
{
$transformer = new PregFilterTransformer();
$options = $this->resolveOptions($transformer, ['pattern' => ['/a/u', '/b/u'], 'replacement' => 'c']);

$this->expectException(\CleverAge\ProcessBundle\Exception\UnexpectedValueException::class);
$this->expectExceptionMessage('The regular expression "/a/u", "/b/u" has failed: Malformed UTF-8 characters');

$transformer->transform("a\xff", $options);
}
}
Loading
Loading