Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Latest
* [#392](https://github.com/cleverage/process-bundle/issues/392) Documentation: reset of the process configurations between two messages of a worker (`kernel.reset`, #355), permissions of the JSON buffer of `process_launcher`, `ProcessManager` subclasses in `UPGRADE.md` (v5.5), broken link in the CHANGELOG.

## Fixes
* [#400](https://github.com/cleverage/process-bundle/issues/400) `slugify`: new `strip_tags` option (`true` by default) to keep a text containing a `<` that is not HTML (`strip_tags()` removed everything after it); the result is lowercased with `mb_strtolower()`; a scalar or `\Stringable` input is cast to string, `null`, an array or another object throws an `UnexpectedValueException` instead of a `TypeError`. Documentation: the default transliterator removes the letters that are not decomposed (`Cœur` becomes `c_ur`), `Any-Latin; Latin-ASCII` is recommended. Add tests.
* [#396](https://github.com/cleverage/process-bundle/issues/396) A failure of `finalize()` on a process that has succeeded makes the process fail with a `ProcessFailedException`, once every task is finalized: the process was flagged and logged as failed, but ended successfully (`cleverage_process.end` event, the command exiting with `0`, a parent process going on). `ProcessLauncherTask` is finalizable: the sub-processes still running when the process fails or is stopped are stopped (a warning is logged) and the JSON buffers not output yet are removed, instead of going on with their side effects after the end of the process. Update documentation, add tests.
* [#394](https://github.com/cleverage/process-bundle/issues/394) `ProcessManager::execute()`: an exception thrown by a listener of `cleverage_process.fail` (`ProcessEvent::EVENT_PROCESS_FAILED`) is logged as an error and no longer replaces the error of the process, which is rethrown to the command, the parent process or Messenger. The debug message "has error outputs but it's error strategy 'stop' implies they will never be reached" is removed: the error branch of a task with the `stop` strategy is executed before the process stops. Update documentation, add tests.
* [#388](https://github.com/cleverage/process-bundle/issues/388) The temporary files of `file_splitter`, `csv_splitter` and of the JSON buffer of `process_launcher` (#343) are created with the `0077` umask: they are no longer readable by other users before their permissions are set to `0600`. Add tests.
Expand Down
11 changes: 9 additions & 2 deletions docs/reference/transformers/slugify_transformer.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ SlugifyTransformer

Convert a string into a slug. The value is transliterated with a
[\Transliterator](https://www.php.net/manual/en/class.transliterator.php) (by default, accents are removed), HTML
tags are stripped, the result is trimmed and lowercased, every sequence of characters matching `replace` is replaced by
tags are stripped (unless `strip_tags` is `false`), the result is trimmed and lowercased (multibyte characters included), every sequence of characters matching `replace` is replaced by
`separator`, and leading and trailing separators are removed.

Requires the `intl` PHP extension (suggested by the bundle): without it, the transformer fails with an explicit message.
Expand All @@ -17,7 +17,8 @@ Transformer reference
Accepted inputs
---------------

`string`
`string`, or a scalar or `\Stringable` cast to string. `null`, an array or another object throws an
`UnexpectedValueException`.

Possible outputs
----------------
Expand All @@ -32,6 +33,7 @@ Options
| `transliterator` | `string` | | `'NFD; [:Nonspacing Mark:] Remove; NFC'` | Transliterator identifier, passed to `\Transliterator::create()` (see Notes) |
| `replace` | `string` | | `'/[^a-z0-9]+/'` | Regular expression of the characters to replace (applied on the lowercased string) |
| `separator` | `string` | | `'_'` | Replacement string, also trimmed from both ends of the result |
| `strip_tags` | `bool` | | `true` | Strip the HTML tags with `strip_tags()`, which removes everything after a `<` that does not open a tag (`'Tom<Jerry'` becomes `'tom'`, `'<3 love'` becomes `''`): set it to `false` for a text that is not HTML |

Examples
--------
Expand All @@ -55,6 +57,11 @@ slugify:
Notes
-----

The default transliterator only removes the accents (the combining marks): the letters that are not decomposed into a
base letter and an accent are then removed by the default `replace` pattern (`'Cœur'` becomes `'c_ur'`, `'Straße'`
`'stra_e'`, `'Łódź'` `'odz'`), and a text without latin letters gives `''`. `'Any-Latin; Latin-ASCII'` is recommended:
it transliterates them (`'coeur'`, `'strasse'`, `'lodz'`) and the other scripts too (see the example above).

An invalid `transliterator` identifier (rejected by `\Transliterator::create()`), or an option that is not a string,
raises an `InvalidOptionsException` when the options are resolved, i.e. when the transformer is configured, not on the
first transformed value.
Expand Down
15 changes: 14 additions & 1 deletion src/Transformer/String/SlugifyTransformer.php
Original file line number Diff line number Diff line change
Expand Up @@ -31,18 +31,28 @@ class SlugifyTransformer implements ConfigurableTransformerInterface
*/
public function transform(mixed $value, array $options = []): string
{
if (\is_int($value) || \is_float($value) || \is_bool($value) || $value instanceof \Stringable) {
$value = (string) $value;
} elseif (!\is_string($value)) {
throw new UnexpectedValueException(\sprintf('The slugify transformer expects a string, %s given.', get_debug_type($value)));
}
/** @var \Transliterator $transliterator */
$transliterator = $options['transliterator'];
$string = $transliterator->transliterate($value);
if (false === $string) {
throw new UnexpectedValueException("Unable to transliterate the value: {$transliterator->getErrorMessage()}");
}

if ($options['strip_tags']) {
$string = strip_tags($string);
}

return trim(
(string) preg_replace(
$options['replace'],
(string) $options['separator'],
strtolower(trim(strip_tags($string)))
// The characters left by the transliterator may be multibyte
mb_strtolower(trim($string))
),
$options['separator']
);
Expand All @@ -63,8 +73,11 @@ public function configureOptions(OptionsResolver $resolver): void
'transliterator' => 'NFD; [:Nonspacing Mark:] Remove; NFC',
'replace' => '/[^a-z0-9]+/',
'separator' => '_',
// strip_tags() removes everything after a "<" (e.g. "Tom<Jerry" gives "tom")
'strip_tags' => true,
]
);
$resolver->setAllowedTypes('strip_tags', ['bool']);
$resolver->setAllowedTypes('transliterator', ['string']);
$resolver->setAllowedTypes('replace', ['string']);
$resolver->setAllowedTypes('separator', ['string']);
Expand Down
78 changes: 78 additions & 0 deletions tests/Transformer/String/SlugifyTransformerTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ public static function provideInvalidOptionTypes(): iterable
yield 'transliterator' => [['transliterator' => 123]];
yield 'replace' => [['replace' => ['/a/']]];
yield 'separator' => [['separator' => []]];
yield 'strip_tags' => [['strip_tags' => 'yes']];
}

/**
Expand Down Expand Up @@ -88,6 +89,83 @@ public function testValueThatCannotBeTransliteratedThrows(): void
$transformer->transform("a\xff", $this->resolveOptions($transformer));
}

/**
* The default transliterator only removes the accents: the letters that are not decomposed are removed with the
* replace pattern. Any-Latin; Latin-ASCII keeps them.
*/
public function testLettersWithoutDecomposition(): void
{
$transformer = new SlugifyTransformer();

$this->assertSame('c_ur_stra_e_odz', $transformer->transform('Cœur Straße Łódź', $this->resolveOptions($transformer)));
$this->assertSame('coeur_strasse_lodz', $transformer->transform('Cœur Straße Łódź', $this->resolveOptions($transformer, [
'transliterator' => 'Any-Latin; Latin-ASCII',
])));
}

/**
* strip_tags() removes everything after a "<".
*/
public function testStripTags(): void
{
$transformer = new SlugifyTransformer();

$this->assertSame('tom', $transformer->transform('Tom<Jerry', $this->resolveOptions($transformer)));
$this->assertSame('tom_jerry', $transformer->transform('Tom<Jerry', $this->resolveOptions($transformer, ['strip_tags' => false])));
$this->assertSame('3_love', $transformer->transform('<3 love', $this->resolveOptions($transformer, ['strip_tags' => false])));
}

/**
* The characters left by the transliterator are lowercased too (strtolower() only lowercased ASCII letters).
*/
public function testMultibyteLowercase(): void
{
$transformer = new SlugifyTransformer();
$options = $this->resolveOptions($transformer, ['transliterator' => 'NFC', 'replace' => '/[^\p{L}0-9]+/u']);

$this->assertSame('élan_ça', $transformer->transform('ÉLAN ÇA', $options));
}

public function testScalarAndStringableValues(): void
{
$transformer = new SlugifyTransformer();
$options = $this->resolveOptions($transformer);
$stringable = new class implements \Stringable {
public function __toString(): string
{
return 'Hélène';
}
};

$this->assertSame('123', $transformer->transform(123, $options));
$this->assertSame('1_5', $transformer->transform(1.5, $options));
$this->assertSame('helene', $transformer->transform($stringable, $options));
}

/**
* @return iterable<string, array{mixed}>
*/
public static function provideInvalidValues(): iterable
{
yield 'null' => [null];
yield 'array' => [['a']];
yield 'object' => [new \stdClass()];
}

/**
* A raw TypeError was thrown.
*/
#[\PHPUnit\Framework\Attributes\DataProvider('provideInvalidValues')]
public function testInvalidValues(mixed $value): void
{
$transformer = new SlugifyTransformer();

$this->expectException(\UnexpectedValueException::class);
$this->expectExceptionMessage('The slugify transformer expects a string, '.get_debug_type($value).' given.');

$transformer->transform($value, $this->resolveOptions($transformer));
}

/**
* @param array<string, mixed> $options
*
Expand Down
Loading