Permalink: Slug input field strips unicode characters

[LuckynaSan]

Bug Description

The permalink slug field does not show foreign language special characters when using it as a title or when entering it in the slug field.

AMP Bug (out of scope for this project)

The story bookend is being displayed with HTML markup when the story title or permalink slug contains language special characters.

Example story URL: https://feignednorth.s4-tastewp.com/web-stories/%e0%b0%87%e0%b0%82%e0%b0%a1%e0%b0%bf%e0%b0%af%e0%b0%be-%e0%b0%b8%e0%b0%bf%e0%b0%ae%e0%b1%86%e0%b0%82%e0%b0%9f%e0%b1%8d%e0%b0%b8%e0%b1%8d-%e0%b0%aa%e0%b1%88%e0%b0%a8-%e0%b0%95%e0%b1%87%e0%b0%b8/

Expected Behaviour

The slug should display the language characters.

Steps to Reproduce

1. Create a new story
2. Use “ఇండియా-సిమెంట్స్-పైన-కేస” as title
3. Publish the story.
4. Note Permalink Slug field.

-> Compare the same with Gutenberg where this just works.

Screenshots

Additional Context

Support Topic: https://wordpress.org/support/topic/doesnt-tech-url

• Plugin Version: 1.25.1
• WordPress Version:
• Operating System: macOS
• Browser: Chrome

[swissspidy]

The issue with the permalink field is something we can fix, the other part is not in our hands and needs to be reported in the amphtml repo.

[LuckynaSan]

I have opened the other part here: ampproject/amphtml#38510

[swissspidy]

Gutenberg nowadays uses the MIT remove-accents package. We can potentially use that one as well.

Also, I think we should use String.prototype.normalize() with NFC, not NFD

We can also use replaceAll() to make the cleanForSlug function more readable