Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Apply Unicode normalization to avoid trouble with Umlauts #403

Closed
annakasprzik opened this issue Jun 5, 2019 · 3 comments
Closed

Apply Unicode normalization to avoid trouble with Umlauts #403

annakasprzik opened this issue Jun 5, 2019 · 3 comments
Labels
bug Something is broken
Milestone
0.9.2

Comments

@annakasprzik
Copy link

Search terms containing Umlauts are truncated at the "a/o/u", see screenshot
Antipa
@annakasprzik
Copy link
Author

Also, search terms with Umlaut are not highlighted when there is an exact string match.

@nichtich nichtich transferred this issue from gbv/coli-conc.gbv.de Jun 6, 2019
@stefandesu stefandesu mentioned this issue Jun 6, 2019
Closed
@stefandesu
Copy link
Member

After further investigation, I've found out that the Umlaut that the GND API returns seem to be non-standard. A normal ä is encoded as %C3%A4, while the ä from GND (actually ä which is a different character) is encoded as a%CC%88 (so I guess it's an a and the dots separately). That's also why it cuts off at a because it is separate from the dots (but if together, it is shown as ä).

I'll think about a way to deal with this, probably some kind of unicode normalization.

@nichtich
Copy link
Member

nichtich commented Jun 6, 2019

All incoming JSON data should be normalized to NFC as specified in JSKOS spec.

@nichtich nichtich changed the title cocoda seems to have trouble with Umlaut Apply Unicode normalization to avoid trouble with Umlauts Jun 6, 2019
@stefandesu stefandesu added this to the 0.9.2 milestone Jun 6, 2019
@DennisTobola DennisTobola added the workflow affects the mapping workflow label Jun 6, 2019
@stefandesu stefandesu added bug Something is broken and removed workflow affects the mapping workflow labels Jun 6, 2019
stefandesu added a commit to gbv/jskos-tools that referenced this issue Jun 7, 2019
@stefandesu
Add method for unicode normalization
ce3d90e 
Came up in gbv/cocoda#403.
stefandesu added a commit that referenced this issue Jun 7, 2019
@stefandesu
Apply unicode normalization to incoming data (#403)
8663db7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something is broken
Projects
None yet
Milestone
0.9.2
Development

No branches or pull requests
4 participants
@nichtich @stefandesu @annakasprzik @DennisTobola