A PHP implementation of OpenAI's BPE tokenizer tiktoken.
-
Updated
Jun 16, 2024 - PHP
A PHP implementation of OpenAI's BPE tokenizer tiktoken.
PHP port for openai/tiktoken (most)
Add a description, image, and links to the tiktoken topic page so that developers can more easily learn about it.
To associate your repository with the tiktoken topic, visit your repo's landing page and select "manage topics."