tokenizers
This package is deprecated. PyPI now publishes newer versions of this package for riscv64, and we will no longer maintain this package. Please use the version from PyPI instead.
If you need a specific version between the latest available here and the first available on PyPI, please open an Issue and select
Package build request.
Source code: https://github.com/huggingface/tokenizers
pip install tokenizers --index-url https://pypi.riseproject.dev/simple/
License: Apache 2.0
Download files: https://gitlab.com/riseproject/python/wheel_builder/-/packages/51636794
pip install tokenizers==0.22.1 --index-url https://pypi.riseproject.dev/simple/
License: Apache 2.0
Download files: https://gitlab.com/riseproject/python/wheel_builder/-/packages/48310436
pip install tokenizers==0.22.0 --index-url https://pypi.riseproject.dev/simple/
License: Apache 2.0
Download files: https://gitlab.com/riseproject/python/wheel_builder/-/packages/48310477
pip install tokenizers==0.21.1 --index-url https://pypi.riseproject.dev/simple/
License: Apache 2.0
Download files: https://gitlab.com/riseproject/python/wheel_builder/-/packages/36897711
pip install tokenizers==0.21.0 --index-url https://pypi.riseproject.dev/simple/
License: Apache 2.0
Download files: https://gitlab.com/riseproject/python/wheel_builder/-/packages/36897719
pip install tokenizers==0.20.3 --index-url https://pypi.riseproject.dev/simple/
License: Apache 2.0
Download files: https://gitlab.com/riseproject/python/wheel_builder/-/packages/36897733
License file included in the wheel.
test_tutorial_train_from_iterators.py disabled due to missing dependency.