tokenizers
PyPIactive
tokenizers last released 0.23.3 on 2026-10-09. Its latest version requires Python >=3.10.
Facts
- Latest release
- 0.23.3, (15 days ago)
- First published
- (6 years ago)
- Releases in the last 12 months
- 9
- Median days between releases
- 18
- Median days between major versions
- 2,515
- Monthly downloads
- 166,389,324
- License
- Apache Software License
- Repository
- github.com/huggingface/tokenizers
- Source
- pypi.org (checked 2026-10-19)
Major versions
2 major versions have been published.
| Major | First version | First released | Latest version | Latest released |
|---|---|---|---|---|
| 1 | 1.0.0rc1 | 2026-09-21 | 1.0.0rc2 | 2026-09-21 |
| 0 | 0.0.1 | 2019-11-01 | 0.23.3 | 2026-10-09 |
Runtime requirements
The latest version of tokenizers declares these runtime constraints.
| Runtime | Constraint | Minimum version | Since version | Since |
|---|---|---|---|---|
| Python | >=3.10 | 3.10 | 0.0.1 | 2019-11-01 |
Supported runtimes
The latest version of tokenizers runs on 6 tracked runtime releases.
Keelbench tracks end-of-life dates for 469 products, updated daily. Check your repositories against these dates with Deps.