Hacker News

Show HN: FlashTokenizer – 10x faster C++ tokenizer for Python

Hacker News - 4 hours 8 min ago

I built a tokenizer in C++ with a Python binding that outperforms HuggingFace tokenizers by 10x on large inputs. It's optimized for minimal memory usage and latency.

Benchmarks and comparison included in README. Would love feedback or contributions!

Comments URL: https://news.ycombinator.com/item?id=43565699

Points: 3

# Comments: 0

Categories: Hacker News

Raising an Agent

Hacker News - 4 hours 26 min ago

Article URL: https://ampcode.com/podcast

Comments URL: https://news.ycombinator.com/item?id=43565547

Points: 1

# Comments: 0

Categories: Hacker News

The Naming of Methods

Hacker News - 4 hours 50 min ago
Categories: Hacker News

Pages