Swift Package Index

Packages for keyword “tokenizer”

These packages are available as a package collection, usable in Xcode or SwiftPM.

• swift-casification

  Package for tokenizing and case swapping for free strings

  • CaptureContext/swift-casification
  • Active 2 months ago
  • 0 stars
  • Has documentation

• kaleidoscope-lexer

  A Lexer Named Kaleidoscope, powered by Swift Macro

  • FlickerSoul/kaleidoscope-lexer
  • Active 2 months ago
  • 6 stars
  • Has documentation

• GPTEncoder

  Swift BPE Encoder/Decoder for OpenAI GPT Models. A programmatic interface for tokenizing text for OpenAI ChatGPT API.

  • alfianlosari/GPTEncoder
  • Active 3 years ago
  • 86 stars

• TiktokenSwift

  Swift bindings for OpenAI's tiktoken tokenizer using UniFFI. Count tokens, estimate costs, and manage context windows in your iOS and macOS apps

  • narner/TiktokenSwift
  • Active 8 months ago
  • 25 stars

• swift-tiktoken

  A pure Swift implementation of OpenAI's tiktoken tokenizer

  • DePasqualeOrg/swift-tiktoken
  • Active 4 months ago
  • 3 stars

5 packages.