Latest | 1.1.0 |
---|---|
Homepage | https://github.com/Meniny/Tokenizer |
License | MIT |
Platforms | ios 8.0, osx 10.10, tvos 9.0, watchos 2.0 |
Frameworks | Foundation |
Authors |
Introduction
What’s this?
Tokenizer
is a tiny human language tokenizer framework in Swift..
Preview
Installation
CocoaPods
pod 'Tokenizer'
Contribution
You are welcome to fork and submit pull requests.
License
Tokenizer
is open-sourced software, licensed under the MIT
license.
Usage
import Tokenizer
let stackoverflow = "Stack Overflow is the largest, most trusted online community for developers to learn, share their programming knowledge, and build their careers."
let baidu = "全球最大的中文搜索引擎、最大的中文网站。1999年底,身在美国硅谷的李彦宏看到了中国互联网及中文搜索引擎服务的巨大发展潜力,抱着技术改变世界的梦想,他毅然辞掉硅谷的高薪工作,携搜索引擎专利技术,于 2000年1月1日在中关村创建了百度公司。"
dump(stackoverflow.tokenize())
dump(Tokenizer.tokenize(text: baidu))
Latest podspec
{ "name": "Tokenizer", "version": "1.1.0", "summary": "A tiny human language tokenizer framework in Swift.", "description": "Tokenizer is a tiny human language tokenizer framework in Swift..", "homepage": "https://github.com/Meniny/Tokenizer", "license": "MIT", "authors": { "Elias Abel": "[email protected]" }, "source": { "git": "https://github.com/Meniny/Tokenizer.git", "tag": "1.1.0" }, "social_media_url": "https://meniny.cn/", "platforms": { "ios": "8.0", "osx": "10.10", "tvos": "9.0", "watchos": "2.0" }, "source_files": "Tokenizer/**/*.{h,swift}", "public_header_files": "Tokenizer/**/*{.h}", "frameworks": "Foundation", "pushed_with_swift_version": "4.0" }
Thu, 28 Dec 2017 16:00:06 +0000