Latest	1.1.0
Homepage	https://github.com/Meniny/Tokenizer
License	MIT
Platforms	ios 8.0, osx 10.10, tvos 9.0, watchos 2.0
Frameworks	Foundation
Authors	Elias Abel

Introduction

What’s this?

Tokenizer is a tiny human language tokenizer framework in Swift..

Preview

Installation

CocoaPods

pod 'Tokenizer'

Contribution

You are welcome to fork and submit pull requests.

License

Tokenizer is open-sourced software, licensed under the MIT license.

Usage

import Tokenizer

let stackoverflow = "Stack Overflow is the largest, most trusted online community for developers to learn, share their programming knowledge, and build their careers."
let baidu = "全球最大的中文搜索引擎、最大的中文网站。1999年底,身在美国硅谷的李彦宏看到了中国互联网及中文搜索引擎服务的巨大发展潜力，抱着技术改变世界的梦想，他毅然辞掉硅谷的高薪工作，携搜索引擎专利技术，于 2000年1月1日在中关村创建了百度公司。"

dump(stackoverflow.tokenize())
dump(Tokenizer.tokenize(text: baidu))

Latest podspec

{
    "name": "Tokenizer",
    "version": "1.1.0",
    "summary": "A tiny human language tokenizer framework in Swift.",
    "description": "Tokenizer is a tiny human language tokenizer framework in Swift..",
    "homepage": "https://github.com/Meniny/Tokenizer",
    "license": "MIT",
    "authors": {
        "Elias Abel": "[email protected]"
    },
    "source": {
        "git": "https://github.com/Meniny/Tokenizer.git",
        "tag": "1.1.0"
    },
    "social_media_url": "https://meniny.cn/",
    "platforms": {
        "ios": "8.0",
        "osx": "10.10",
        "tvos": "9.0",
        "watchos": "2.0"
    },
    "source_files": "Tokenizer/**/*.{h,swift}",
    "public_header_files": "Tokenizer/**/*{.h}",
    "frameworks": "Foundation",
    "pushed_with_swift_version": "4.0"
}

Thu, 28 Dec 2017 16:00:06 +0000