[Tokenizers] Port LLaMA Tokenizer and SentencePiece algorithm

[ericstj]

The SentencePiece algorithm should be added to Microsoft.ML.Tokenizers. This is a dependency of LLaMATokenizer which we also wish to enable.

We can see reference implementations in
https://github.com/microsoft/BlingFire (MIT license)
https://github.com/google/sentencepiece (Apache license)
https://github.com/huggingface/tokenizers (Apache license)
https://huggingface.co/docs/transformers/main/en/model_doc/llama

Hugging face also has Llama2 - might be interesting to understand if that's also worth including or designing for later inclusion.

LLaMA Tokenizer:
https://arxiv.org/abs/2203.13474
https://arxiv.org/pdf/2203.13474.pdf

Sentence Piece:
https://arxiv.org/abs/1808.06226
https://arxiv.org/pdf/1808.06226.pdf

[ericstj]

SentencePiece has tokenizers/normalizers/model-file structure. We only require a subset of this (Tokenizer).

We will port this subset and make it work with the data sources used by SentencePiece BPE tokenizer

[ericstj]

#7078