SDAI
Toggle table of contents
Main site
Telegram
Discord
android
Target filter
android
Switch theme
Search in API
Skip to content
SDAI
feature/onnx
/
com.shifthackz.aisdv1.feature.onnx.ai.tokenizer
/
EnglishTextTokenizer
/
loadBpeRanks
load
Bpe
Ranks
private
fun
loadBpeRanks
(
)
:
Map
<
Pair
<
String
,
String
>
,
Int
?
>
(
source
)
Loads SDAI data through
loadBpeRanks
.
Return
Result produced by
loadBpeRanks
.
Author
Dmitriy Moroz