tokenized

tokenized

美 　英

• 网络标记化的；词法分析；符号化

例句

The contents of this file are tokenized into a list of strings and bound to a list called Rules.

文件的内容被分割放入一个字符串列表，然后绑定到名为Rules的变量。

Should be specified as the string of hexadecimal digits representing the tokenized form of the public key.

应指定为十六进制数字的字符串以表示标记形式的公钥。

In some specific cases , changes made to the word breakers have the potential to impact how some data is tokenized .

在某些特定情况下，对断字符所做的更改可能会影响对某些数据进行标记化的方式。

Erlang also gives us programmatic access to the tokenized, parsed and abstract forms of an expression.

Erlang还允许我们编程访问经过词法分析（tokenized）、语法分析之后、抽象形式的表达式。

The identified messages are tokenized , classified, and aggregated and changes in the frequency of tokenized messages are identified.

识别过的消息被做标记、分类和聚类处理，消息中标记的频率变化也被识别出来。

These considerations relate to how your text is tokenized and then indexed by Full-Text Engine.

这些注意事项均与全文引擎如何对文本进行词汇切分再编制其索引有关。

A display of the tokenized messages are identified.

标记消息通过用户界面显示。