TokenizerAnnotator

CoreNLP/simple/annotator. TokenizerAnnotator

Class representing an TokenizerAnnotator. Identifies Tokens

Constructor

new TokenizerAnnotator(optionsopt)

Source:
See:

Create an Annotator

Parameters:
Name Type Attributes Description
options Object <optional>

a key-value map of options, without the annotator prefix

Requires:
  • module:tokenize

Extends

  • Annotator

Requires

  • module:tokenize