Token Format

From Medialab

Processes exchange Token streams as discussed in Software Architecture.

The representation of Token is flexible, so that each process can extend it by adding its own particular features.

A Token consists in:

  • a mapping: FeaturesValues
  • a mapping: Relation → 〈TokenId, Label

where Features express specific features of a token and <Relatsion express relations by menas of labeled links to other tokens.

Tokens are the constituents of sentences (whose structure is specified in Sentence Format). Sentences are grouped into corpora whose format is described in Corpus Format.