Text Classification Instances

These Instances are designed for any classification task over a single passage of text. The input is the passage (e.g., a sentence, a document, etc.), and the output is a single label (e.g., positive / negative sentiment, spam / not spam, essay grade, etc.).

TextClassificationInstances

class deep_qa.data.instances.text_classification.text_classification_instance.IndexedTextClassificationInstance(word_indices: typing.List[int], label, index: int = None)

Bases: deep_qa.data.instances.instance.IndexedInstance

as_training_data()

Convert this IndexedInstance to NumPy arrays suitable for use as training data to Keras models.

Returns:

train_data : (inputs, label)

The IndexedInstance as NumPy arrays to be uesd in Keras. Note that inputs might itself be a complex tuple, depending on the Instance type.

classmethod empty_instance()

Returns an empty, unpadded instance of this class. Necessary for option padding in multiple choice instances.

get_padding_lengths() → typing.Dict[str, int]

Returns the length of this instance in all dimensions that require padding.

Different kinds of instances have different fields that are padded, such as sentence length, number of background sentences, number of options, etc.

Returns:

padding_lengths: Dict[str, int]

A dictionary mapping padding keys (like “num_sentence_words”) to lengths.

pad(padding_lengths: typing.Dict[str, int])

Add zero-padding to make each data example of equal length for use in the neural network.

This modifies the current object.

Parameters:

padding_lengths: Dict[str, int]

In this dictionary, each str refers to a type of token (e.g. num_sentence_words), and the corresponding int is the value. This dictionary must have the same keys as was returned by get_padding_lengths(). We will use these lengths to pad the instance in all of the necessary dimensions to the given leangths.

class deep_qa.data.instances.text_classification.text_classification_instance.TextClassificationInstance(text: str, label: bool, index: int = None)

Bases: deep_qa.data.instances.instance.TextInstance

A TextClassificationInstance is a TextInstance that is a single passage of text, where that passage has some associated (categorical, or possibly real-valued) label.

classmethod read_from_line(line: str)

Reads a TextClassificationInstance object from a line. The format has one of four options:

1. [sentence]
2. [sentence index][tab][sentence]
3. [sentence][tab][label]
4. [sentence index][tab][sentence][tab][label]

If no label is given, we use None as the label.

to_indexed_instance(data_indexer: deep_qa.data.data_indexer.DataIndexer)

Converts the words in this Instance into indices using the DataIndexer.

Parameters:

data_indexer : DataIndexer

DataIndexer to use in converting the Instance to an IndexedInstance.

Returns:

indexed_instance : IndexedInstance

A TextInstance that has had all of its strings converted into indices.

words() → typing.Dict[str, typing.List[str]]

Returns a list of all of the words in this instance, contained in a namespace dictionary.

This is mainly used for computing word counts when fitting a word vocabulary on a dataset. The namespace dictionary allows you to have several embedding matrices with different vocab sizes, e.g., for words and for characters (in fact, words and characters are the only use cases I can think of for now, but this allows you to do other more crazy things if you want). You can call the namespaces whatever you want, but if you want the DataIndexer to work correctly without namespace arguments, you should use the key ‘words’ to represent word tokens.

Returns:

namespace : Dictionary of {str: List[str]}

The str key refers to vocabularies, and the List[str] should contain the tokens in that vocabulary. For example, you should use the key words to represent word tokens, and the correspoding value in the dictionary would be a list of all the words in the instance.