relevanceai.operations_new.processing.feature_extractor#

Module Contents#

class relevanceai.operations_new.processing.feature_extractor.FeatureExtractor(model: torch.nn.Module, layer_name: Optional[str] = None)#
save_outputs_hook(self, layer_name)#

It returns a function that takes 3 arguments (inputs, outputs, and a layer name) and assigns the output to the layer name in the _features dictionary

Parameters

layer_name – The name of the layer to save the output of.

Return type

A function that takes 3 arguments.

forward(self, x: Union[torch.Tensor, List[Any]]) List[Any]#

The function takes in an input tensor x and passes it through the model. The output of the model is not used, but the output of the model’s features is returned

Parameters

x – input tensor

Return type

The features of the model

class relevanceai.operations_new.processing.feature_extractor.EmbeddingsExtractor(model: torch.nn.Module)#
forward(self, tokens: Union[torch.Tensor, List[int]], embedding_layer: Optional[str] = None) List[List[float]]#

It takes a list of tokens, and returns a list of embeddings for those tokens

Parameters
  • tokens (List[int]) – List[int]

  • embedding_layer (Optional[str]) – The name of the embedding layer to use.

Return type

A list of lists of floats.