Required inputs and outputs

Integrating new code into the Indico platform only works if you know what your code will be getting as inputs and if the platform knows how to handle what your code will be producing for outputs.

When writing a blueprint, developers are required to pre-define inputs and outputs. Each type of custom blueprint (Model, Filter, and Output) requires specific inputs and outputs based on

  • the type of task the component performs
  • its place within a workflow

This article explains which inputs and outputs are required and which are available for different custom component types.

πŸ“˜

Task output types

When a task yields data objects, the workflows library automatically groups them and allows downstream tasks to read them in as PartitionStreams or FullPartitionStreams.

Model blueprint

Models take in documents and make predictions. Depending on their location in the workflow, models can also read in predictions from other models-- if Model B comes after Model A in the execution order, then Model B can optionally read in the predictions of Model A.

Inputs

NameData ObjectContainerRequired or Optional
documentsSpanGroupIOPartitionStream OR FullPartitionStreamRequired
predictionsLabelGroupIOPartitionStream OR FullPartitionStreamOptional

Outputs

NameData ObjectRequired or Optional
predictionsLabelGroupIORequired

Example Model Task

Here's an example of a custom Model task that reads in documents and outputs predictions:

Filter blueprint

The purpose of Filters is to divide documents into two or more categories. Each category is represented by a KeyChain.

Inputs

While the Indico platform does not restrict the inputs of a Filter component, the optional inputs below might be useful.

NameData ObjectContainerRequired or Optional
documentsSpanGroupIOPartitionStream OR FullPartitionStreamOptional
predictionsLabelGroupIOPartitionStream OR FullPartitionStreamOptional

Outputs

NameData ObjectRequired or Optional
documentsSpanGroupIOs wrapped in a KeyChainRequired

Example filter task

Output blueprint

The purpose of an Output component is to modify the result file. Because it is the last component in a workflow, an Output component can access any data outputted by the other Indico-native or custom components.

Inputs

NameData ObjectWrapperRequired or Optional
documentsSpanGroupIO or KeyChainPartitionStream or FullPartitionStreamOptional
predictionsLabelGroupIOPartitionStream or FullPartitionStreamOptional
result_fileIOValuePartitionStreamOptional

Outputs

NameData ObjectRequired or Optional
result_fileIOValueRequired

Example output task