🔍 Introduction

Before adding your custom component to a workflow, you must register it using the Indico platform API. Registering is the process of letting other services in the cluster know that your blueprint exists and is ready to be used in a workflow.

The recipe below shows how to use the SDK to register a blueprint:

The API for registering a blueprint includes numerous arguments that allow you to customize how blueprints are displayed and run. Some arguments are optional; others are required. This article will cover how to specify

• 🖼 How the blueprint is displayed in the Gallery
• 🎛 How the IPA platform runs the component
• 🔐 Which workflows the blueprint can be added to

Prerequisites

This guide assumes your custom component code has already been packaged into a Docker container and deployed to your cluster. For more information on these steps, see Containerizing a Custom Component and Deploying a Custom Component.

📥 API Inputs

📥

Inputs

🖼 Configuring how the blueprint is display in the gallery

name: str Name of the blueprint

description: str Description of this blueprint

tags: list[str] List of tags associated with this blueprint

icon: str | None Image to use when displaying this blueprint in the gallery. Can be the name of an Indico-provided image, or the storage location of a custom image. If left unspecified, the platform will use a default icon.

footer: str | None Footnote for this blueprint's description; can be

🎛 Specifying the runtime configuration

component_family: ComponentFamilyFamily this blueprint belongs to; supported families are Output, Filter, and Model

config: dict Blueprint configuration options

🔐 Defining who can access the blueprint

all_access: bool This blueprint can be added to all workflows. Default is False.

dataset_ids: list[int] | None This blueprint can only be added to workflows associated with these dataset ids

🖼 Defining Blueprint Display

The name, icon, description, andfooterfields determine how your blueprint will be displayed in the gallery.

Using a Default Icon

If no value is provided for icon, or you use an empty string, the platform will use a default icon depending on the component type.

Default Icon	Component Type
	Model
	Filter
	Output

Using an Indico-provided icon

The platform offers some preexisting icons. To use these icons, set the icon to the pre-existing icon's name.

Icon	Name
	privateai-logo
	microsoft-form-recognizer-logo
	base64ai
	huggingface-logo

📤 Uploading a Custom Icon

To upload and use a custom icon:

1. Upload the icon to Indico using the UploadDocument API Call. The call outputs a file path.
2. Assemble the file URL by concatenating the Indico file prefix indico-file:///storage to the beginning of the file path.
3. Pass the URL to the icon field when registering your blueprint.

For more instructions on uploading an icon, see the recipe below

Tagging Custom Components

All custom components accept a list of tags. Indico requires that all custom components include custom in their list of tags

For custom models, users can filter by tags in the gallery.

To help others find your model, we suggest tagging it with the model type, such as Extraction or Forms.

🎛 Specifying Blueprint Runtime Configuration

All custom blueprints in the IPA platform require runtime configuration. This runtime configuration contains values that do not change no matter which workflow users add the component to.

Runtime Configuration Required for All Blueprints

All custom blueprints require specific config fields in order to run on the indico platform.

ℹ️ For example, this configuration...

{
  "inputs": [{"name": "documents", "ioClass": "PartitionStream"}],
  "outputs": [{"name": "predictions", "ioClass": "PartitionStream"}],
  "submissionLauncher": {
    "service": "yourapp_yourqueue", 
    "name": "name_of_my_task" // name of task in code
  } 
}

... will run a task defined in the app service as:

taskdef = Task(
  service="yourapp_yourqueue",
  name="name_of_my_task",
  inputs=[IODef("documents", PartitionStream[SpanGroupIO])],
  outputs=[IODef("predictions", PartitionStream[LabelGroupIO])]
)

Specific for Custom Models

ℹ️ The following config is for a classification model that works on text or document datasets and classifies documents as foo or bar

{  
  "dataType": "STRING",  
  "taskType": "CLASSIFICATION",  
  "fields": [  
    {"name": "foo"}, {"name": "bar"}, {"name": "baz"}  
  ]  
}

Specific for Custom Filters

ℹ️ For example, this configuration...

{  
  "branches": ["foo", "bar", "baz"],  
  "links": [["foo"], ["bar"], ["bar", "baz"]]  
}

...results in a filter that performs two actions:

1. Sorts input into three categories: foo, bar, and baz
2. Groups the input into four output streams:
   1. allfoo inputs
   2. all barinputs
   3. allbar inputs and all baz inputs
   4. any inputs that did not qualify for the other three streams

Specific Custom Output

Specific for Individual Component

custom_config is where you define any custom configurations you would like to pass to your task each time it runs, above and beyond what Indico requires. For safety and security, we require custom configs to be dictionaries where all keys and values are strings.

{
  "name" : "my component",
  "tags": ["custom"],
  "custom_config": {
  		"settings_file": "indico-file:///storage/uploads/...",
  		"batch_size": "20"
   }
}

If the the custom configuration for your component is complex, we recommend saving it to a config file and uploading the file using the UploadDocument. That API call outputs a URL, and if you include the URL in your custom_config section, your component can read config values from the file.

When you register a blueprint, the Indico platform merges custom_config intoconfig before saving the component. You can access both the config and custom config from inside a task with context.config.

from indicore.storage.storage_client import MainStorageClient
from jetstream.workflows.context import JSContext

STORAGE: MainStorageClient = MainStorageClient()

async def my_task(context):
  # notice we use  context.config, NOT context.custom_config
  batch_size: int = int(context.config.batch_size)
    
	settings: str = await STORAGE.read(StorageObject.from_uri(context.config.settings_file))

🔐 Defining Blueprint Access

Use the allAccess or datasetIds fields to control who has access to the blueprint.

• IfallAccess is set to true, then this blueprint can be added to any workflow on the platform
• If datasetIds is not None, then the component can only be added to workflows associated with those datasets

NOTE: IfallAccess and datasetIds are both set, the IPA platform will ignore the dataset IDs and allow all users to access the blueprint.

📤Outputs

📤

Outputs

Output of the RegisterCustomBlueprint API is a TaskBlueprint object which has the following fields.

These fields are set by the platform:

id: int : platform-generated unique id for this blueprint

enabled: bool: whether the blueprint can currently be added to workflows

And these fields have the same values as what was inputted when calling the API:

name: str

component_family: str

icon: str

description: str

footer: str

tags: list[str]

config: dict