mindformers.pipeline.ZeroShotImageClassificationPipeline

class mindformers.pipeline.ZeroShotImageClassificationPipeline(model: Union[str, mindformers.models.base_model.BaseModel, mindspore.train.model.Model], tokenizer: Optional[mindformers.models.base_tokenizer.BaseTokenizer] = None, image_processor: Optional[mindformers.models.base_processor.BaseImageProcessor] = None, **kwargs)

Pipeline For Zero Shot Image Classification

参数

• model (Union[str, BaseModel]) – The model used to perform task, the input could be a supported model name, or a model instance inherited from BaseModel.

• tokenizer (Optional[BaseTokenizer]) – A tokenizer for text processing.

• image_processor (Optional[BaseImageProcessor]) – The image_processor of model, it could be None if the model do not need image_processor.

引发

• TypeError – If input model, tokenizer, and image_processor’s types are not corrected.

• ValueError – if the input model is not in support list.

实际案例

>>> from mindformers.tools.image_tools import load_image
>>> from mindformers.pipeline import ZeroShotImageClassificationPipeline
>>> classifier = ZeroShotImageClassificationPipeline(
...     model='clip_vit_b_32',
...     candidate_labels=["sunflower", "tree", "dog", "cat", "toy"],
...     hypothesis_template="This is a photo of {}."
...     )
>>> img = load_image("https://ascend-repo-modelzoo.obs.cn-east-2."
...                  "myhuaweicloud.com/XFormer_for_mindspore/clip/sunflower.png")
>>> classifier(img)
    [[{'score': 0.99995565, 'label': 'sunflower'},
    {'score': 2.5318595e-05, 'label': 'toy'},
    {'score': 9.903885e-06, 'label': 'dog'},
    {'score': 6.75336e-06, 'label': 'tree'},
    {'score': 2.396818e-06, 'label': 'cat'}]]

forward(model_inputs: dict, **forward_params)

Forward process

参数

model_inputs (dict) – Outputs of preprocess.

返回

Probs dict.

postprocess(model_outputs: dict, **postprocess_params)

Postprocess

参数

• model_outputs (dict) – Outputs of forward process.

• top_k (int) – Return top_k probs of result

返回

Classification results.

preprocess(inputs: dict, **preprocess_params)

Preprocess of ZeroShotImageClassificationPipeline

参数

• inputs (Union[url, PIL.Image, tensor, numpy]) – The image to be classified.

• candidate_labels (List[str]) – The candidate labels for classification.

• hypothesis_template (Optional[str]) – Prompt for text input.

• max_length (Optional[int]) – Max length of tokenizer’s output

• padding (Optional[Union[False, "max_length"]]) – Padding for max_length

• return_tensors (Optional["ms"]) – The type of returned tensors

返回

Processed data.

引发

ValueError – If candidate_labels or hypothesis_template is None.