Visualizer — mmengine 0.11.0rc0 documentation (original) (raw)

class mmengine.visualization.Visualizer(name='visualizer', image=None, vis_backends=None, save_dir=None, fig_save_cfg={'frameon': False}, fig_show_cfg={'frameon': False})[source]¶

MMEngine provides a Visualizer class that uses the Matplotliblibrary as the backend. It has the following functions:

• Basic drawing methods
  • draw_bboxes: draw single or multiple bounding boxes
  • draw_texts: draw single or multiple text boxes
  • draw_points: draw single or multiple points
  • draw_lines: draw single or multiple line segments
  • draw_circles: draw single or multiple circles
  • draw_polygons: draw single or multiple polygons
  • draw_binary_masks: draw single or multiple binary masks
  • draw_featmap: draw feature map
• Basic visualizer backend methods
  • add_configs: write config to all vis storage backends
  • add_graph: write model graph to all vis storage backends
  • add_image: write image to all vis storage backends
  • add_scalar: write scalar to all vis storage backends
  • add_scalars: write scalars to all vis storage backends
  • add_datasample: write datasample to all vis storage backends. The abstract drawing interface used by the user
• Basic info methods
  • set_image: sets the original image data
  • get_image: get the image data in Numpy format after drawing
  • show: visualization
  • close: close all resources that have been opened
  • get_backend: get the specified vis backend

All the basic drawing methods support chain calls, which is convenient for overlaydrawing and display. Each downstream algorithm library can inheritVisualizer and implement the add_datasample logic. For example,DetLocalVisualizer in MMDetection inherits from Visualizerand implements functions, such as visual detection boxes, instance masks, and semantic segmentation maps in the add_datasample interface.

Parameters:

• name (str) – Name of the instance. Defaults to ‘visualizer’.
• image (np.ndarray , optional) – the origin image to draw. The format should be RGB. Defaults to None.
• vis_backends (list, optional) – Visual backend config list. Defaults to None.
• save_dir (str, optional) – Save file dir for all storage backends. If it is None, the backend storage will not save any data.
• fig_save_cfg (dict) – Keyword parameters of figure for saving. Defaults to empty dict.
• fig_show_cfg (dict) – Keyword parameters of figure for showing. Defaults to empty dict.

Examples

Basic info methods

vis = Visualizer() vis.set_image(image) vis.get_image() vis.show()

Basic drawing methods

vis = Visualizer(image=image) vis.draw_bboxes(np.array([0, 0, 1, 1]), edge_colors='g') vis.draw_bboxes(bbox=np.array([[1, 1, 2, 2], [2, 2, 3, 3]]), edge_colors=['g', 'r']) vis.draw_lines(x_datas=np.array([1, 3]), y_datas=np.array([1, 3]), colors='r', line_widths=1) vis.draw_lines(x_datas=np.array([[1, 3], [2, 4]]), y_datas=np.array([[1, 3], [2, 4]]), colors=['r', 'r'], line_widths=[1, 2]) vis.draw_texts(text='MMEngine', position=np.array([2, 2]), colors='b') vis.draw_texts(text=['MMEngine','OpenMMLab'], position=np.array([[2, 2], [5, 5]]), colors=['b', 'b']) vis.draw_circles(circle_coord=np.array([2, 2]), radius=np.array[1]) vis.draw_circles(circle_coord=np.array([[2, 2], [3, 5]), radius=np.array[1, 2], colors=['g', 'r']) square = np.array([[0, 0], [100, 0], [100, 100], [0, 100]]) vis.draw_polygons(polygons=square, edge_colors='g') squares = [np.array([[0, 0], [100, 0], [100, 100], [0, 100]]), np.array([[0, 0], [50, 0], [50, 50], [0, 50]])] vis.draw_polygons(polygons=squares, edge_colors=['g', 'r']) vis.draw_binary_masks(binary_mask, alpha=0.6) heatmap = vis.draw_featmap(featmap, img, channel_reduction='select_max') heatmap = vis.draw_featmap(featmap, img, channel_reduction=None, topk=8, arrangement=(4, 2)) heatmap = vis.draw_featmap(featmap, img, channel_reduction=None, topk=-1)

chain calls

vis.draw_bboxes().draw_texts().draw_circle().draw_binary_masks()

Backend related methods

vis = Visualizer(vis_backends=[dict(type='LocalVisBackend')], save_dir='temp_dir') cfg = Config(dict(a=1, b=dict(b1=[0, 1]))) vis.add_config(cfg) image=np.random.randint(0, 256, size=(10, 10, 3)).astype(np.uint8) vis.add_image('image',image) vis.add_scaler('mAP', 0.6) vis.add_scalars({'loss': 0.1,'acc':0.8})

inherit

class DetLocalVisualizer(Visualizer): def add_datasample(self, name, image: np.ndarray, gt_sample: Optional['BaseDataElement'] = None, pred_sample: Optional['BaseDataElement'] = None, draw_gt: bool = True, draw_pred: bool = True, show: bool = False, wait_time: int = 0, step: int = 0) -> None: pass

add_config(config, **kwargs)[source]¶

Record the config.

Parameters:

config (Config) – The Config object.

add_datasample(name, image, data_sample=None, draw_gt=True, draw_pred=True, show=False, wait_time=0, step=0)[source]¶

Draw datasample.

Parameters:

• image (ndarray) –
• data_sample (BaseDataElement | None) –
• draw_gt (bool) –
• draw_pred (bool) –
• show (bool) –
• wait_time (int) –
• step (int) –

Return type:

None

add_graph(model, data_batch, **kwargs)[source]¶

Record the model graph.

Parameters:

• model (torch.nn.Module) – Model to draw.
• data_batch (Sequence _[_dict]) – Batch of data from dataloader.

Return type:

None

add_image(name, image, step=0)[source]¶

Record the image.

Parameters:

• name (str) – The image identifier.
• image (np.ndarray , optional) – The image to be saved. The format should be RGB. Defaults to None.
• step (int) – Global step value to record. Defaults to 0.

Return type:

None

add_scalar(name, value, step=0, **kwargs)[source]¶

Record the scalar data.

Parameters:

• name (str) – The scalar identifier.
• value (float, int) – Value to save.
• step (int) – Global step value to record. Defaults to 0.

Return type:

None

add_scalars(scalar_dict, step=0, file_path=None, **kwargs)[source]¶

Record the scalars’ data.

Parameters:

• scalar_dict (dict) – Key-value pair storing the tag and corresponding values.
• step (int) – Global step value to record. Defaults to 0.
• file_path (str, optional) – The scalar’s data will be saved to the file_path file at the same time if the file_path parameter is specified. Defaults to None.

Return type:

None

close()[source]¶

Close an opened object.

Return type:

None

property dataset_meta_: dict | None_¶

Meta info of the dataset.

Type:

Optional[dict]

draw_bboxes(bboxes, edge_colors='g', line_styles='-', line_widths=2, face_colors='none', alpha=0.8)[source]¶

Draw single or multiple bboxes.

Parameters:

• bboxes (Union _[_ _np.ndarray_ _,_ torch.Tensor]) – The bboxes to draw with the format of(x1,y1,x2,y2).
• edge_colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The colors of bboxes. colors can have the same length with lines or just single value. If colors is single value, all the lines will have the same colors. Refer to matplotlib. colors for full list of formats that are accepted. Defaults to ‘g’.
• line_styles (Union [_str,_ List _[_str] ]) – The linestyle of lines. line_styles can have the same length with texts or just single value. If line_styles is single value, all the lines will have the same linestyle. Reference tohttps://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestylefor more details. Defaults to ‘-‘.
• line_widths (Union [_ _Union_ _[_int,_ float] , List [_ _Union_ _[_int,_ float] ] ]) – The linewidth of lines. line_widths can have the same length with lines or just single value. If line_widths is single value, all the lines will have the same linewidth. Defaults to 2.
• face_colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The face colors. Defaults to None.
• alpha (Union [_int,_ float]) – The transparency of bboxes. Defaults to 0.8.

Return type:

Visualizer

draw_binary_masks(binary_masks, colors='g', alphas=0.8)[source]¶

Draw single or multiple binary masks.

Parameters:

• binary_masks (np.ndarray , torch.Tensor) – The binary_masks to draw with of shape (N, H, W), where H is the image height and W is the image width. Each value in the array is either a 0 or 1 value of uint8 type.
• colors (np.ndarray) – The colors which binary_masks will convert to.colors can have the same length with binary_masks or just single value. If colors is single value, all the binary_masks will convert to the same colors. The colors format is RGB. Defaults to np.array([0, 255, 0]).
• alphas (Union [_int,_ List _[_int] ]) – The transparency of masks. Defaults to 0.8.

Return type:

Visualizer

draw_circles(center, radius, edge_colors='g', line_styles='-', line_widths=2, face_colors='none', alpha=0.8)[source]¶

Draw single or multiple circles.

Parameters:

• center (Union _[_ _np.ndarray_ _,_ torch.Tensor]) – The x coordinate of each line’ start and end points.
• radius (Union _[_ _np.ndarray_ _,_ torch.Tensor]) – The y coordinate of each line’ start and end points.
• edge_colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The colors of circles. colors can have the same length with lines or just single value. If colors is single value, all the lines will have the same colors. Reference tohttps://matplotlib.org/stable/gallery/color/named_colors.htmlfor more details. Defaults to ‘g.
• line_styles (Union [_str,_ List _[_str] ]) – The linestyle of lines. line_styles can have the same length with texts or just single value. If line_styles is single value, all the lines will have the same linestyle. Reference tohttps://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestylefor more details. Defaults to ‘-‘.
• line_widths (Union [_ _Union_ _[_int,_ float] , List [_ _Union_ _[_int,_ float] ] ]) – The linewidth of lines. line_widths can have the same length with lines or just single value. If line_widths is single value, all the lines will have the same linewidth. Defaults to 2.
• face_colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The face colors. Defaults to None.
• alpha (Union [_int,_ float]) – The transparency of circles. Defaults to 0.8.

Return type:

Visualizer

static draw_featmap(featmap, overlaid_image=None, channel_reduction='squeeze_mean', topk=20, arrangement=(4, 5), resize_shape=None, alpha=0.5)[source]¶

Draw featmap.

• If overlaid_image is not None, the final output image will be the weighted sum of img and featmap.
• If resize_shape is specified, featmap and overlaid_imageare interpolated.
• If resize_shape is None and overlaid_image is not None, the feature map will be interpolated to the spatial size of the image in the case where the spatial dimensions of overlaid_image andfeatmap are different.
• If channel_reduction is “squeeze_mean” and “select_max”, it will compress featmap to single channel image and weighted sum to overlaid_image.
• If channel_reduction is None
  • If topk <= 0, featmap is assert to be one or three channel and treated as image and will be weighted sum to overlaid_image.
  • If topk > 0, it will select topk channel to show by the sum of each channel. At the same time, you can specify the arrangementto set the window layout.

Parameters:

• featmap (torch.Tensor) – The featmap to draw which format is (C, H, W).
• overlaid_image (np.ndarray , optional) – The overlaid image. Defaults to None.
• channel_reduction (str, optional) – Reduce multiple channels to a single channel. The optional value is ‘squeeze_mean’ or ‘select_max’. Defaults to ‘squeeze_mean’.
• topk (int) – If channel_reduction is not None and topk > 0, it will select topk channel to show by the sum of each channel. if topk <= 0, tensor_chw is assert to be one or three. Defaults to 20.
• arrangement (Tuple [_int,_ int]) – The arrangement of featmap when channel_reduction is None and topk > 0. Defaults to (4, 5).
• resize_shape (tuple, optional) – The shape to scale the feature map. Defaults to None.
• alpha (Union [_int,_ List _[_int] ]) – The transparency of featmap. Defaults to 0.5.

Returns:

RGB image.

Return type:

np.ndarray

draw_lines(x_datas, y_datas, colors='g', line_styles='-', line_widths=2)[source]¶

Draw single or multiple line segments.

Parameters:

• x_datas (Union _[_ _np.ndarray_ _,_ torch.Tensor]) – The x coordinate of each line’ start and end points.
• y_datas (Union _[_ _np.ndarray_ _,_ torch.Tensor]) – The y coordinate of each line’ start and end points.
• colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The colors of lines. colors can have the same length with lines or just single value. If colors is single value, all the lines will have the same colors. Reference tohttps://matplotlib.org/stable/gallery/color/named_colors.htmlfor more details. Defaults to ‘g’.
• line_styles (Union [_str,_ List _[_str] ]) – The linestyle of lines. line_styles can have the same length with texts or just single value. If line_styles is single value, all the lines will have the same linestyle. Reference tohttps://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestylefor more details. Defaults to ‘-‘.
• line_widths (Union [_ _Union_ _[_int,_ float] , List [_ _Union_ _[_int,_ float] ] ]) – The linewidth of lines. line_widths can have the same length with lines or just single value. If line_widths is single value, all the lines will have the same linewidth. Defaults to 2.

Return type:

Visualizer

draw_points(positions, colors='g', marker=None, sizes=None)[source]¶

Draw single or multiple points.

Parameters:

• positions (Union _[_ _np.ndarray_ _,_ torch.Tensor]) – Positions to draw.
• colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The colors of points. colors can have the same length with points or just single value. If colors is single value, all the points will have the same colors. Reference tohttps://matplotlib.org/stable/gallery/color/named_colors.htmlfor more details. Defaults to ‘g.
• marker (str, optional) – The marker style. See matplotlib.markers for more information about marker styles. Defaults to None.
• sizes (Optional _[_ _Union_ _[_ _np.ndarray_ _,_ torch.Tensor] ]) – The marker size. Defaults to None.

draw_polygons(polygons, edge_colors='g', line_styles='-', line_widths=2, face_colors='none', alpha=0.8)[source]¶

Draw single or multiple bboxes.

Parameters:

• polygons (Union _[_ _Union_ _[_ _np.ndarray_ _,_ torch.Tensor] , List _[_ _Union_ _[_ _np.ndarray_ _,_ torch.Tensor] ] ]) – The polygons to draw with the format of (x1,y1,x2,y2,…,xn,yn).
• edge_colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The colors of polygons. colors can have the same length with lines or just single value. If colors is single value, all the lines will have the same colors. Refer tomatplotlib.colors for full list of formats that are accepted. Defaults to ‘g.
• line_styles (Union [_str,_ List _[_str] ]) – The linestyle of lines. line_styles can have the same length with texts or just single value. If line_styles is single value, all the lines will have the same linestyle. Reference tohttps://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestylefor more details. Defaults to ‘-‘.
• line_widths (Union [_ _Union_ _[_int,_ float] , List [_ _Union_ _[_int,_ float] ] ]) – The linewidth of lines. line_widths can have the same length with lines or just single value. If line_widths is single value, all the lines will have the same linewidth. Defaults to 2.
• face_colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The face colors. Defaults to None.
• alpha (Union [_int,_ float]) – The transparency of polygons. Defaults to 0.8.

Return type:

Visualizer

draw_texts(texts, positions, font_sizes=None, colors='g', vertical_alignments='top', horizontal_alignments='left', font_families='sans-serif', bboxes=None, font_properties=None)[source]¶

Draw single or multiple text boxes.

Parameters:

• texts (Union [_str,_ List _[_str] ]) – Texts to draw.
• positions (Union _[_ _np.ndarray_ _,_ torch.Tensor]) – The position to draw the texts, which should have the same length with texts and each dim contain x and y.
• font_sizes (Union [_int,_ List _[_int] ] , optional) – The font size of texts. font_sizes can have the same length with texts or just single value. If font_sizes is single value, all the texts will have the same font size. Defaults to None.
• colors (Union [_str,_ tuple, List _[_str] , List _[_tuple] ]) – The colors of texts. colors can have the same length with texts or just single value. If colors is single value, all the texts will have the same colors. Reference tohttps://matplotlib.org/stable/gallery/color/named_colors.htmlfor more details. Defaults to ‘g.
• vertical_alignments (Union [_str,_ List _[_str] ]) – The verticalalignment of texts. verticalalignment controls whether the y positional argument for the text indicates the bottom, center or top side of the text bounding box.vertical_alignments can have the same length with texts or just single value. If vertical_alignments is single value, all the texts will have the same verticalalignment. verticalalignment can be ‘center’ or ‘top’, ‘bottom’ or ‘baseline’. Defaults to ‘top’.
• horizontal_alignments (Union [_str,_ List _[_str] ]) – The horizontalalignment of texts. Horizontalalignment controls whether the x positional argument for the text indicates the left, center or right side of the text bounding box.horizontal_alignments can have the same length with texts or just single value. If horizontal_alignments is single value, all the texts will have the same horizontalalignment. Horizontalalignment can be ‘center’,’right’ or ‘left’. Defaults to ‘left’.
• font_families (Union [_str,_ List _[_str] ]) – The font family of texts. font_families can have the same length with texts or just single value. If font_families is single value, all the texts will have the same font family. font_familiy can be ‘serif’, ‘sans-serif’, ‘cursive’, ‘fantasy’ or ‘monospace’. Defaults to ‘sans-serif’.
• bboxes (Union [_dict,_ List _[_dict] ] , optional) – The bounding box of the texts. If bboxes is None, there are no bounding box around texts. bboxes can have the same length with texts or just single value. If bboxes is single value, all the texts will have the same bbox. Reference tohttps://matplotlib.org/stable/api/_as_gen/matplotlib.patches.FancyBboxPatch.html#matplotlib.patches.FancyBboxPatchfor more details. Defaults to None.
• font_properties (Union [ FontProperties , List [ FontProperties ] ] , optional) – The font properties of texts. FontProperties is a font_manager.FontProperties() object. If you want to draw Chinese texts, you need to prepare a font file that can show Chinese characters properly. For example: simhei.ttf, simsun.ttc, simkai.ttf and so on. Then set font_properties=matplotlib.font_manager.FontProperties(fname='path/to/font_file') font_properties can have the same length with texts or just single value. If font_properties is single value, all the texts will have the same font properties. Defaults to None.New in version 0.6.0.

Return type:

Visualizer

get_backend(name)[source]¶

Get vis backend by name.

Parameters:

name (str) – The name of vis backend

Returns:

The vis backend.

Return type:

BaseVisBackend

get_image()[source]¶

Get the drawn image. The format is RGB.

Returns:

the drawn image which channel is RGB.

Return type:

np.ndarray

classmethod get_instance(name, **kwargs)[source]¶

Make subclass can get latest created instance byVisualizer.get_current_instance().

Downstream codebase may need to get the latest created instance without knowing the specific Visualizer type. For example, mmdetection builds visualizer in runner and some component which cannot access runner wants to get latest created visualizer. In this case, the component does not know which type of visualizer has been built and cannot get target instance. Therefore, Visualizeroverrides the get_instance() and its subclass will register the created instance to _instance_dict additionally.get_current_instance() will return the latest created subclass instance.

Examples

class DetLocalVisualizer(Visualizer): def init(self, name): super().init(name)

visualizer1 = DetLocalVisualizer.get_instance('name1') visualizer2 = Visualizer.get_current_instance() visualizer3 = DetLocalVisualizer.get_current_instance() assert id(visualizer1) == id(visualizer2) == id(visualizer3)

Parameters:

name (str) – Name of instance.

Returns:

Corresponding name instance.

Return type:

object

set_image(image)[source]¶

Set the image to draw.

Parameters:

image (np.ndarray) – The image to draw.

Return type:

None

show(drawn_img=None, win_name='image', wait_time=0.0, continue_key=' ', backend='matplotlib')[source]¶

Show the drawn image.

Parameters:

• drawn_img (np.ndarray , optional) – The image to show. If drawn_img is None, it will show the image got by Visualizer. Defaults to None.
• win_name (str) – The image title. Defaults to ‘image’.
• wait_time (float) – Delay in seconds. 0 is the special value that means “forever”. Defaults to 0.
• continue_key (str) – The key for users to continue. Defaults to the space key.
• backend (str) – The backend to show the image. Defaults to ‘matplotlib’. New in version 0.7.3.

Return type:

None