7.10.4. object_detector.CallObjectDetectorService

class object_detector.CallObjectDetectorService

Bases: object

Service that calls the object detector and returns the detected objects.

This service calls the object detector server and returns the detected objects. It also visualizes the detected objects and publishes the visualization images.

Parameters:

  • rgb (sensor_msgs.msg.Image) – RGB image of the scene

  • depth (sensor_msgs.msg.Image) – Depth image of the scene

srv

This service. Calls the object detector when called

Type:

rospy.Service

label_image_pub

Publisher for the label image visualization. This is only published if the object detector returns a valid label image. If the object detector doesn’t return a valid label image, the label image is generated from the bounding boxes. The label image is visualized by coloring each object with a unique color. The topic is ‘/grasping_pipeline/obj_det_label_image’.

Type:

rospy.Publisher

bb_image_pub

Publisher for the bounding box visualization. The bounding boxes are visualized by drawing a rectangle around each object. The topic is ‘/grasping_pipeline/obj_det_bb_image’.

Type:

rospy.Publisher

Parameters:

  • topic (str) – Name of the object detector topic. Loaded from the ‘grasping_pipeline/object_detector_topic’ parameter

  • timeout (int) – Timeout duration for the object detector. Loaded from the ‘grasping_pipeline/timeout_duration’ parameter

Returns:

  • mask_detections (List[sensor_msgs.msg.Image]) – List of masks for each detected object. Each mask is a binary image where the object is white and the background is black.

  • bb_detections (List[sensor_msgs.msg.RegionOfInterest]) – List of bounding boxes for each detected object. Each bounding box is a rectangle that encloses the object.

  • class_names (List[str]) – List of class names for each detected object.

  • class_confidences (List[float]) – List of confidences for each detected object.

__init__()

Methods

__init__()

convert_label_img_to_2D_BB(label_img)

Converts the label image to 2D bounding boxes.

execute(req)

Calls the object detector and returns the detected objects.

split_label_image_into_masks_np(label_image)

Splits the label image into masks.

split_label_image_into_masks_ros(label_image)

Splits the label image into masks.
convert_label_img_to_2D_BB(label_img)

Converts the label image to 2D bounding boxes.

Converts the label image to 2D bounding boxes. The bounding boxes are calculated by finding the minimumand maximum row and column indices of each object in the label image.

Parameters:

label_img (np.ndarray) – The label image. Each object in the label image has to have a unique label.

Returns:

List of bounding boxes for each detected object.

Return type:

List[sensor_msgs.msg.RegionOfInterest]

execute(req)

Calls the object detector and returns the detected objects.

This function calls the object detector server and returns the detected objects. It also visualizes the detected objects and publishes the visualization images. It converts the label image from the object detector to masks if the label image is valid. If no bounding boxes were passed by the object detector, it converts the label image to bounding boxes.

Parameters:

req (grasping_pipeline_msgs.srv.CallObjectDetectorRequest) – Request containing the RGB and Depth images

Raises:

rospy.ServiceException – If the object detector fails to detect objects or times out or if neither a valid label image nor bounding boxes were returned by the object detector.

Returns:

Response containing the detected objects. The response contains the masks, bounding boxes, class names and confidences for the detected objects.

Return type:

grasping_pipeline_msgs.srv.CallObjectDetectorResponse

split_label_image_into_masks_np(label_image)

Splits the label image into masks.

Splits the label image into masks. Each mask is a binary image where the object is white and the background is black.

Parameters:

label_image (np.ndarray) – The label image. Each object in the label image has to have a unique label.

Returns:

List of masks for each detected object. The background pixels have a value of 0 and the object pixels have a value of != 0.

Return type:

List[np.ndarray]

split_label_image_into_masks_ros(label_image)

Splits the label image into masks.

Splits the label image into masks. Each mask is a binary image where the object is white and the background is black.

Parameters:

label_image (sensor_msgs.msg.Image) – The label image. Each object in the label image has to have a unique label.

Returns:

List of masks for each detected object. The background pixels have a value of 0 and the object pixels have a value of != 0.

Return type:

List[sensor_msgs.msg.Image]