The Pickit low-level URCap interface

This article documents the Pickit low-level URCap interface, which consists of a set of global variables, commands and helper functions that allow you to explicitly write the logic of a pick and place application. A companion article describes an example robot program written using the low-level interface.

Note

The low-level interface exists mainly for backward compatibility with the URCap version 1, and for power users that need to implement application logic that cannot be represented by the pick and place template.

The pick and place template is the preferred way to set up a new pick and place application, as it requires a lower programming effort, and implements a robust execution logic under the hood.

Global variables

Most of the global variables are exposed by the pick and place template.

Note

The URCap plugin exposes global variables under their version 2 names (CamelCase), and also under the version 1 names (snake_case) for backward compatibility. For new programs, the version 2 names are recommended.

• PickitPick, also exposed as pickit_pose.

• PickitPrePick, also exposed as pickit_pre_pose.

• PickitObjType, also exposed as pickit_type.

• PickitObjDim, also exposed as pickit_dim.

Detection

These variables are updated upon each detection, and refer to the currently selected object for picking. They are used by the pick and place template in the Pick sequence.

PickitPick
Object pick point, represented as a pose variable.

PickitPrePick
Object pre-pick point, represented as a pose variable. This point is used for performing a linear approach motion to PickitPick. The strategy used for computing the pre-pick point can be specified in the Pick node configuration.

PickitPostPick
Object post-pick point, represented as a pose variable. This point is used for performing a linear retreat motion from PickitPick. The strategy used for computing the post-pick point can be specified in the Pick node configuration.

These variables also refer to the currently selected object for picking, and expose additional information that can be used when picking an object. They are not used by default by the pick and place template. Learn more about when and how you should use them in the smart placing examples.

PickitPickId
ID of the pick point that was selected for picking, represented as an integer.

PickitPickRefId
ID of the pick point with respect to which PickitPickId was created, represented as an integer. If the pick point associated to PickitPickId was not created with respect to another pick point, this value will be the same as PickitPickId.

PickitPickOff

Pick point offset, represented as a pose variable. This is the relative transformation between the reference pick point (identified by PickitPickRefId) and the pick point that was selected for picking (identified by PickitPickId). If the Dropoff point is expressed with respect to PickitPickRefId, you can compensate for the offset to PickitPickId by post-multiplying Dropoff by PickitPickOff, i.e. DropoffCorrected = pose_trans(Dropoff, PickitPickOff)

These variables expose additional information about the object beyond where to pick it. They also refer to the latest object detection results sent by Pickit, and are not used by default by the pick and place template.

PickitObjType
Object type, represented as an integer. The mapping between the object type and its identifier is the following: Pickit Teach model ID Use this value to conditionally perform an action depending on the detected model. Pickit Flex and Pattern Square 21 Rectangle 22 Circle 23 Ellipse 24 Cylinder 32 Sphere 33 Blob 50

PickitObjDim

Object dimensions, in meters, represented as a 3D array. Depending on the object type, the array should be interpreted as follows: Pickit Teach [bbox x, bbox y, bbox z] Where bbox x represents the size of the object bounding box along its x-axis. Pickit Flex and Pattern Square [length, length, 0] Rectangle [length, width, 0] Circle [diameter, diameter, 0] Ellipse [length, width, 0] Cylinder [length, diameter, diameter] Sphere [diameter, diameter, diameter] Blob [bbox x, bbox y, bbox z]

PickitObjAge
Object age, represented as a floating-point number. This object age is the duration, in seconds, elapsed between the capturing of the camera image and the moment the object information is sent to the robot.

Calibration

These global variables are updated every time a new robot-camera calibration is performed.

PickitCal
The calibration transform, robot_T_camera, represented as a pose variable, where: robot is the robot base (for camera fixed), or the robot flange (for camera-on-robot). camera is the camera optical frame. This variable is updated by pickit_compute_calib().

PickitCalErr
Calibration error, represented as a pose variable. This variable is updated by pickit_validate_calib().

PickitCalErrPos
Position calibration error, represented as a floating-point number (in millimeters). This value contains the norm of the calibration error translational component. This variable is updated by pickit_compute_calib() and pickit_validate_calib().

PickitCalErrAng
Orientation calibration error, represented as a floating-point number (in degrees). This value contains the angle of the calibration error rotational component, computed using the axis-angle representation. This variable is updated by pickit_compute_calib() and pickit_validate_calib().

Commands

This section presents a set of commands that add to Polyscope’s existing ones. In Polyscope 5, they can be inserted in a robot program by selecting Program in the header bar, then, on the left panel URCaps → Pickit: commands.

Next, navigate to the Command tab on the right panel and select an entry from the Pickit command drop-down.

System commmands

The commands in this section are related to the Pickit system’s state.

Check if robot mode enabled
Checks whether robot mode is enabled in Pickit. If robot mode is enabled, program execution continues. If robot mode is not enabled, a pop-up asks the user if it wants to enable it from the robot (as long as there are no unsaved changes). Robot mode can also be enabled from the Pickit web interface. All commands listed below require robot mode to be enabled.

Detection commands

The following commands take care of triggering detections and collecting the results.

Find object(s)

Trigger a Pickit object detection using the currently active setup and product configuration. This call should be followed by a call to Get result, which waits until a response for the detection request is ready. Note that it’s valid (and sometimes encouraged) to perform robot motions or other non Pickit actions between calls to Find objects and Get result, for instance. Refer to the cycle time optimization section of the Pick and place program article for the motivation behind performing robot motions while a Pickit detection is ongoing.

Get next object

Request the next detected object. A single call to Find object(s) or Find objects with retries might yield the detection of multiple objects. Get next object allows to request the next available object, if any, without the need of triggering a new detection and the time overhead it entails. The next Pickit command after Get next object should always be Get result, which waits until a response for the request is ready. It’s recommended to use this command only when objects in the detection region have not moved (significantly) since calling Find object(s). A good example of when to use Get next object is when a detection is unreachable by the robot. An example of when using Get next object is not ideal would be the following bin picking scenario: Trigger Pickit detection that finds multiple objects. First object is picked. Since objects are randomly placed in bin, neighboring objects move and fall into place. Call Get next object and attempt to pick next object. If the next object is one of the neighboring parts that moved, the pick might fail. When the objects in the detection region have moved, it’s better to re-trigger Find object(s) or Find objects with retries instead.

Find objects with retries

Trigger a Pickit object detection with retries, using the currently active setup and product configuration. This call should be followed by a call to Get result, which waits until a response for the detection request is ready. Parameters Max tries When no objects are found but the ROI is not empty, Pickit can retry object detection. A value of one performs no retries. Refer to the Find objects command for usage tips.

Get result

Wait for Pickit to reply with detection results. Get result should always be the next Pickit command after a Find object(s), Find objects with retries or Get next object request. It blocks until a reply from Pickit is received, and the success of the request can then be queried by calling pickit_object_found(). When an object has been found, the following global variables are populated: Object pick pose: PickitPick Object pre-pick pose: PickitPrePick. This pose is computed by applying an offset to PickitPick along a specified direction, as specified by the command parameters. Object dimensions: PickitObjDim Object type: PickitObjType Parameters Pre-pick offset: base frame PickitPrePick is computed by applying an offset along the z-axis of the specified frame. Valid options are object frame or robot base frame. Pre-pick offset Offset in mm applied to compute PickitPrePick.

Configuration commands

Select

Loads the specified setup and product configuration. This configuration specifies the behavior of Pickit detections, e.g. what to look for, in which part of the field of view. Parameters Setup Any of the setup configurations currently available in the connected Pickit system. Product Any of the product configurations currently available in the connected Pickit system. Available configurations are listed in drop-down menus. Note: If you need to change setup or product based on a runtime decision (not known in advance), consider using the pickit_configure helper instead.

Build background
Build the background cloud used by some of the advanced Region of Interest filters. Calling this function will trigger a camera capture, so if the camera mount is fixed, the robot must not occlude the camera view volume. If instead the camera is robot-mounted, the robot must be in the detection point (more).

Calibration commands

The following command is necessary when performing a robot-camera calibration from the robot.

Find calibration plate
Trigger a calibration plate detection. If the plate detection fails, the program is terminated. Plate detection failure can occur if the plate is not completely visible, or if the marker detection is poor.

Monitoring commands

Save snapshot

Save a snapshot with the latest detection results. The saved snapshot can then be loaded or downloaded by going to the Snapshots page on the Pickit web interface and searching for a file whose name contains the capture timestamp. Example usage: Trigger saving a snapshot on the action after end sequence, when no objects are found (but the ROI is not empty), so it’s possible to investigate the cause.

Save snapshot to
Save a snapshot with the latest detection results to a specific subfolder. The saved snapshot can then be loaded or downloaded by going to the Snapshots page on the Pickit web interface and searching for a file whose name contains the capture timestamp. Parameters Subfolder ID Subfolder in which the snapshot should be saved. Values between 1 and 255 are allowed.

Scripting functions

In addition to the URCap commands, the Pickit interface also offers URScript functions. These functions can be selected from the Function drop-down of the script editor.

System functions

System functions interact with Pickit at a system level. This function allows the programmer to manually finish pick and place.

pickit_break()

This function allows the programmer to manually finish pick and place. Call this function from any of the sequences (Object detection, Pick or Place) to finish the program, when an application-specific condition is met. Pick and place finishes after the current sequence completes. In the above example, pickit_break() is called at the top of the Place sequence. The remaining of the sequence (dropoff, tool release) will still be executed, unless you explicitly skip it using a conditional. Once the sequence from which pickit_break() is called completes, pick and place will finish and the Action after end sequence will be executed. See also pickit_break_called().

pickit_break_called()
This function checks if pickit_break() was called. This check is included in the default implementation of action after end. Return true if pickit_break() was called.

pickit_shutdown()
Cleanly shut down the Pickit processor. Return True on success.

Detection functions

pickit_capture_image()

Capture a camera image without triggering object detection. This function blocks until image capture has completed, and is especially useful when working with a robot-mounted camera, where the robot must remain stationary during image capture, but not during detection. This function is meant to be used prior to calling pickit_process_image(). If the camera moves during image capture, a warning will be shown in the web interface. For fixed camera mounts, it’s usually more convenient to call functions that combine image capture and processing, like Find object(s) or Find objects with retries. Return true if image capture was successful.

pickit_process_image()

Trigger a Pickit object detection without image capture using the currently active setup and product configuration. This function uses the latest captured camera image, which typically comes from calling pickit_capture_image() just before. This call should be paired with a call to Get result, which waits until a response for he detection request is ready. Refer to the documentation of pickit_capture_image() to learn more about the recommended usage of this function.

pickit_object_found()

Check if the last detection found at least one pickable object. When pickit_object_found() returns false, it can be due to: The ROI is not empty, but Pickit doesn’t detect the active product. The ROI is empty. You can use this function if you need to discriminate between these two situations, even though other functions can be used for the same purpose. Notice that the pick and place template already distinguishes the two cases, although it does not use this function. This function returns false when Pickit replied with no detection results (nominal usecase); or if called without making a request to Pickit and collecting the results with Get result (should be avoided, as it makes no sense). Return true if at least one pickable object was detected. A similar function pickit_no_object_found() also exists, which returns true if no objects were detected, and the ROI is not empty.

pickit_no_object_found()

Check if the last detection found no pickable objects because the requested object is not found, but the ROI is not empty. This check is included in the default implementation of action after end of the pick and place template. If you are not using the template, but the low-level URCap commands, you can call this function after a Get result. Return true if there are no pickable objects, but the ROI is not empty. A similar function pickit_object_found() also exists, which returns true if at least one pickable object was detected.

pickit_object_reachable()
Check if the object retried by the last call to Get result has reachable pick and pre-pick poses. Return true if the global variables PickitPrePick and PickitPick contain poses that are reachable by the robot. Note that pickit_object_reachable() == true implies pickit_object_found() == true. Therefore, checking for reachable objects should be done after checking for detected objects.

pickit_no_object_reachable()
Check if all detected and pickable objects are not reachable by the robot. This check is included in the default implementation of action after end. If you are not using the template, but the low-level URCap commands, you can call this function after a Get result. Return true If there are detected objects, but all are unreachable.

pickit_no_image_captured()

Check if object detection was unsuccessful due to a failure to capture a camera image. When this is the case, it typically indicates a hardware disconnection issue, such as a loose connector or broken cable. This function can be used as trigger to send an alarm to a higher level monitoring system. This check is included in the default implementation of action after end. Return true if there are no pickable objects due to a failed image capture.

pickit_empty_roi()

Check if Pickit detected an empty Region of Interest (ROI) during the last detection. An empty ROI is one of the possible causes for Pickit to find no pickable objects. This function is called in the optional action after end sequence of the pick and place template. If you are not using the template, but the low-level URCap commands, you can call this function after a Get result. Return true if Pickit detected an empty ROI.

pickit_remaining_objects()

Get the number of remaining object detections. After calling Get result, this function returns the total number of object detections minus one, as the first object data is available through the Global variables. This value is also equal to the number of times Get next object can be called. As such, the returned value decreases with each call to Get next object. Return Number of remaining object detections available for query.

Configuration functions

pickit_configure(setup_id, product_id)

Loads the specified setup and product configuration. This function is similar to the Select command, but specifies the setup and product by their respective IDs (integers) as opposed to names from a drop-down. Prefer using this function over the Select command when you need to change the setup or product based on a runtime decision, and the IDs are read from variable values. Parameters setup_id ID (integer) of a setup configurations currently available in the connected Pickit system. product_id ID (integer) of a product configurations currently available in the connected Pickit system. Return true on configuration success: setup and product IDs exist and loaded successfully.

The following are advanced functions that can save and modify the Pickit configuration.

pickit_save_product()
Save the active product file. Return True on success.

pickit_save_setup()
Save the active setup file. Return True on success.

pickit_set_cylinder_dim(length, diameter)

Set the dimensions of a Teach cylinder model (diameter and length). When this call succeeds, the active product file will have unsaved changes. If desired, you can save them by calling pickit_save_product(). Learn more about the conditions that apply to this request here. Parameters length: new cylinder length (in millimeters). diameter: new cylinder diameter (in millimeters). Return True on success.

Calibration functions

pickit_configure_calib()

Set the robot-camera calibration configuration using the calibration method, camera mount and transform (if applicable) already set in the web interface calibration wizard. Equivalent to calling the advanced version (see below) like so: method = 3 # METHOD_DEFAULT. camera_mount = 0 # Ignored. transform = p[0, 0, 0, 0, 0, 0] # Ignored. pickit_configure_calibration(method, camera_mount, transform) Return true if the operation was successful.

pickit_configure_calib(method, camera_mount, transform)

Set the robot-camera calibration configuration (advanced version). Parameters method: Calibration method (single-pose, multi-pose, or manual). camera_mount: Camera mount (fixed or camera on robot). transform: Required for manual and single-pose methods, not for multi-pose calibration. Learn more about the values of the parameters and the conditions that apply to them here. Return true if the operation was successful.

pickit_find_calib_plate()
Trigger detection of the robot-camera calibration plate. You can alternatively call the Find calibration plate URCap command, which doesn’t return a boolean, but instead terminates program execution when failing to detect the plate. Return true if the plate was successfully detected.

pickit_compute_calib()
Trigger the computation of the robot-camera calibration. When the function returns true, the PickitCal, PickitCalErrPos and PickitCalErrAng variables are updated. Return true if the operation was successful.

pickit_validate_calib(distance_tol, angle_tol)

Validate if the current robot-camera calibration is still valid. It’s considered valid if the location of the detected calibration plate is within the specified distance and angular tolerances. When the function returns true, the PickitCalErr, PickitCalErrPos and PickitCalErrAng variables are updated. Parameters distance_tol: Distance tolerance (in millimeters). angle_tol: Angular tolerance (in degrees). Return true if the current robot-camera calibration is valid.