Pickit interface
This article proposes an Application Programming Interface (API) to be used by robot programs to perform vision-guided pick and place with Pickit.
It consists of a number of global variables and functions that encapsulate the socket communication between a robot and Pickit. The interface is meant to be robot-independent, and consists of global variables and functions written in pseudo-code, using a syntax similar to that of Python. Translating it to a specific robot programming language should consist of adapting to the syntax, features and best practices of the target language. Some deviation from the proposed interface might be necessary due to target language limitations, and the implementer is left with the responsibility of making the best compromise between what is possible and the proposed interface.
Check the Robot Automation Scenarios article to see some API usage examples.
Note
This article assumes an existing implementation of the socket communication layer.
Global variables
Detection variables
This section enumerates all output variables populated by Pickit when valid
detection results are available, that is, when calling pickit_get_result() returns True.
These variables are typically used by a bin picking program in the pick and place hooks. They refer to the currently selected object for picking. PickitPick is the most commonly used variable, normally in the pick hook. All other variables are used for implementing smart picking or placing. Click on the entries below to expand them and learn more:
PickitPick
Object pick point, represented as a pose variable. From this point it’s possible to compute |
PickitPickId
ID of the pick point that was selected for picking, represented as an integer. |
PickitPickRefId
ID of the reference pick point of the PickitPickId, represented as an integer. If PickitPickId was not created with respect to another pick point
(that is, it has |
PickitPickOff
Pick point offset, represented as a pose variable. This is the relative transformation between the reference pick point (identified by PickitPickRefId) and the pick point that was selected for picking (identified by PickitPickId). This offset can be used to place an object consistently, regardless of
how it was picked (which pick point,
whether flexible pick orientation was
used). If a Learn more on how to use this variable in the smart placing examples. |
PickitObjType
Object type, represented as an integer. The mapping between the object type and its identifier is the following: Pickit Teach Teach model ID
Pickit Flex and Pattern
Pickit Bags
|
PickitObjDim
Object dimensions, represented as a 3D array, in the units used by the robot. Depending on the object type, the array should be interpreted as follows:
Pickit Flex and Pattern
|
PickitObjAge
Object age, represented as a floating-point number. The object age is the duration, in seconds, elapsed between the capturing of the camera image and the moment the object information is sent to the robot. |
Calibration variables
This section enumerates all output variables populated by Pickit when computing a new calibration, or validating an existing calibration.
Click on the entries below to expand them and learn more:
PickitCal
The calibration transform,
This variable is updated by pickit_compute_calibration(). |
PickitCalErr
Calibration error, represented as a pose variable. This variable is updated by pickit_validate_calibration(). |
PickitCalErrPos
Position calibration error, represented as a floating-point number, in the units used by the robot. This value contains the norm of the calibration error translational component. This variable is updated by pickit_compute_calibration() and pickit_validate_calibration(). |
PickitCalErrAng
Orientation calibration error, represented as a floating-point number, in degrees. This value contains the angle of the calibration error rotational component, computed using the axis-angle representation. This variable is updated by pickit_compute_calibration() and pickit_validate_calibration(). |
Functions
System functions
The following functions send system-level commands to Pickit.
pickit_is_running()
Check whether robot mode is enabled in Pickit.
All functions in this article except pickit_find_calibration_plate() require robot mode to be enabled, which can be done from the Pickit web interface. |
pickit_shutdown()
Cleanly shut down the Pickit processor.
|
Detection functions
All functions in this section relate to performing object detection and collecting the results. Click on the entries below to expand them and learn more.
pickit_capture_image()
Capture a camera image without triggering object detection. This function blocks until image capture has completed (no need to call pickit_get_result()), and is especially useful when working with a robot-mounted camera, where the robot must remain stationary during image capture, but not during detection. This function is meant to be used prior to calling pickit_process_image(). If the camera moves during image capture, a warning will be shown in the web interface. For fixed camera mounts, it’s usually more convenient to call functions that combine image capture and processing, like pickit_find_objects() or pickit_find_objects_with_retries(retries).
|
The functions below send a request to Pickit, but don’t wait for the response to arrive. Results from these requests are collected by calling pickit_get_result(), documented below, which waits until a response is ready.
pickit_find_objects()
Trigger a Pickit object detection using the currently active setup and product configuration. This call should be followed by a call to pickit_get_result(), which waits until a response for the detection request is ready.
|
pickit_find_objects_with_retries(retries)
Trigger a Pickit object detection with retries using the currently active setup and product configuration.
As opposed to pickit_find_objects(), when no objects are found (but the Region of Interest (ROI) is not empty), Pickit will retry up to retries times to find objects before giving up.
This call should be followed by a call to pickit_get_result(), which waits until a response for the detection request is ready.
- Parameters
retries Maximum number of detection retries.
- Implementation
Send RC_PICKIT_LOOK_FOR_OBJECTS_WITH_RETRIES and don’t wait for a response.
pickit_get_next_object()
Request the next detected object. This call should be followed by a call to pickit_get_result(), which waits until a response for the request is ready. A single object detection run might yield the detection of multiple objects. This function allows to request the next available object, if any, without the need of triggering a new detection and the time overhead it entails. It’s recommended to use this command only when objects in the detection region have not moved (significantly) since the last detection took place. A good example of when to use pickit_get_next_object() is when a detection is unreachable by the robot. An example of when using it is not recommended would be the following bin picking scenario:
When the objects in the detection region have moved, it’s better to re-trigger object detection instead (by calling pickit_find_objects_with_retries(retries), for instance).
|
pickit_process_image()
Trigger a Pickit object detection without image capture using the currently active setup and product configuration. This function uses the latest captured camera image, which typically comes from calling pickit_capture_image() just before. This call should be followed by a call to pickit_get_result(), which waits until a response for the detection request is ready. Refer to the documentation of pickit_capture_image() to learn more about the recommended usage of this function.
|
Note
You must collect the results of each request before sending a new one.
Tip
Note that all calls except pickit_get_next_object() (which is very fast) trigger a Pickit object detection run, which can potentially take multiple seconds.
In your robot program, it’s encouraged to perform robot motions or other non-Pickit-detection actions between calls to these functions and pickit_get_result() to save cycle time in your application. Refer to the pick and place loop of the simple pick and place program to learn more.
This function collects the results of the above requests:
pickit_get_result()
Wait for Pickit to reply with detection results. pickit_get_result() should always be paired to one of the above functions.
|
Configuration functions
These functions are used to change the active product and setup configuration.
pickit_configure(setup, product)
Loads the specified Pickit configuration (setup and product). Parameters
|
pickit_save_setup()
Save the active setup file.
|
pickit_save_product()
Save the active product file.
|
pickit_set_cylinder_dim(length, diameter)
Set the dimensions of a Teach cylinder model (diameter and length). Learn more about the conditions that apply to this request here. Parameters
|
pickit_build_background()
Build the background cloud used by some of the advanced Region of Interest filters. Calling this function will trigger a camera capture. So, if the camera mount is fixed, the robot must not occlude the camera view volume. If instead the camera is robot-mounted, the robot must be in the detection point (more).
|
Calibration functions
These functions are used to perform the robot-camera calibration.
Note
You don’t need to use these functions if your robot integration allows collecting calibration poses interactively from the calibration wizard, and you perform robot-camera calibration infrequently.
You should use these functions if you want to fully automate robot-camera calibration.
pickit_configure_calibration()
Set the robot-camera calibration configuration using the calibration method, camera mount and transform (if applicable) already set in the web interface calibration wizard. Equivalent to calling the advanced version (see below) like so: method = 3 # METHOD_DEFAULT.
camera_mount = 0 # Ignored.
transform = [0, 0, 0, 0, 0, 0] # Ignored.
pickit_configure_calibration(method, camera_mount, transform)
|
pickit_configure_calibration(method, camera_mount, transform)
Parameters
|
pickit_find_calibration_plate()
Trigger detection of the robot-camera calibration plate. Robot mode doesn’t need to be enabled in Pickit to call this function.
|
pickit_compute_calibration()
Trigger the computation of the robot-camera calibration.
|
pickit_validate_calibration(distance_tol, angle_tol)
Validate if the current robot-camera calibration is still valid. It’s considered valid if the location of the detected calibration plate is within the specified distance and angular tolerances. Parameters
|
Monitoring functions
pickit_save_snapshot(subfolder_id=0)
Save a snapshot with the latest detection results. Snapshots will be saved in the For an example usage, refer to the description of the after_end hook of the pick and place program. Parameters
|
Helper functions
The following functions are helpers that don’t require extra communication with Pickit.
They use the information received in the most recent detection results.
These helpers are typically used as part of conditional expressions, such as an if statement.
pickit_object_found()
Check if the last call to pickit_get_result() produced valid detection results.
|
pickit_is_reachable(PickitPick, PrePick, PostPick)
Check if the pick point and the linear approach and retreat trajectories are reachable by the robot. Parameters
This is an optional, but recommended function that requires the robot programming language to expose functionality like joint limits, inverse kinematics and safety plane checks. The exact implementation is open and left to the developer:
|
pickit_is_reachable(Point)
Check if a point is reachable by the robot. Parameters
This function is an overload of pickit_is_reachable(PickitPick, PrePick, PostPick) that only checks the reachability of a single point, instead of a pick trajectory. This is an optional, but recommended function that requires the robot programming language to expose functionality like joint limits, inverse kinematics and safety plane checks.
|
pickit_empty_roi()
Check if the last call to pickit_get_result() detected an empty Region of Interest (ROI). When
pickit_object_found()
returns
Use this function if you need to discriminate between these two situations.
|
pickit_no_image_captured()
Check if object detection was unsuccessful due to a failure to capture a camera image. When this is the case, it typically indicates a hardware disconnection issue, such as a loose connector or broken cable. This function can be used as trigger to send an alarm to a higher level monitoring system.
|
pickit_remaining_objects()
Get the number of remaining detected objects. After calling pickit_get_result(), this function returns the total number of object detections minus one, as the first object data is available through the output picking variables. This value is also equal to the number of times pickit_get_next_object() can be called. As such, the returned value decreases with each call to pickit_get_next_object().
|
Robot brands
These articles document the Pickit interface adapted to the different robot brands.