Warning

You are reading the documentation for an older Pickit release (2.3). Documentation for the latest release (3.4) can be found here.

The Pickit interface

This article proposes an Application Programming Interface (API) to be used by robot programs to perform vision-guided pick and place with Pickit. It consists of a number of global variables and functions that encapsulate the socket communication between a robot and Pickit.

The interface is meant to be robot-independent, and consists of global variables and functions written in pseudo-code, using a syntax similar to that of Python. Translating it to a specific robot programming language should consist of adapting to the syntax, features and best practices of the target language. Some deviation from the proposed interface might be necessary due to target language limitations, and the implementer is left with the responsibility of making the best compromise between what is possible and the proposed interface.

Refer to the following articles for examples of this interface in use:

• Simple pick and place program

• Complete pick and place program

• Calibration program

Note

This article assumes an existing implementation of the socket communication layer.

Output global variables

This section enumerates all output variables populated by Pickit when valid detection results are available, that is, when calling pickit_get_result() returns True.

These variables are typically used by a pick and place program in the pick and place hooks. They refer to the currently selected object for picking. PickitPick is the most commonly used variable, normally in the pick hook. All other variables are used for implementing smart picking or placing. Click on the entries below to expand them and learn more:

PickitPick

Object pick point, represented as a pose variable. From this point it’s possible to compute PickitPrePick and PickitPostPick for performing linear approach and retreat motions. Learn more about how they are computed here.

PickitPickId

ID of the pick point that was selected for picking, represented as an integer.

PickitPickRefId

ID of the reference pick point of the PickitPickId, represented as an integer. If PickitPickId was not created with respect to another pick point (that is, it has Origin as reference), this value will be the same as PickitPickId.

PickitPickOff

Pick point offset, represented as a pose variable. This is the relative transformation between the reference pick point (identified by PickitPickRefId) and the pick point that was selected for picking (identified by PickitPickId). This offset can be used to place an object consistently, regardless of how it was picked (which pick point, whether flexible pick orientation was used). If a Dropoff point was specified for an object picked from PickitPickRefId, it can be translated to PickitPickId by post-multiplying Dropoff by PickitPickOff. Learn more on how to use this variable in the smart placing examples.

PickitObjType

Object type, represented as an integer. The mapping between the object type and its identifier is the following: Pickit Teach Teach model ID Use this value to conditionally perform an action depending on the detected model. Pickit Flex and Pattern Square 21 Rectangle 22 Circle 23 Ellipse 24 Cylinder 32 Sphere 33 Blob 50 Pickit Bags An integer that holds information about the bag pattern and the detected layer orientation, according to this table.

PickitObjDim

Object dimensions, in meters, represented as a 3D array. Depending on the object type, the array should be interpreted as follows: Pickit Teach [bbox x, bbox y, bbox z] Where bbox x represents the size of the object bounding box along its x-axis. Pickit Flex and Pattern Square [length, length, 0] Rectangle [length, width, 0] Circle [diameter, diameter, 0] Ellipse [length, width, 0] Cylinder [length, diameter, diameter] Sphere [diameter, diameter, diameter] Blob [bbox x, bbox y, bbox z]

PickitObjAge

Object age, represented as a floating-point number. The object age is the duration, in seconds, elapsed between the capturing of the camera image and the moment the object information is sent to the robot.

Functions

The following functions relate to performing object detection. They send a request to Pickit, but don’t wait for the response to arrive. Results from these requests are collected by calling pickit_get_result(), which waits until a response is ready. Click on the entries below to expand them and learn more:

pickit_find_objects()

Trigger a Pickit object detection using the currently active setup and product configuration. Implementation Send RC_PICKIT_LOOK_FOR_OBJECTS and don’t wait for a response.

pickit_find_objects_with_retries(retries)

Trigger a Pickit object detection with retries using the currently active setup and product configuration. As opposed to pickit_find_objects(), when no objects are found (but the Region of Interest (ROI) is not empty), Pickit will retry up to retries times to find objects before giving up. Parameters retries Maximum number of detection retries. Implementation Send RC_PICKIT_LOOK_FOR_OBJECTS_WITH_RETRIES and don’t wait for a response.

pickit_process_image()

Trigger a Pickit object detection without image capture using the currently active setup and product configuration. This function uses the latest captured camera image, which typically comes from calling pickit_capture_image() just before. Refer to the documentation of pickit_capture_image() to learn more about the recommended usage of this function. Implementation Send RC_PICKIT_PROCESS_IMAGE and don’t wait for a response.

pickit_get_next_object()

Request the next detected object. A single object detection run might yield the detection of multiple objects. This function allows to request the next available object, if any, without the need of triggering a new detection and the time overhead it entails. It’s recommended to use this command only when objects in the detection region have not moved (significantly) since the last detection took place. A good example of when to use pickit_get_next_object() is when a detection is unreachable by the robot. An example of when using it is not recommended would be the following bin picking scenario: Trigger a Pickit detection that finds multiple objects. The first object is picked. Since objects are randomly placed in bin, neighboring objects move and fall into place. Call pickit_get_next_object() and attempt to pick next object. If the next object is one of the neighboring parts that moved, the pick is likely to fail. When the objects in the detection region have moved, it’s better to re-trigger object detection instead (by calling pickit_find_objects_with_retries(retries), for instance). Implementation Send RC_PICKIT_NEXT_OBJECT and don’t wait for a response.

Note

You must collect the results of each request before sending a new one.

Tip

Note that all calls except pickit_get_next_object() (which is very fast) trigger a Pickit object detection run, which can potentially take multiple seconds.

In your robot program, it’s encouraged to perform robot motions or other non-Pickit-detection actions between calls to these functions and pickit_get_result() to save cycle time in your application. Refer to the pick and place loop of the simple pick and place program to learn more.

This function collects the results of the above requests:

pickit_get_result()

Wait for Pickit to reply with detection results. pickit_get_result() should always be paired to one of the above functions. Implementation Block until a reply from Pickit is received. When an object has been found, populate the output global variables with valid content, which requires these steps: Populate PickitPick, PickitObjType, PickitObjDim, and PickitObjAge from the received response. Send RC_PICKIT_GET_PICK_POINT_DATA and wait for a new response. If the response status is PICKIT_GET_PICK_POINT_DATA_OK, populate from the newly received response these variables: PickitPickId, PickitPickRefId and PickitPickOff. Otherwise raise a non-recoverable error and halt program execution. The success of the original request can be queried by calling pickit_object_found() after pickit_get_result().

The following functions are short-running. The robot sends a request to Pickit and waits for the response, which only consists of a status code.

pickit_is_running()

Check whether robot mode is enabled in Pickit. Implementation Send RC_PICKIT_CHECK_MODE and wait for a response. Return True if the response status is PICKIT_ROBOT_MODE. All functions in this article except pickit_find_calibration_plate() require robot mode to be enabled, which can be done from the Pickit web interface.

pickit_configure(setup, product)

Loads the specified Pickit configuration (setup and product). Parameters IDs of valid setup and product configurations currently available in the connected Pickit system. Available configuration IDs are listed in the Pickit web interface. Implementation Send RC_PICKIT_CONFIGURE and wait for a response. If the response status is not PICKIT_CONFIG_OK, a non-recoverable error is raised and program execution is halted. This can happen, for instance, when the parameters correspond to non-existing setup or product IDs.

pickit_save_snapshot()

Save a snapshot with the latest detection results. For an example usage, refer to the description of the after_end hook of the pick and place program. Implementation Send RC_PICKIT_SAVE_SNAPSHOT and wait for a response. Return True if the response status is PICKIT_SAVE_SNAPSHOT_OK.

pickit_capture_image()

Capture a camera image without triggering object detection. This function blocks until image capture has completed, and is especially useful when working with a robot-mounted camera, where the robot must remain stationary during image capture, but not during detection. This function is meant to be used prior to calling pickit_process_image(). For fixed camera mounts, it’s usually more convenient to call functions that combine image capture and processing, like pickit_find_objects() or pickit_find_objects_with_retries(retries). Implementation Send RC_PICKIT_CAPTURE_IMAGE and wait for a response. Return True if the response status is PICKIT_IMAGE_CAPTURED.

pickit_build_background()

Build the background cloud used by some of the advanced Region of Interest filters. Calling this function will trigger a camera capture. So, if the camera mount is fixed, the robot must not occlude the camera view volume. If instead the camera is robot-mounted, the robot must be in the detection point (more). Implementation Send RC_PICKIT_BUILD_BACKGROUND and wait for a response. Return True if the response status is PICKIT_BUILD_BKG_CLOUD_OK.

The following functions are helpers that don’t require extra communication with Pickit. They use the information received in the most recent detection results. These helpers are typically used as part of conditional expressions, such as an if statement.

pickit_empty_roi()

Check if the last call to pickit_get_result() detected an empty Region of Interest (ROI). When pickit_object_found() returns False, it can be due to: The ROI is not empty, but Pickit doesn’t detect the active product. The ROI is empty. Use this function if you need to discriminate between these two situations. Return True if Pickit detected an empty ROI.

pickit_object_found()

Check if the last call to pickit_get_result() produced valid detection results. Return True if detection results are available. When results are available, the output global variables have valid contents. This function returns False when Pickit replied with no detection results (nominal usecase); or if pickit_get_result()  was called without a previous detection request to Pickit (should be avoided, as it makes no sense).

pickit_is_reachable(PickitPick, PrePick, PostPick)

Check if the pick point and the linear approach and retreat trajectories are reachable by the robot. Parameters PickitPick: The pick point. PrePick: Point determining the linear approach motion to the pick point. PostPick: Point determining the linear retreat motion from the pick point. This is an optional, but recommended function that requires the robot programming language to expose functionality like joint limits, inverse kinematics and safety plane checks. The exact implementation is open and left to the developer: The simplest implementation only checks the validity of the three input points. A more advanced implementation can additionally check points linearly sampled in the approach and retreat trajectories, that is, between PrePick → PickitPick and PickitPick → PostPick. Return True if the checked points are reachable by the robot.

pickit_no_image_captured()

Check if object detection was unsuccessful due to a failure to capture a camera image. When this is the case, it typically indicates a hardware disconnection issue, such as a loose connector or broken cable. This function can be used as trigger to send an alarm to a higher level monitoring system. Return True if object detection was unsuccessful due to a failure to capture a camera image.

pickit_remaining_objects()

Get the number of remaining detected objects. After calling pickit_get_result(), this function returns the total number of object detections minus one, as the first object data is available through the output global variables. This value is also equal to the number of times pickit_get_next_object() can be called. As such, the returned value decreases with each call to pickit_get_next_object(). Return Number of remaining object detections available for query.

Calibration functions

There is a single function meant to be used in a calibration robot program, and not in a pick and place program.

pickit_find_calibration_plate()

Trigger detection of the robot-camera calibration plate. This function requires the Pickit web interface to be in the Calibration page, hence robot mode should be disabled. Implementation Send RC_PICKIT_FIND_CALIB_PLATE and wait for a response. If the respose status is not PICKIT_FIND_CALIB_PLATE_OK, a non-recoverable error is raised and program execution is halted.