Warning
You are reading the documentation for an older Pickit release (2.1). Documentation for the latest release (3.3) can be found here.
The low-level communication structures between a robot and Pickit on the TCP/IP socket level
This article documents the low-level communication structures between a robot and Pickit on the TCP/IP socket level. This information is only required to interface a new robot brand to Pickit.
Protocol
The robot Pickit communication is based on TCP/IP socket communication. Pickit is the server (slave) and the robot is the client (master). Hence, the robot is responsible for opening the socket communication and needs to know:
The IP address of the Pickit port labelled ROBOT,
The port number which is 5001 by default.
Pickit communicates data packages using the network byte order convention (Big Endian).
Above you can find an example of the different behaviour between network byte order and host byte order.
Warning
The robot client sends requests using the command message, and the Pickit server answers with the response message. These messages have a static size, and don’t have a begin and/or end character. While the TCP/IP protocol prevents data loss, the robot client implementation is responsible for keeping track of the boundary between messages by counting the number of sent/received bytes and comparing with the expected message size.
There is no explicit coupling between a command and a response in the communication. Therefore, the robot client should not send a second command before having confirmed that a response to the first command has been received. It is the responsibility of the client implementation or the robot program to prevent this from happening.
The Pickit server only sends response messages after having received an explicit request from the robot client in a command message. Apart from this request-response exchange, Pickit also expects to receive periodic robot flange pose updates from the client. The rate of these periodic updates depends on the robot brand, but is typically in the range of 10 to 50 messages per second. Note that both command requests and robot flange pose updates use the same message structure, robot_to_pickit_data, described in more detail in the command message section. The structure of the response messages, pickit_to_robot_data, is described in the response message section.
A Pickit data package consists of a number of int32s in network byte order. As such, floating-point data like distances and angles are multiplied by a factor MULT before being sent as an int32, and are divided by MULT after being received. This integer conversion factor MULT has the value of 10000.
For position information Pickit sends and expects to receive values in meter before/after compensating with the MULT factor. For the orientation information it depends on the robot type. For UR radians are used, for ABB unit quaternions are used and for the other brands degrees are used. Also here, this is the unit before/after compensating with the MULT factor.
Command message from robot to Pickit
The data package communicated to Pickit contains the actual robot pose, an (optional) robot command and the desired configuration. The latter allows to request Pickit to configure according to an existing setup and product type. To find out the number associated to a setup or product file, go to the Setup page (for setup file) or Detection or Picking page (for product file) and press the Open button on top of the page. The number appears next to the file name.
In summary, the robot is required to send this structure to the Pickit Socket Interface:
struct robot_to_pickit_data {
int32 actual_pose[7];
int32 command;
int32 setup;
int32 product;
meta_data meta;
};
Metadata fields are documented in the Message metadata section of this article. The remaining fields are explained below.
Field name |
Type / units |
Comment |
|---|---|---|
actual_pose |
int32[7] [0-2] : position (m) [3-6] : orientation (rad,o,/) |
Actual robot flange pose in robot base frame consisting of position in Cartesian space and orientation in axis angle, quaternion or Euler angles convention. The orientation convention and units depend on the robot brand.
To populate array elements, each value has to be multiplied by the MULT factor before being sent.
|
command |
int32 |
A single command from robot to Pickit. |
RC_PICKIT_NO_COMMAND |
Use when sending a periodic pose update. Pickit does not reply to these requests. |
|
RC_PICKIT_CHECK_MODE |
Check the current mode of Pickit (RUNNING, IDLE or CALIBRATION) |
|
RC_PICKIT_FIND_CALIB_PLATE |
Trigger the localisation of the Pickit camera-to-robot calibration plate. |
|
RC_PICKIT_LOOK_FOR_OBJECTS |
Trigger camera to look for new objects in its current workspace. Pickit will respond with the amount of objects currently found in the workspace, which may be zero, as well as the pose of the first object if one was detected. |
|
RC_PICKIT_NEXT_OBJECT |
Request camera to return next localised object stored in the Pickit buffer and which was found with RC_PICKIT_LOOK_FOR_OBJECTS. Please note that the RC_PICKIT_LOOK_FOR_OBJECTS command already returns the first object, if one was detected. |
|
RC_PICKIT_CONFIGURE |
Request Pickit to load a specific setup and product type. |
|
RC_PICKIT_SAVE_SCENE |
Request Pickit to save the latest image as well as the whole configuration into a file for later diagnosis. |
|
setup |
int32 |
A number matching to a setup known by the Pickit system. Used only when the command type is RC_PICKIT_CONFIGURE. |
product |
int32 |
A number matching to a product type known by the Pickit system. Used only when the command type is RC_PICKIT_CONFIGURE. |
Below are the values for the robot command constants expected by Pickit:
#define RC_PICKIT_NO_COMMAND -1
#define RC_PICKIT_CHECK_MODE 0
#define RC_PICKIT_FIND_CALIB_PLATE 10
#define RC_PICKIT_LOOK_FOR_OBJECTS 20
#define RC_PICKIT_NEXT_OBJECT 30
#define RC_PICKIT_CONFIGURE 40
#define RC_PICKIT_SAVE_SCENE 50
All command messages (not just periodic pose updates) should contain a valid actual_pose field.
Response message from Pickit to robot
Except for the RC_PICKIT_NO_COMMAND command, each robot command sent to Pickit will result in one response message from Pickit. These messages contain a Pickit status value as well as the actual object data for one object.
The robot receives this structure from the Pickit interface:
struct pickit_to_robot_data {
int32 object_pose[7];
int32 object_age;
int32 object_type;
int32 object_dimensions[3];
int32 objects_remaining;
int32 status;
meta_data meta;
};
Metadata fields are documented in the Message metadata section of this article. The remaining fields are explained below.
Field name |
Type / units |
Comment |
|---|---|---|
object_pose |
int32[7] [0-2] : position (m) [3-6] : orientation (rad, o,/) |
An object pose expressed relative to the robot base frame consisting of position in cartesian space and orientation in axis angle, quaternion or Euler angles convention. This convention as well as the units depend on the robot brand. When reading array elements, each value has to be divided by the MULT factor. |
object_age |
int32 (seconds) |
The amount of time that has passed between the capturing of the camera data and the moment the object information is sent to the robot. This value has to be divided by the MULT factor. |
object_type |
int32 |
The type of object detected at object_pose |
PICKIT_TYPE_SQUARE |
A square has been detected with center at object_pose |
|
PICKIT_TYPE_RECTANGLE |
A rectangle has been detected with center at object_pose |
|
PICKIT_TYPE_CIRCLE |
A circle has been detected with center at object_pose |
|
PICKIT_TYPE_ELLIPSE |
An ellipse has been detected with center at object_pose |
|
PICKIT_TYPE_CYLINDER |
A cylinder has been detected with center at object_pose |
|
PICKIT_TYPE_SPHERE |
A sphere has been detected with center at object_pose |
|
PICKIT_TYPE_POINT_CLOUD |
A Pickit Teach model has been detected |
|
PICKIT_TYPE_BLOB |
An object without a specified shape has been detected |
|
object_dimensions |
int32[3] [0]: length or diameter (m) [1]: width or diameter (m) [2]: height (m) |
PICKIT_TYPE_SQUARE
[0] and [1] contain the side length of the square
PICKIT_TYPE_RECTANGLE
[0] and [1] respectively contain the length and width of the rectangle
PICKIT_TYPE_CIRCLE
[0] and [1] contain the diameter of the circle
PICKIT_TYPE_ELLIPSE
[0] and [1] respectively contain the length and width of the ellipse
PICKIT_TYPE_CYLINDER
[0] and [1] respectively contain cylinder length and diameter
PICKIT_TYPE_SPHERE
[0] and [1] contain the diameter of the sphere
PICKIT_TYPE_POINT_CLOUD
[0], [1] and [2] respectively contain the length, width and height of the point cloud bounding box
PICKIT_TYPE_BLOB
[0], [1] and [2] respectively contain the length, width and height of the blob bounding box
When reading array elements, each value has to be divided by the MULT factor. |
objects_remaining |
int32 |
Only one object per pickit_to_robot_data message can be communicated. If this field is non-zero, it contains the number of remaining objects that can be sent in next messages to the robot. |
status |
int32 |
Contains the Pickit status or a response to previously received robot commands. |
PICKIT_UNKNOWN_COMMAND |
Pickit received an unknown command. |
|
PICKIT_RUNNING_MODE |
Pickit is in running mode. |
|
PICKIT_IDLE_MODE |
Pickit is in idle mode. |
|
PICKIT_CALIBRATION_MODE |
Pickit is in calibration mode. |
|
PICKIT_FIND_CALIB_PLATE_OK |
The calibration plate has been successfully detected. |
|
PICKIT_FIND_CALIB_PLATE_FAILED |
The calibration plate has not been detected. |
|
PICKIT_OBJECT_FOUND |
A pickable object has been detected. |
|
PICKIT_NO_OBJECTS |
No pickable objects were detected. |
|
PICKIT_NO_IMAGE_CAPTURED |
Pickit failed to capture a camera image, most possibly due to a hardware failure (e.g. disconnected camera). |
|
PICKIT_EMPTY_ROI |
An empty Region of Interest (ROI) has been detected. |
|
PICKIT_CONFIG_OK |
Loading the requested Pickit configuration suceeded. |
|
PICKIT_CONFIG_FAILED |
Loading the requested Pickit configuration failed. |
|
PICKIT_SAVE_SCENE_OK |
Pickit snapshot saving succeeded. |
|
PICKIT_SAVE_SCENE_FAILED |
Pickit snapshot saving failed. |
Below are the values of the Pickit status constants communicated by Pickit:
#define PICKIT_UNKNOWN_COMMAND -99
#define PICKIT_RUNNING_MODE 0
#define PICKIT_IDLE_MODE 1
#define PICKIT_CALIBRATION_MODE 2
#define PICKIT_FIND_CALIB_PLATE_OK 10
#define PICKIT_FIND_CALIB_PLATE_FAILED 11
#define PICKIT_OBJECT_FOUND 20
#define PICKIT_NO_OBJECTS 21
#define PICKIT_NO_IMAGE_CAPTURED 22
#define PICKIT_EMPTY_ROI 23
#define PICKIT_CONFIG_OK 40
#define PICKIT_CONFIG_FAILED 41
#define PICKIT_SAVE_SCENE_OK 50
#define PICKIT_SAVE_SCENE_FAILED 51
Below are the values of the object type constants communicated by Pickit:
#define PICKIT_TYPE_SQUARE 21
#define PICKIT_TYPE_RECTANGLE 22
#define PICKIT_TYPE_CIRCLE 23
#define PICKIT_TYPE_ELLIPSE 24
#define PICKIT_TYPE_CYLINDER 32
#define PICKIT_TYPE_SPHERE 33
#define PICKIT_TYPE_POINT_CLOUD 35 // See remark below for Teach on 1.9+ versions.
#define PICKIT_TYPE_BLOB 50
Warning
From version 1.9+, PICKIT_TYPE_POINT_CLOUD is no longer 35 with the Pickit Teach detection engine, but representing the ID Teach model the object was detected from.
Message metadata
To guarantee correct interpretation of the data on both the robot and the Pickit side, the following metadata is always sent along with the structures:
struct meta_data {
int32 robot_type;
int32 interface_version;
};
Each field is explained below. All int32 are expressed in Network Byte Format.
Field name |
Type / units |
Comment |
|---|---|---|
robot_type |
int32 |
The type of robot Pickit is connected to:
|
interface_version |
int32 |
The version of the robot-Pickit communication.
To get this number, all dots are removed from the actual version number.
The current version is
1.1, so the communicated value is 11. |
To add support for a robot type not adhering to one of the above conventions, it’s recommended to use the GENERIC (quaternions) convention above. The robot-side interface would then take the responsibility of converting back and forth between the representation used by Pickit and the robot.