Warning

You are reading the documentation for an older Pickit release (1.10). Documentation for the latest release (3.4) can be found here.

The low-level communication structures between a robot and Pickit on the TCP/IP socket level

This article documents the low-level communication structures between a robot and Pickit on the TCP/IP socket level. This information is only required to interface a new robot brand to Pickit.

Protocol

The robot Pickit communication is based on TCP/IP socket communication. Pickit is the server (slave) and the robot is the client (master). Hence, the robot is responsible for opening the socket communication and needs to know:

The IP address of the Pickit port labelled ROBOT,
The port number which is 5001 by default.
Pickit communicates data packages using the network byte order convention (Big Endian).

Above you can find an example of the different behaviour between network byte order and host byte order.

Warning

The robot client sends requests using the command message, and the Pickit server answers with the response message. These messages have a static size, and don’t have a begin and/or end character. While the TCP/IP protocol prevents data loss, the robot client implementation is responsible for keeping track of the boundary between messages by counting the number of sent/received bytes and comparing with the expected message size.

There is no explicit coupling between a command and a response in the communication. Therefore, the robot client should not send a second command before having confirmed that a response to the first command has been received. It is the responsibility of the client implementation or the robot program to prevent this from happening.

The Pickit server only sends response messages after having received an explicit request from the robot client in a command message. Apart from this request-response exchange, Pickit also expects to receive periodic robot flange pose updates from the client. The rate of these periodic updates depends on the robot brand, but is typically in the range of 10 to 50 messages per second. Note that both command requests and robot flange pose updates use the same message structure, robot_to_pickit_data, described in more detail in the command message section. The structure of the response messages, pickit_to_robot_data, is described in the response message section.

A Pickit data package consists of a number of int32s in network byte order. As such, floating-point data like distances and angles are multiplied by a factor MULT before being sent as an int32, and are divided by MULT after being received. This integer conversion factor MULT has the value of 10000.

For position information Pickit sends and expects to receive values in meter before/after compensating with the MULT factor. For the orientation information it depends on the robot type. For UR radians are used, for ABB unit quaternions are used and for the other brands degrees are used. Also here, this is the unit before/after compensating with the MULT factor.

Command message from robot to Pickit

The data package communicated to Pickit contains the actual robot pose, an (optional) robot command and the desired configuration. The latter allows to request Pickit to configure according to an existing setup and product type. What number matches to what file, can be derived from the Configuration page of the Pickit web interface.

In summary, the robot is required to send this structure to the Pickit Socket Interface:

struct robot_to_pickit_data {
    int32 actual_pose[7];
    int32 command;
    int32 setup;
    int32 product;
    meta_data meta;
};

Metadata fields are documented in the Message metadata section of this article. The remaining fields are explained below.

Field name	Type / units	Comment
actual_pose	int32[7] [0-2] : position (m) [3-6] : orientation (rad,o,/)	Actual robot flange pose in robot base frame consisting of position in Cartesian space and orientation in axis angle, quaternion or Euler angles convention. The orientation convention and units depend on the robot brand. To populate array elements, each value has to be multiplied by the MULT factor before being sent.
command	int32	A single command from robot to Pickit.
	RC_PICKIT_NO_COMMAND	Use when sending a periodic pose update. Pickit does not reply to these requests.
	RC_PICKIT_FIND_CALIB_PLATE	Trigger the localisation of the Pickit camera-to-robot calibration plate.
	RC_PICKIT_LOOK_FOR_OBJECTS	Trigger camera to look for new objects in its current workspace. Pickit will respond with the amount of objects currently found in the workspace, which may be zero.
	RC_PICKIT_NEXT_OBJECT	Request camera to return next localised object stored in the Pickit buffer and which was found with RC_PICKIT_LOOK_FOR_OBJECTS.
	RC_PICKIT_CONFIGURE	Request Pickit to load a specific setup and product type.
	RC_PICKIT_SAVE_SCENE	Request Pickit to save the latest image as well as the whole configuration into a file for later diagnosis.
setup	int32	A number matching to a setup known by the Pickit system. Used only when the command type is RC_PICKIT_CONFIGURE.
product	int32	A number matching to a product type known by the Pickit system. Used only when the command type is RC_PICKIT_CONFIGURE.

Below are the values for the robot command constants expected by Pickit:

#define RC_PICKIT_NO_COMMAND         -1
#define RC_PICKIT_CHECK_MODE         0
#define RC_PICKIT_FIND_CALIB_PLATE   10
#define RC_PICKIT_LOOK_FOR_OBJECTS   20
#define RC_PICKIT_NEXT_OBJECT        30
#define RC_PICKIT_CONFIGURE          40
#define RC_PICKIT_SAVE_SCENE         50

All command messages (not just periodic pose updates) should contain a valid actual_pose field.

Response message from Pickit to robot

Except for the RC_PICKIT_CALIBRATE command, each robot command sent to Pickit will result in one response message from Pickit. These messages contain a Pickit status value as well as the actual object data for one object.

The robot receives this structure from the Pickit interface:

struct pickit_to_robot_data {
     int32 object_pose[7];
     int32 object_age;
     int32 object_type;
     int32 object_dimensions[3];
     int32 objects_remaining;
     int32 status;
     meta_data meta;
};

Metadata fields are documented in the Message metadata section of this article. The remaining fields are explained below.

Field name	Type / units	Comment
object_pose	int32[7] [0-2] : position (m) [3-6] : orientation (rad, o,/)	An object pose expressed relative to the robot base frame consisting of position in cartesian space and orientation in axis angle, quaternion or Euler angles convention. This convention as well as the units depend on the robot brand. When reading array elements, each value has to be divided by the MULT factor.
object_age	int32 (seconds)	The amount of time that has passed between the capturing of the camera data and the moment the object information is sent to the robot. This value has to be divided by the MULT factor.
object_type	int32	The type of object detected at object_pose
	PICKIT_TYPE_SQUARE	A square has been detected with center at object_pose
	PICKIT_TYPE_RECTANGLE	A rectangle has been detected with center at object_pose
	PICKIT_TYPE_CIRCLE	A circle has been detected with center at object_pose
	PICKIT_TYPE_ELLIPSE	An ellipse has been detected with center at object_pose
	PICKIT_TYPE_CYLINDER	A cylinder has been detected with center at object_pose
	PICKIT_TYPE_SPHERE	A sphere has been detected with center at object_pose
	PICKIT_TYPE_POINT_CLOUD	A Pickit Teach model has been detected
	PICKIT_TYPE_BLOB	An object without a specified shape has been detected
object_dimensions	int32[3] [0]: length or diameter (m) [1]: width or diameter (m) [2]: height (m)	PICKIT_TYPE_SQUARE [0] and [1] contain the side length of the square PICKIT_TYPE_RECTANGLE [0] and [1] respectively contain the length and width of the rectangle PICKIT_TYPE_CIRCLE [0] and [1] contain the diameter of the circle PICKIT_TYPE_ELLIPSE [0] and [1] respectively contain the length and width of the ellipse PICKIT_TYPE_CYLINDER [0] and [1] respectively contain cylinder length and diameter PICKIT_TYPE_SPHERE [0] and [1] contain the diameter of the sphere PICKIT_TYPE_POINT_CLOUD [0], [1] and [2] respectively contain the length, width and height of the point cloud bounding box PICKIT_TYPE_BLOB [0], [1] and [2] respectively contain the length, width and height of the blob bounding box When reading array elements, each value has to be divided by the MULT factor.
objects_remaining	int32	Only one object per pickit_to_robot_data message can be communicated. If this field is non-zero, it contains the number of remaining objects that can be sent in next messages to the robot.
status	int32	Contains the Pickit status or a response to previously received robot commands.
	PICKIT_COMMAND_OK	Generic all nominal response code.
	PICKIT_COMMAND_FAILED	Generic error response code.The previous command could not be processed. See log messages for diagnostics.
	PICKIT_OBJECT_FOUND	A pick position has been found
	PICKIT_NO_OBJECTS	No objects for picking were detected. See status messages for diagnostics.

Below are the values of the Pickit status constants communicated by Pickit:

#define PICKIT_RUNNING_MODE                0
#define PICKIT_IDLE_MODE                   1
#define PICKIT_CALIBRATION_MODE            2

#define PICKIT_FIND_CALIB_PLATE_OK        10
#define PICKIT_FIND_CALIB_PLATE_FAILED    11
#define PICKIT_OBJECT_FOUND               20
#define PICKIT_NO_OBJECTS                 21
#define PICKIT_NO_IMAGE_CAPTURED          22

#define PICKIT_CONFIG_OK                  40
#define PICKIT_CONFIG_FAILED              41
#define PICKIT_SAVE_SCENE_OK              50
#define PICKIT_SAVE_SCENE_FAILED          51

Below are the values of the object type constants communicated by Pickit:

#define PICKIT_TYPE_SQUARE               21
#define PICKIT_TYPE_RECTANGLE            22
#define PICKIT_TYPE_CIRCLE               23
#define PICKIT_TYPE_ELLIPSE              24
#define PICKIT_TYPE_CYLINDER             32
#define PICKIT_TYPE_SPHERE               33
#define PICKIT_TYPE_POINT_CLOUD          35 ## See remark below for Teach on 1.9+ versions
#define PICKIT_TYPE_BLOB                 50

Warning

From version 1.9+, PICKIT_TYPE_POINT_CLOUD is no longer 35 with the Pickit Teach detection engine, but representing the ID Teach model the object was detected from.

Message metadata

To guarantee correct interpretation of the data on both the robot and the Pickit side, the following metadata is always sent along with the structures:

struct meta_data {
    int32 robot_type;
    int32 interface_version;
};

Each field is explained below. All int32 are expressed in Network Byte Format.

Field name

Type / units

Comment

robot_type

int32

The type of robot Pickit is connected to:

UNIVERSAL ROBOT → Angle-axis

ABB, GENERIC → Quaternions (w,x,y,z)

STAUBLI → Euler Angles (x-y’-z”)

FANUC, YASKAWA → Fixed Angles (x-y-z)

KUKA → Euler Angles (z-y’-x”)

COMAU → Euler Angles (z-y’-z”)

interface_version

int32

The version of the robot-Pickit communication.
To get this number, all dots are removed from the actual version number.
Example: Version 1.1 -> 11

To add support for a robot type not adhering to one of the above conventions, it’s recommended to use the GENERIC (quaternions) convention above. The robot-side interface would then take the responsibility of converting back and forth between the representation used by Pickit and the robot.