ABB example bin picking program
This example program requires that Pickit is installed and set up with your robot. For installation instructions, please refer to the ABB installation and setup article.
Example program
By default no main() module is loaded. One of the Pickit example programs can be loaded from HOME > ABB_Pick-it > Pick-it > Application examples.
Note
These example programs only works with Pickit software version of 2.2 or greater. If you are using a software version prior 2.2, please contact us at support@pickit3d.com, and we will assist you in finding a solution.
Below the main() part of the example program Pickit_example_bin_picking is shown. This program is specific to bin picking applications, for a simpler example program see ABB example simple pick and place. More information about the logic behind this example program can be found in the robot-independent bin picking program.
PROC main()
VAR bool pickSucceeded:=FALSE;
VAR bool pickingCompleted:=FALSE;
VAR bool isReachable:=TRUE;
VAR num numNoObjects:= 0;
VAR num numPickFail:= 0;
check_robot_mode;
before_start;
pickit_configure setup, product; ! Load desired setup and product configuration.
! First detection.
goto_detection;
detect_objects;
WaitUntil pickit_get_results();
! Main loop.
WHILE NOT pickingCompleted DO
IF pickit_object_found() THEN
numNoObjects:= 0; ! Reset counter.
! Calculate approach and retreat.
PickitPick:=pickit_get_pose();
PrePick:=RelTool(PickitPick,0,0,prePickOffset);
PostPick:=RelTool(PickitPick,0,0,postPickOffset);
BinEntry:=PrePick;
BinEntry.trans.z:=binEntryZ;
BinExit:=PostPick;
BinExit.trans.z:=binExitZ;
isReachable:=pickit_is_pose_reachable(BinEntry,toolPickit) AND
pickit_is_pose_reachable(PrePick,toolPickit) AND
pickit_is_pose_reachable(PickitPick,toolPickit) AND
pickit_is_pose_reachable(PostPick,toolPickit) AND
pickit_is_pose_reachable(BinExit,toolPickit);
IF isReachable THEN
! Object is pickable!
pick;
pickSucceeded:=is_object_picked();
goto_detection;
detect_objects;
IF pickSucceeded THEN
numPickFail := 0;
place;
ELSE
! Picking failed, skip place.
ErrWrite\I,"Mispick.","Pick failed, skipping place motion.";
INCR numPickFail; ! Increment counter.
on_pick_failure;
ENDIF
WaitUntil pickit_get_results();
ELSE
ErrWrite\I,"Object is unreachable, going for next object.","";
pickit_next_object;
WaitUntil pickit_get_results();
ENDIF
ELSE
ErrWrite\I,"There are no pickable object.","Retrying detecting.";
INCR numNoObjects;
goto_detection;
detect_objects;
WaitUntil pickit_get_results();
ENDIF
! Exit loop if picking is complete.
IF Pickit_roi_empty() THEN
ErrWrite\I,"The ROI is empty.",
"Picking complete.";
pickingCompleted:=TRUE;
ELSEIF Pickit_no_image_captured() THEN
ErrWrite "Failed to capture a camera image.",
"Picking complete.";
pickingCompleted:=TRUE;
ELSEIF NOT isReachable THEN
ErrWrite\I,"All detected objects are unreachable.",
"Picking complete.";
pickingCompleted:=TRUE;
ELSEIF numNoObjects >= maxNoObjects THEN
ErrWrite\I,"Too many consecutive detections with no object found.",
"Picking complete.";
pickingCompleted:=TRUE;
ELSEIF numPickFail >= maxPickFail THEN
ErrWrite\I,"Too many consecutive failed picks.",
"Picking complete.";
pickingCompleted:=TRUE;
ENDIF
ENDWHILE
after_end;
ERROR
IF ERRNO=ERR_COLL_STOP THEN
StartMove; ! Allow to start moving again.
TRYNEXT; ! Exit Pick routine and go to next instruction.
ENDIF
ENDPROC
Required program inputs
Before main() the user is required to enter values for all the constants which are going to be used inside the program.
! Detection settings.
CONST num setup:=1;
CONST num product:=2;
CONST bool isCameraRobotMounted:=TRUE;
! Pick sequence settings.
PERS tooldata toolPickit:=[TRUE,[[0,0,0],[1,0,0,0]],[0.001,[0,0,0.001],[1,0,0,0],0,0,0]];
CONST num prePickOffset:=-100;
CONST num postPickOffset:=-100;
CONST num binEntryZ:=500;
CONST num binExitZ:=500;
CONST speeddata extractionSpeed:=v500;
CONST speeddata approachSpeed:=v200;
CONST speeddata fastSpeed:=v800;
! Completion condition settings.
CONST num maxNoObjects:= 4; ! Maximum allowed consecutive detections with no object found.
CONST num maxPickFail:= 4; ! Maximum allowed consecutive failed picks.
! Fixed poses.
CONST robtarget Detect:=[[0,0,0],[1,0,0,0],[0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
CONST robtarget Dropoff:=[[0,0,0],[1,0,0,0],[0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
CONST jointtarget AbovePickArea:=[[0,0,0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
CONST jointtarget HomePose:=[[0,0,0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
Detection settings
setup: the desired Pickit setup id.
product: the desired Picket product id.
isCameraRobotMounted: set as
TRUEif the camera is mounted in the robot arm, whileFALSEif it is in a fixed position.
Pick sequence settings
toolPickit: this will be the tool used in all the motion commands, the user need to set the actual TCP value.
prePickOffset: the z-offset for
PrePickrelative to the tool frame.postPickOffset: the z-offset for
PostPickrelative to the tool frame.binEntryZ: the z absolute value for
BinEntryrelative to the robot base.binExitZ: the z absolute value for
BinExitrelative to the robot base.
extractionSpeed, approachSpeed and fastSpeed: user defined speed data used in the robot motion.
Completion condition settings
maxNoObjects: maximum allowed consecutive detections with no object found.
maxPickFail: maximum allowed consecutive failed picks.
Define fixed points
Detect: where to perform object detection from. For more details see Robot position during image capture.
Dropoff: where to place the picked part.
AbovePickArea: A point roughly above the pick area from which the above two can be reached without collision.
HomePose: the start pose of the application.
Note
At least one of the fixed poses should be saved in joint space (jointtarget). This is because in every cycle we want to ensure that the robot goes back to the same joint positions and does not make a complete turn in some joint angles (for example J6 from 0° to 360°)
Program procedures
Below are shown all the procedure that will be called by the main() function and are related to Pickit functionality and the gripper status.
PROC before_start()
MoveAbsJ HomePose,v500,z100,toolPickit;
gripper_release;
! <<< Add Initialization logic here.
ENDPROC
PROC after_end()
Pickit_save_snapshot;
Break;
ENDPROC
PROC goto_detection()
MoveJ Detect,fastSpeed,z0,toolPickit;
IF isCameraRobotMounted THEN
! Wait for no vibrations in the camera.
! Adapt the duration depending on your application.
WaitTime\InPos,0.2;
ENDIF
ENDPROC
PROC detect_objects()
IF isCameraRobotMounted THEN
pickit_capture_image;
pickit_process_image;
ELSE
! Fixed camera.
pickit_look_for_object;
ENDIF
ENDPROC
PROC pick()
!Moving to an object.
MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
MoveL BinEntry,fastSpeed,z100,toolPickit;
MoveL PrePick,extractionSpeed,z20,toolPickit;
MoveL PickitPick,approachSpeed,fine,toolPickit;
gripper_grasp;
MoveL PostPick,extractionSpeed,z20,toolPickit;
MoveL BinExit,extractionSpeed,z100,toolPickit;
MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
ERROR
IF ERRNO=ERR_COLL_STOP THEN
ErrWrite\W,"Collision detected, trying to recover.","";
gripper_release;
StopMove;
ClearPath;
StorePath;
MotionSup\Off;
MoveL BinExit,extractionSpeed,z100,toolPickit;
MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
RestoPath;
MotionSup\On;
RAISE ;
ENDIF
ENDPROC
PROC place()
MoveJ Dropoff,fastSpeed,fine,toolPickit;
gripper_release;
WaitTime\InPos,0.5;
ENDPROC
PROC on_pick_failure()
gripper_release;
ENDPROC
In pick() the user needs to add the application-specific grasping logic.
In the same way the releasing logic is needed in place(), before_start(), on_pick_failure() and in the collision handling inside pick().
Advanced topics
Collision recovery
To avoid production downtime, it is recommended to use a collision recovery strategy in the robot program. This way, if a collision is detected by the robot while trying to pick a part, the robot does not raise an error requiring operator intervention. Instead, the robot goes back to the detection point and tries to pick another part.
In ABB, this is done using the motion supervision error handling module. To use this functionality, you must first set the system parameter CollisionErrorHandling to YES in Control Panel > Configuration => Topic > Controller > General Rapid.
See below for an example pick procedure with collision recovery.
PROC pick()
!Moving to an object.
MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
MoveL BinEntry,fastSpeed,z100,toolPickit;
MoveL PrePick,extractionSpeed,z20,toolPickit;
MoveL PickitPick,approachSpeed,fine,toolPickit;
gripper_grasp;
MoveL PostPick,extractionSpeed,z20,toolPickit;
MoveL BinExit,extractionSpeed,z100,toolPickit;
MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
ERROR
IF ERRNO=ERR_COLL_STOP THEN
ErrWrite\W,"Collision detected, trying to recover.","";
gripper_release;
StopMove;
ClearPath;
StorePath;
MotionSup\Off;
MoveL BinExit,extractionSpeed,z100,toolPickit;
MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
RestoPath;
MotionSup\On;
RAISE ;
ENDIF
ENDPROC
Calibration validation
In applications with robot-mounted cameras, it is a good practice to perform calibration validation before running the program. An incorrect or outdated calibration can lead to inaccurate picking.
An automated calibration validation step can be added to the robot program in the before_start() procedure, before any detection is performed:
! Add the position where the camera on robot can see the fixed calibration plate.
CONST robtarget LookAtTheMarkers:=[[0,0,0],[1,0,0,0],[0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
VAR num distance_threshold := 10; ! mm use 4 in case of a HD camera
VAR num angle_threshold := 2.0; ! degrees use 1 in case of a HD camera
PROC before_start()
MoveAbsJ HomePose,v500,z100,toolPickit;
gripper_release;
IF isCameraRobotMounted THEN
MoveJ LookAtTheMarkers, v100, fine, PICKIT_TOOL;
WaitTime 1.0;
IF (NOT pickit_validate_calibration(distance_threshold, angle_threshold)) THEN
ErrWrite \I, "Pickit: ValidateCalibration", "High calibration validation error, stopping";
UIMsgBox "High calibration validation error, stopping";
stop;
ENDIF
ENDIF
ENDPROC
The calibration plate needs to be fixed to a wall or to the ground, such that its location with respect to the robot base is exactly the same as when calibration ran (learn more). If the calibration has an error bigger then the threshold the program will raise an error and stop. In this case after checking that no fixture issues are present te user can to perform a new calibration.
Execute the picking program
Attention
Before running the robot program for the first time, make sure that:
There exists a valid robot-camera calibration.
The Tool Center Point (TCP) has been correctly specified.
The robot speed is set to a low value, so unexpected behavior can be identified early enough to prevent the robot from colliding with people or the environment.
Pickit is in robot mode, which is enabled in the Pickit web interface.
Now you can run the program. Happy picking!