ABB example bin picking program

This example program requires that Pickit is installed and set up with your robot. For installation instructions, please refer to the ABB installation and setup article.

Example program

By default no main() module is loaded. One of the Pickit example programs can be loaded from HOME > ABB_Pick-it > Pick-it > Application examples.

Note

These example programs only works with Pickit software version of 2.2 or greater. If you are using a software version prior 2.2, please contact us at support@pickit3d.com, and we will assist you in finding a solution.

Below the main() part of the example program Pickit_example_bin_picking is shown. This program is specific to bin picking applications, for a simpler example program see ABB example simple pick and place. More information about the logic behind this example program can be found in the robot-independent complete picking example.

PROC main()
    VAR bool pickSucceeded:=FALSE;
    VAR bool pickingCompleted:=FALSE;
    VAR bool isReachable:=TRUE;
    VAR num numNoObjects:= 0;
    VAR num numPickFail:= 0;

    check_robot_mode;
    before_start;
    pickit_configure setup, product; ! Load desired setup and product configuration.

    ! First detection.
    goto_detection;
    detect_objects;
    WaitUntil pickit_get_results();

    ! Main loop.
    WHILE NOT pickingCompleted DO
        IF pickit_object_found() THEN

            numNoObjects:= 0; ! Reset counter.

            ! Calculate approach and retreat.
            PickitPick:=pickit_get_pose();
            PrePick:=RelTool(PickitPick,0,0,prePickOffset);
            PostPick:=RelTool(PickitPick,0,0,postPickOffset);
            BinEntry:=PrePick;
            BinEntry.trans.z:=binEntryZ;
            BinExit:=PostPick;
            BinExit.trans.z:=binExitZ;

            isReachable:=pickit_is_pose_reachable(BinEntry,toolPickit) AND
                         pickit_is_pose_reachable(PrePick,toolPickit) AND
                         pickit_is_pose_reachable(PickitPick,toolPickit) AND
                         pickit_is_pose_reachable(PostPick,toolPickit) AND
                         pickit_is_pose_reachable(BinExit,toolPickit);

            IF isReachable THEN
                ! Object is pickable!
                pick;
                pickSucceeded:=is_object_picked();
                goto_detection;
                detect_objects;
                IF pickSucceeded THEN
                    numPickFail := 0;
                    place;
                ELSE
                    ! Picking failed, skip place.
                    ErrWrite\I,"Mispick.","Pick failed, skipping place motion.";
                    INCR numPickFail; ! Increment counter.
                    on_pick_failure;
                ENDIF
                WaitUntil pickit_get_results();
            ELSE
                ErrWrite\I,"Object is unreachable, going for next object.","";
                pickit_next_object;
                WaitUntil pickit_get_results();
            ENDIF
        ELSE
            ErrWrite\I,"There are no pickable object.","Retrying detecting.";
            INCR numNoObjects;
            goto_detection;
            detect_objects;
            WaitUntil pickit_get_results();
        ENDIF

        ! Exit loop if picking is complete.
        IF Pickit_roi_empty() THEN
            ErrWrite\I,"The ROI is empty.",
                "Picking complete.";
            pickingCompleted:=TRUE;
        ELSEIF Pickit_no_image_captured() THEN
            ErrWrite "Failed to capture a camera image.",
                "Picking complete.";
            pickingCompleted:=TRUE;
        ELSEIF NOT isReachable THEN
            ErrWrite\I,"All detected objects are unreachable.",
                "Picking complete.";
            pickingCompleted:=TRUE;
        ELSEIF numNoObjects >= maxNoObjects THEN
            ErrWrite\I,"Too many consecutive detections with no object found.",
                "Picking complete.";
            pickingCompleted:=TRUE;
        ELSEIF numPickFail >= maxPickFail THEN
            ErrWrite\I,"Too many consecutive failed picks.",
                "Picking complete.";
            pickingCompleted:=TRUE;
        ENDIF

    ENDWHILE

    after_end;

ERROR
    IF ERRNO=ERR_COLL_STOP THEN
        StartMove; ! Allow to start moving again.
        TRYNEXT; ! Exit Pick routine and go to next instruction.
    ENDIF
ENDPROC

Required program inputs

Before main() the user is required to enter values for all the constants which are going to be used inside the program.

! Detection settings.
CONST num setup:=1;
CONST num product:=2;
CONST bool isCameraRobotMounted:=TRUE;

! Pick sequence settings.
PERS tooldata toolPickit:=[TRUE,[[0,0,0],[1,0,0,0]],[0.001,[0,0,0.001],[1,0,0,0],0,0,0]];
CONST num prePickOffset:=-100;
CONST num postPickOffset:=-100;
CONST num binEntryZ:=500;
CONST num binExitZ:=500;
CONST speeddata extractionSpeed:=v500;
CONST speeddata approachSpeed:=v200;
CONST speeddata fastSpeed:=v800;

! Completion condition settings.
CONST num maxNoObjects:= 4; ! Maximum allowed consecutive detections with no object found.
CONST num maxPickFail:= 4; ! Maximum allowed consecutive failed picks.


! Fixed poses.
CONST robtarget Detect:=[[0,0,0],[1,0,0,0],[0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
CONST robtarget Dropoff:=[[0,0,0],[1,0,0,0],[0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
CONST jointtarget AbovePickArea:=[[0,0,0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];
CONST jointtarget HomePose:=[[0,0,0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];

  1. Detection settings

    • setup: the desired Pickit setup id.

    • product: the desired Picket product id.

    • isCameraRobotMounted: set as TRUE if the camera is mounted in the robot arm, while FALSE if it is in a fixed position.

  2. Pick sequence settings

    • toolPickit: this will be the tool used in all the motion commands, the user need to set the actual TCP value.

    • prePickOffset: the z-offset for PrePick relative to the tool frame.

    • postPickOffset: the z-offset for PostPick relative to the tool frame.

    • binEntryZ: the z absolute value for BinEntry relative to the robot base.

    • binExitZ: the z absolute value for BinExit relative to the robot base.

    ../../_images/pick-sequence-5-point-offset.png

    • extractionSpeed, approachSpeed and fastSpeed: user defined speed data used in the robot motion.

  3. Completion condition settings

    • maxNoObjects: maximum allowed consecutive detections with no object found.

    • maxPickFail: maximum allowed consecutive failed picks.

  4. Define fixed points

    • Detect: where to perform object detection from. For more details see Robot position during image capture.

    • Dropoff: where to place the picked part.

    • AbovePickArea: A point roughly above the pick area from which the above two can be reached without collision.

    • HomePose: the start pose of the application.

    ../../_images/waypoints.png

Note

At least one of the fixed poses should be saved in joint space (jointtarget). This is because in every cycle we want to ensure that the robot goes back to the same joint positions and does not make a complete turn in some joint angles (for example J6 from 0° to 360°)

Program procedures

Below are shown all the procedure that will be called by the main() function and are related to Pickit functionality and the gripper status.

PROC before_start()
    MoveAbsJ HomePose,v500,z100,toolPickit;
    gripper_release;
    ! <<< Add Initialization logic here.
ENDPROC

PROC after_end()
    Pickit_save_snapshot;
    Break;
ENDPROC

PROC goto_detection()
    MoveJ Detect,fastSpeed,z0,toolPickit;
    IF isCameraRobotMounted THEN
        ! Wait for no vibrations in the camera.
        ! Adapt the duration depending on your application.
        WaitTime\InPos,0.2;
    ENDIF
ENDPROC

PROC detect_objects()
    IF isCameraRobotMounted THEN
        pickit_capture_image;
        pickit_process_image;
    ELSE
        ! Fixed camera.
        pickit_look_for_object;
    ENDIF
ENDPROC

PROC pick()
    !Moving to an object.
    MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
    MoveL BinEntry,fastSpeed,z100,toolPickit;
    MoveL PrePick,extractionSpeed,z20,toolPickit;
    MoveL PickitPick,approachSpeed,fine,toolPickit;
    gripper_grasp;
    MoveL PostPick,extractionSpeed,z20,toolPickit;
    MoveL BinExit,extractionSpeed,z100,toolPickit;
    MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
ERROR
    IF ERRNO=ERR_COLL_STOP THEN
        ErrWrite\W,"Collision detected, trying to recover.","";
        gripper_release;
        StopMove;
        ClearPath;
        StorePath;
        MotionSup\Off;
        MoveL BinExit,extractionSpeed,z100,toolPickit;
        MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
        RestoPath;
        MotionSup\On;
        RAISE ;
    ENDIF
ENDPROC

PROC place()
    MoveJ Dropoff,fastSpeed,fine,toolPickit;
    gripper_release;
    WaitTime\InPos,0.5;
ENDPROC

PROC on_pick_failure()
    gripper_release;
ENDPROC

In pick() the user needs to add the application-specific grasping logic. In the same way the releasing logic is needed in place(), before_start(), on_pick_failure() and in the collision handling inside pick().

Advanced topics

Collision recovery

To avoid production downtime, it is recommended to use a collision recovery strategy in the robot program. This way, if a collision is detected by the robot while trying to pick a part, the robot does not raise an error requiring operator intervention. Instead, the robot goes back to the detection point and tries to pick another part.

In ABB, this is done using the motion supervision error handling module. To use this functionality, you must first set the system parameter CollisionErrorHandling to YES in Control Panel > Configuration => Topic > Controller > General Rapid.

See below for an example pick procedure with collision recovery.

PROC pick()
    !Moving to an object.
    MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
    MoveL BinEntry,fastSpeed,z100,toolPickit;
    MoveL PrePick,extractionSpeed,z20,toolPickit;
    MoveL PickitPick,approachSpeed,fine,toolPickit;
    gripper_grasp;
    MoveL PostPick,extractionSpeed,z20,toolPickit;
    MoveL BinExit,extractionSpeed,z100,toolPickit;
    MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
ERROR
    IF ERRNO=ERR_COLL_STOP THEN
        ErrWrite\W,"Collision detected, trying to recover.","";
        gripper_release;
        StopMove;
        ClearPath;
        StorePath;
        MotionSup\Off;
        MoveL BinExit,extractionSpeed,z100,toolPickit;
        MoveAbsJ AbovePickArea,fastSpeed,z100,toolPickit;
        RestoPath;
        MotionSup\On;
        RAISE ;
    ENDIF
ENDPROC

Calibration validation

In applications with robot-mounted cameras, it is a good practice to perform calibration validation before running the program. An incorrect or outdated calibration can lead to inaccurate picking.

An automated calibration validation step can be added to the robot program in the before_start() procedure, before any detection is performed:

! Add the position where the camera on robot can see the fixed calibration plate.
CONST robtarget LookAtTheMarkers:=[[0,0,0],[1,0,0,0],[0,0,0,0],[9e9,9e9,9e9,9e9,9e9,9e9]];

VAR num distance_threshold := 10;   ! mm         use 4 in case of a HD camera
VAR num angle_threshold := 2.0;     ! degrees    use 1 in case of a HD camera

PROC before_start()
    MoveAbsJ HomePose,v500,z100,toolPickit;
    gripper_release;

    IF isCameraRobotMounted THEN

        MoveJ LookAtTheMarkers, v100, fine, PICKIT_TOOL;
        WaitTime 1.0;

        IF (NOT pickit_validate_calibration(distance_threshold, angle_threshold)) THEN
            ErrWrite \I, "Pickit: ValidateCalibration", "High calibration validation error, stopping";
            UIMsgBox "High calibration validation error, stopping";
            stop;
        ENDIF
    ENDIF
ENDPROC

The calibration plate needs to be fixed to a wall or to the ground, such that its location with respect to the robot base is exactly the same as when calibration ran (learn more). If the calibration has an error bigger then the threshold the program will raise an error and stop. In this case after checking that no fixture issues are present te user can to perform a new calibration.

Execute the picking program

Attention

Before running the robot program for the first time, make sure that:

  • There exists a valid robot-camera calibration.

  • The Tool Center Point (TCP) has been correctly specified.

  • The robot speed is set to a low value, so unexpected behavior can be identified early enough to prevent the robot from colliding with people or the environment.

  • Pickit is in robot mode, which is enabled in the Pickit web interface.

Now you can run the program. Happy picking!