How to use Pickit Bags

This article guides you through the necessary steps to properly use the Pickit Bags engine.

Note

By default the Pickit Bags engine is not exposed in the web interface. To enable it, open the Settings, navigate to Advanced settings > Application-specific engines and turn on the Enable Bags engine toggle.

Teach the Region of Interest

When teaching the Region of Interest, make sure to attach its origin to the Robot base frame, if the camera is mounted on the robot flange. Place the markers on the floor, but not on the pallet. The XY-plane of the ROI frame should be the floor plane.

Empty wooden pallet on the floor with ROI markers placed on the floor around it, and the ROI box and reference frame with its XY-plane on the floor

Fine-tune the Region of Interest such that it includes the top of the pallet, but not the floor. Make sure that the Region of Interest is large enough to include the whole pallet with bags.

Pallet configuration

Bag pattern

In the Detection page, section Bag pattern, specify the pattern in which the bags are disposed on the pallet.

The Pickit Bags engine supports five patterns: 3-bag, 4-bag (crossing and parallel), 5-bag and 8-bag patterns.

Diagram of the five supported bag patterns seen from above: 3 bags, 4 bags crossing, 4 bags parallel, 5 bags and 8 bags

Note

By default, the Pickit Bags engine assumes that the layers are full (no missing bags from the pattern). There is an option to detect whether the top layer is full or incomplete, but it should only be used for specific cases.

Real-life examples of pallets for each supported bag pattern are shown below:

3-bag pattern

The 3-bag pattern consists of two horizontal bags and one vertical bag.

Top view of a pallet layer in the 3-bag pattern: one vertical bag on the left and two horizontal bags on the right

4-bag pattern (crossing)

In the 4-bag (crossing) pattern, there are two horizontal and two vertical bags, forming a square. There may be significant overlap between the horizontal and vertical bags.

Two pallet layers in the 4-bag crossing pattern, two horizontal and two vertical bags forming a square: left, bags do not overlap; right, bags overlap significantly

4-bag pattern (parallel)

In the 4-bag (parallel) pattern, the bags are disposed in parallel, in two groups of two.

Top view of a pallet layer in the 4-bag parallel pattern, with four blue bags in two groups of two

5-bag pattern

The 5-bag pattern includes three vertical bags and two horizontal bags.

Top view of a pallet layer in the 5-bag pattern: three vertical bags at the top and two horizontal bags at the bottom

8-bag pattern

Finally, the 8-bag pattern consists of six horizontal bags and two vertical bags.

Top view of a pallet layer in the 8-bag pattern: two vertical bags on the left and six horizontal bags in two columns

Horizontal overlapping bags

In the 5-bag pattern, it is common that the horizontal bags overlap (left bag above right bag, or vice-versa). If you want Pickit to detect which bag is overlapping, enable the checkbox Automatically compute which of the horizontal bags is overlapping. Otherwise (if there is no or little overlap, or if the overlap is always consistent for all layers), keep this checkbox disabled, and instead specify a fixed bag picking order.

5-bag pattern layer where the two horizontal bags, outlined in yellow and numbered 4 and 5, overlap each other

Warning

The overlapping check can only be applied to the two horizontal bags of the 5-bag pattern. For other overlapping bags (within the 5-bag pattern or other bag patterns), the picking order must be set such that overlapping bags are picked first. The 4-bag (crossing) pattern is the only exception: in case bags are overlapping, Pickit automatically detects an appropriate picking order.

Pallet dimensions

Specify the length and width of the pallet. Except for the 4-bag (crossing) pattern, the detection of the layer orientation assumes a rectangular pallet shape (that is, one side is significantly longer than the other).

Top view of a 5-bag pallet layer with the pallet LENGTH and WIDTH dimensions marked around it

Minimum pallet height

Specify the minimum height of the last (bottom) layer of bags, relatively to the floor. We recommend measuring the height of the wooden pallet and adding around 40 mm. This value allows Pickit to know when the pallet is empty.

Note

The minimum pallet height is expressed relatively to the ROI frame. It is therefore important that the ROI frame XY-plane is located at the floor.

Empty wooden pallet with the minimum pallet height marked from the floor to the top of the pallet

Full and/or incomplete layers

Select the status of the top layer to be full (all bags in the pattern are present) or incomplete (one or more bags are missing from the pattern). Select Detect automatically for Pickit to detect whether the top layer of the pallet is full or incomplete.

Four 4-bag crossing pallet layers with detected bags outlined in yellow: incomplete layers with 3, 2 and 1 bags in the top layer, and a full layer with all 4 bags

Warning

For a correct full/incomplete layer detection, it is important that the XY-plane of the Region of Interest is parallel to the floor, and not tilted.

Warning

For this feature to work well, the 3D shape of the top layer should be flat and regular. For instance, ideally the bags are shaped like a brick and lying flat (not too tilted relatively to the floor). Also, only points on the top of the pallet should be visible to the Pickit camera, and not on the sides.

If that is not the case for your application, it is recommended to always enforce full layers.

Optimize detections

In this section, we recommend to use no image fusion and no downsampling.

Picking the bags

Bag pick point offset

Depending on the type of bags, it might not be optimal to pick the bags from the center, as the heaviest part is sometimes located towards the bottom of the bag. For this reason, you can provide an offset to the pick pose, along the X-axis (red) of the bag. You can find this option in the Picking page, section Bag pick point offset.

5-bag layer with five detected, numbered bags: left, pick frames at the bag centers with no pick frame offset; right, pick frames shifted along the bag X-axis with a negative offset

Bag picking order

In section Bag picking order, you can specify the order at which you want the bags to be picked, for each possible layer orientation. If bags are overlapping on the pallet, the picking order is a crucial setting for a successful pick. Typically, the best is to pick the bags in the reverse order as they were palletized.

The image below shows two different 3-bag pallets. On the left, the vertical bag is overlapping the horizontal bags, and therefore it is preferred to pick the vertical bag first. On the right, we want to first pick the two horizontal bags, as they overlap the vertical bag.

Two 3-bag pallets with numbered picking order: left, the overlapping vertical bag is picked first; right, the two overlapping horizontal bags are picked before the vertical bag

Note

Suppose that you are detecting a 5-bag pattern, and have the checkbox Automatically compute which of the horizontal bags is overlapping enabled. Pickit will respect the picking order selected in this section, except for the order among the two horizontal bags, depending on which of them is detected to be on top.

Note

If you are detecting a 4-bag (crossing) pattern, Pickit automatically detects the best bag picking order for you. Thus, the Bag picking order section is not shown in the Picking page for that pattern.

Robot programming with Pickit Bags

Typically, the layers of the pallet are organized such that the orientation of the bag pattern alternates from layer to layer, flipping horizontally or vertically (depending on the pattern) from the previous layer. Detecting the correct orientation of the bag pattern (also referred as layer orientation) is the first step to correctly detect the bags.

The image below shows the possible layer orientations for each bag pattern.

Table of possible layer orientations per bag pattern: 3, 5 and 8 bag have orientations 0 and 1, 4 bag crossing has orientations 0 to 3 with numbered bags, and 4 bag parallel has only orientation 0

  • For 3, 5 and 8 bags, consecutive layers usually switch between orientations 0 and 1.

  • For the 4-bag pattern (crossing) there are four possible orientations. Consecutive layers usually switch between orientations 0 and 1 or between 2 and 3. Notice that the position of the individual bags is the same for orientations 0 and 2 and for 1 and 3, the difference being only the picking order. The picking order is relevant if neighboring bags are overlapping.

  • For the 4-bag pattern (parallel) there is only one possible orientation.

The detected bags are sent to the robot or PLC one by one: the first bag is sent upon triggering a detection, and the remaining bags are sent one at a time, upon requesting the next detected object. The robot program can, however, have access to the actual layer orientation, too. The socket interface variable object_type, which gets filled in after receiving a detection response from Pickit, contains information on the bag pattern and the detected layer orientation, according to the following table:

Bag pattern

Layer orientation

object_type

3-bag

0

0

1

1

4-bag (crossing)

0

2

1

3

2

4

3

5

4-bag (parallel)

0

6

5-bag

0

7

1

8

8-bag

0

9

1

10