You will need to decide which products will be assigned to a Shipping Group.

There is the option to use the * group which does not have to be assigned to individual products, and can signify either all products or all products not in a shipping group depending on your requirements.

Once your groups are configured in the Manage Attributes section you can assign to a product via the shipping tab in the product listing. If you prefer you can bulk apply groups either with a product import, or using the grid product view.

Note that in Shipping Override it is known as Special Shipping Group, and in Handling Matrix it is referred to as Handling Group.