How should I list products that come in 3 or 4 different colors?

Well, it's really up to you. You need to decide if it makes sense for you to post those products as a single product listing or as 3 or 4 different product listings. 

To set your items as a single product listing, simply adjust the "Quantity" value on the product details page.

The important thing is that you explain to the buyers which colors they will get when they click on the “Buy” button, if you go with the first option.

Also, make sure you mention in the product description field that your buyer has the option of choosing colors and has to tell you which color he prefers during the checkout process. There is a step in the process (second screen) where user can submit their comments to you.



Our recommendation, however, is to setup 3 or 4 separate product listings.
This way you'd eliminate user confusion with colors, as they'd see the same color in all product photos.

it will also allow you to optimize your product page for the color as well, by incorporating that keyword inproduct title and description.

The problem is that the person who is searching on Google for "Blue Hat" might not find your listing on Google, if your title says "Hats in different colors". Of even if he will and land on your product page, he'll see your product photo in a wrong color. That may confuse them and they will leave without reading the line about other color options.

Here are two examples of good product titles optimized for product color, in this instance for "Blue Hat" and "Blue Hat and Scarf".
 

For additional reading about product copy optimization, please see Wrong Keywords attract wrong traffic or don’t attract any, Choosing Right Keywords is not a Guessing Game and How to write good Product Titles?