Tap water

In Medellín and the rest of the Aburrá Valley: yes. EPM (the local utility) treats and tests the water aggressively, and locals drink it straight from the tap. It's some of the best tap water in Latin America.

Outside the valley: probably not. Smaller towns, fincas, and the coast (Cartagena, Santa Marta) often have water that's technically potable but not friendly to a foreign stomach. Stick to bottled or filtered there.

Ice in restaurants in Medellín is fine. Same with brushing teeth, washing produce, and making coffee.