I found out yesterday that I can delete both mail and text (including SMS) from my iPhone while leaving it usable for music, photos, GPS, and other necessities.
My plan is to still keep it turned off unless absolutely necessary, but now when I’m out on a hike, I can take a picture or listen to a playlist without seeing that new work is coming in. Freedom!