That means that the pillar pod switch has is wired wrong, and that the illumination wire (for the switch lighting) is hooked up to a constant power (i.e. directly to the battery) as oppose to a switched power.
If you want to fix it, you're going to need to be good with wiring and reading wiring diagrams. You'll then need to tap that wire into a switched power. I'd probably tap it into a wire that gets power when the dash lights in the vehicle are switched on.