You'd think, but the Blackphone's Security Center manages to do that very easily, with great UX. I use XPrivacy on my HTC One and the UX is a mess of mystery buttons and weird, technical options. Blackphone has a on/off switches for things like "Camera & Audio", "Record audio", "Read contacts", etc.

There's also a tab for non-privacy-related permissions, e.g. vibrate, notification, keep awake, etc, which are on by default, to make sure the app isn't unnecessarily limited by default. It's miles better than the default Android permissions system, and one of the features I like best about the phone.

Android could very easily implement something like that, with a dialog asking if the user wants to grant the permission, with perfect backwards compatibility (since there are devices/software that does it now), and I hope they do.

With a hierarchical UI, the permissions can be controlled easily as finely or coarsely as desired. Something like this:

     - [*] Phone
     |  |--[ ] Make outgoing calls
     |  |--[ ] Answer incoming calls
     |  \--[X] Read call log
     |
     + [X] Access Location Services
     |
     - [ ] Network
     |  |--[ ] Outgoing connections
     |  \--[ ] Incoming connections (listen)
      ...