Go to settings -> apps -> the app you want the restriction gone -> the 3 dots top right
I think you can turn it off there. It is an extra security layer design by Google, because without that restriction and with the access turned on, the app can see pretty much all your touches and swipes, notifications and whatnot throughout the system etc. Edit: spelling