I doubt very much the 8 bit signal is the problem. We have been using 8 bits for years without any such problems.
The number of 'colours' is not the 256 levels you mention, as the colour count of any 8 bit image is in excess of 16 million colours - more than enough for most of the colours we see in the world around us.
The more likely problem is lack of accurate IR filtering for the colour issues, and poor exposure curves for the shadow problem. The in-camera compression will not have helper either, but that tends to show blocking or banding artefacts.
We have a lot of users (Gamma Curve) users doing some fantastic work with DSLR cameras, and the results are often very, very good.
Mob Boss at Light Illusion