The 6.4" display has two CCFL tubes versus the 10.4" which only has one tube. The inverter on the 6.4" kit takes more power to drive the two tubes.
Look at the data sheets for the two inverters, you'll see the difference in power:
6.4" -> ERG K1957
10.4" -> ERG 8M051972
(
http://www.ergpower.com)
Both of these inverters have the ability to be driven at lower brightness, but the kits have no connection to those pins. You can use a PWM to drive the 'enable' pins on those inverters, but this would be a custom wire from the SOM kit to the Display kit. (Left open, they are full brightness)