This error is a bug with Java applications when desktop effects are enabled. For more information about this bug, refer to Sun's bug reports here:
Generally to resolve this issue you will need to disable desktop effects for MATLAB to function normally. To do so:
Click the System menu, go to Preferences to Look & Feel and select Desktop Effects. Set Desktop effects to None.
Click the System menu, go to Preferences, and select Appearance. Go to the Visual Effects tab. Set Desktop effects to None.
It may also be possible to display the MATLAB desktop with these effects enabled. For information on how to enable the MATLAB desktop with desktop effects enabled, please see below.
NOTE: The methods below are not officially supported as other side effects may be caused by changing these settings.
To enable the MATLAB desktop with desktop effects enabled, you can set the following environment variable:
Alternatively you may try installing Java JRE 1.6 on the machine and point MATLAB to that JRE. See the related solution: "How do I change the Java Virtual Machine (JVM) that MATLAB is using? " for more information about using MATLAB with a different Java.