WASCE server can be configured as a Windows service, in this case, the jvm options must be set in geronimosrvw.exe. JAVA_OPTS that is set in other places will be ignored. Note: These jvm options can be set in Arguments list from Startup/Shutdown tab, and they have to be added to the top of the list, it won't take effect if they were placed at the bottom.
Resolving the problem
The required fields are:
Image: Path to java.exe
Working Path: WASCE installation directory
Arguments: each option should be placed in a separate line, and those JVM options should be place on the top of the list.