Hi Abhibrata,
When you set a 3-phase voltage source to 600V Phase-to-Phase (Vrm Ph-Ph) and measure the Phase-to-Ground voltage, the expected peak value that you observe is due to the relationship between 'phase-to-phase' and 'phase-to-ground' voltages, as well as between RMS and peak values in AC systems.
The Phase-to-Phase to Phase-to-Ground Relationship is defined as :
This 346.4V is the RMS value of the phase-to-ground voltage.
The RMS to Peak Value Relationship is defined as:
, or,
Therefore, the peak voltage reading of approximately 489.9V for the phase-to-ground voltage is accurate and expected, based on your 600V phase-to-phase setting.
Hope this helps!