I simulated a few mode fiber in COMSOL and use maximum mesh size of 0.15 um to all domains to ensure accuracy of mode profile. I exported the electric field of the modes in x and y directions with grid step of 0.1 um and calculate the dot product between two mode pair to check the orthogonality (Ex1.conj(Ex2))+(Ey1.conj(Ey2)) but the resulted modes are not orthogonal enough as you can see in the attached fig1, there is a cross-coupling of around -40db below the maximum value. Is this normal or what I have to adjust to optimize the accuracy of the solution? and in general how to improve COMSOL accuracy when fiber has a large number of modes (e.g. 50 modes)?
I have also tried to use adaptive refinement but the dot product didn't improve as in fig2.