Variations in androgens (male sex hormones) and estrogens (female sex hormones) can affect both body asymmetry and the degree/direction of gender differentiation in the brain:

Body asymmetry: Men and women with larger right testicles or breast tend to exhibit more typically masculine behavior (aggression, spatial ability, math proficiency); those with larger left testicles or breats tend to exhibit more typically feminine behavior (nurgurance, verbal ability). (Howard, Pierce J., PhD.The Owner's Manual for the Brain, Second Edition. p 5215-5221. GA: Bard Press, 2000)