Why is it that men’s eyes always drift toward females?

I mean, “men” is the plural form of “man”. So it’s “mens'”… but it looks very strange, and maybe this only applies if the word doesn’t change in plural form?

For example, if it had been “one man” and “several mans”, it would’ve been “several mans’ eyes”, but now that it’s “men”, “several men’s eyes” is how you type it?

Is that the rule? Have I understood this correctly?