Would this be offensive to anyone?

I understand what you mean, it’s just it might hurt people who have been told they can’t wear this because they are a man. It is like how girls have been told they can’t play sports, we have told them that we can so it would be hypocritical to say they can’twear dresses or skirts. It’s ok if your character doesn’t like to dress in a feminine way but maybe try re wording it somehow if you really want to add this line and get someone to check it over first.