Of course!!!
When wearing a long sleeved shirt, a tie or cravat is "de rigeur". Didn't your mother teach you anything about etiquette?
Of course, if the sleeves are rolled up, then it is permissible to remove one's tie.
If one is not wearing trousers, then the tie may be loosened.

Further etiquette states that when wearing shorts, then one requires long socks. Wearing short socks with shorts indicates an utter lack of breeding. Incidentally, it is permissible not to wear socks with soft shoes, such as gym shoes, but mandatory with hard shoes. Sandals are NEVER to be worn with socks!!
One must never let down one's side, you know.

It's been eaten by moths!