I got this idea from a post form @Kevinjohnston
I have always been jokester, especial at work. I enjoy lightening the mood. I'm a person who observes people and knows what irritates them and would take advantage of this. Here's an example..
I had a work friend and who was disciplined. You could also call it OCD. He would follow a set schedule and not veer from it. He would go to bed at a certain time, get up and work out and such. he did this every day. I used to mess with him every once and a while. When he was away from his desk I would move everything around on it. Then watch his reaction. He did have a good sense of humor and knew who did it. We would laugh about it later.
If there was no malice involved and you both laughed about it -its okay.
I am somewhat easily distracted, so I have routines to help me accomplish what I need to during the day. If you were to move stuff around on my desk I'd probably have to hurt you. Stuff like this gets addressed at Askamanager.com A site I highly recommend.