I get wound up when people press the wrong call button, i.e. up when they want to go down. It interrupts the existing journey & means they have to wait longer, but so does everyone else...
When I was in school one building had a bank of three elevators. It was 7 floor plus a basement. For some reason only one car would ever come to the ground floor. The same car, over and over. The other two always stayed around the 4th or 5th floor and never moved. This was very annoying in the morning when everyone was trying to go to class.
One day one of the Facilities workers came by and showed us a trick. Press the down call button and it would force the other two cars to eventually go down to the ground floor after they finished any stops going up. They finally fixed it and put some time based controls in it to force cars to the ground floor during peak morning rush times.