It's a tough pill to swallow, but I'd lay money that you ARE moving the seat and NOT knowing it.
It is very very very subtle, and if you don't realize that you're doing it, it's WAY to easy to hit the buttons on the seat as you're getting out, and not even realize that you're doing it.
Mine does the same thing, and so do a LOT of peoples. I, like all of you, thought that there was something wrong with it. Then, as I was getting out one day, I heard the motor on the seat back run, for a very brief moment. It's hard to hear, if you're not paying very close attention to it. I immediately put the seat back to memory one, and sure enough, it had moved, ever so slightly.
Since that day, I've paid very close attention to this, and I can often hear the seat moving while I'm getting out, and the next time that I get in, the seat will have been moved to a new location.
To illustrate that you don't even realize that it's happening, I have this incident: The wife and I were parked at the shopping mall parking lot. She received a text just as we were parking, and she opened up the door, and slid her leg outside to put it on the ground as she was finishing her reply. When we got back to the Treg to leave, her seat back was COMPLETELY reclined. Neither her nor I ever heard or saw the seat back moving, but it was definitely caused by her leg hitting the switch. The exact same thing played out several weeks later when my wife took someone new with her. This person had short legs, and had a hard time getting in and out of the vehicle. There was one time that she was getting out, and as she slid her legs out, the seat started reclining. In fact it was reclining to the point that it was smashing my daughter in the rear seat directly behind her. It happened to this person THREE times that day, so it is definitely not random.
I hate to be the bearer of bad news, but the reality is that you are most certainly hitting the buttons as you get out, and it is very subtle.
PS: Yes, it does suck...