He rises in his stance, since they are apparently stopping to talk. He's angry that she could possibly be implying what she's implying - not about his identity, but her own. At being told he's not himself he just feels empty. He doesn't feel like himself, anyway. It's old news.

"And who are you to decide that?" he asks, and it's an unsubtle test. He wants to draw her out, because he can't quite bring himself to ask.