Everyone certainly has his own thought.
If he does not speak up, it does not necessary imply that he is a quiet person or he is trying to hide something.
One common reason is that he is too tired to talk, while in some strange occasions, he does not want to join in the battle of debating voices to worsen the situation; sometimes he is being interrupted before he can finish his sentence.
Unfortunately, to speak up is difficult. The worst scenario is when he is being prompted but he cannot finish his sentence before another voice tries to battle his.
There is nothing wrong with being treated as a quiet person and there is no wrong for remaining silence, as long as nobody mistakes him as a backstabber.
It becomes a joke when some people ask you to speak up yet you totally have no chance to speak up.