Yes, you are correct...it is a custom that wasn't always accepted by all observant Jews.  In the early part of the 20th Century, Orthodox German Jews did not all always keep their heads covered.  Most did, but not all.
 
However, that being said, when a custom becomes as widely accepted as covering one's head at all times that it becomes a "tradition", it also usually is accepted as a "law".  I fluctuate between wearing a kipa and not, depending of where I am going.  If the community is less aware of Jewish traditions, I might wear a cap or a funky hat or, even, nothing at all....what is paramount is always being aware that there is something above us at all times.
 
B'Shalom,
R. Dan