Why does the orange float, but when the peel is removed, sinks?
Because the inside of the orange is denser than water! A good example is like a rock wrapped in a life preserver. Take away the life preserver and the rock sinks. It's just like the orange. Take away the peel and the orange sinks.
The inside of the orange is denser than water as although it is mostly made up of water (the tasty juice =D) but there's dissolved sugar and other flavourful compounds which are heavier than water molecules that they displace. The result is the orange juice weighs more than the same volume of water.
An experiment to try:
Try floating diet soda and soda containing sugar in a bowl of water! Only one of them will float. :)