While Onsen refers to hot springs that are naturally available near the volcanic areas of Japan, a Sento is usually a bath that has been man-made.
These article and article explain pretty well the differences. In general they have the same behavioral rules, except that sentos are much cheaper and affordable, being used by everybody (Japanese have in general very small houses and just few of them have normal-size bath tubs). We tried both and we loved them for the same reason!
When it comes to hot springs there is a huge range of options, form the fanciest and most luxurious ones, to the simplest and most affordable. We visited two very different ones, the first at the hotel we stayed in Unzen (Kyushu is very famous for hot springs, together with Kumamoot and Beppu) and the other one in Hakone, near Mount Fuji (picture above), called Yuryo. It was amazing and I would recommend the day trip from Tokyo. This is the instruction flyer that we got there before entering the spa, which gives you a very good idea on the experience. The pricing can vary a lot depending on the place, but you will be surprised to see how much more affordable are spas all over the world compared to the US, Japan included! Another very famous area for onsens is near Nikko, and Kusatsu, near Nagano.
If you are only in Tokyo for few days, you can instead visit the Oedo-onsen Monogatari in Odaiba, Japan's largest hots spring resort.
A public bath-house is called a sento in Japanese. Soaking in the soothing hot waters of a large bathtub warms the body through and through, and many people go to the sento regularly even though they have bathtubs at home. They are also spots where local residents chat with their neighbors, and when you visit one you can get a glimpse of the real city that you can’t see on the regular tourist circuit.
We visited this one in Nippori with our dear friend Tamako and it was si much fun!
This is the guide that Kyoto prefecture published for tourists. It's very explicative and useful.