Ciao Henry,
Yes, the closest outdoor thermal baths are in southern Tuscany so they are not that close. You can head to Bagni di Petriolo, to Bagno Vignoni, San Filippo or the ever popular Saturnia. I think the bagni di Petriolo is the closest, at 1.30 h away.
There are also spots of "natural" hot springs here and there, for example, in northwestern Tuscany at Bagni di Lucca and Equi Terme, along the rivers and away from the towns, some definitely require having help from locals in finding them. But these are the more "popular" and well-known, so that is where it easier to head to.
Keep in mind there are many other thermal areas that are closer, but they are enclosed and no longer open in nature so no longer free. Practically all the towns that have "Terme" in their names are born from having springs there. In case you would prefer to enjoy the warm spring waters closer to Florence, there are very nice spas in Montecatini Terme, San Giuliano Terme, Rapolano Terme and Serre di Rapolano.