My 5 & 6 yr olds both swim very well, but they've been in lessons for the last 4 years. We don't do lifejackets in the pool - it makes it harder for them to swim - but they never swim unattended, and we have a poolfence.
The younger they learn, the better, so you definitely want to start as soon as you can. I think this is the year for my youngest. She has enough independence to really start swimming this year. I'm keeping my fingers crossed. :)