I was in a band that had a B3. It sounded wonderful, but you had to have the separate Leslie speaker cab to go with it because of the sound produced through that rotating speaker. It always took at least 4 people to get either piece up stairs and it was a pain. Was it worth it? I'm not sure. It really wore us out and the band was together for a few years.
Just recently, I played a gig where there was a newer Leslie speaker cab which seemed more compact than the old furniture piece I was familiar with. And, the guy playing it was also playing a more compact and portable Hammond. The combo sounded great in my book and the difference between the old and new was not discernible to me. I mean, the names Hammond and Leslie belong with each other, but I don't think you absolutely need a B3 to get a great sound. I thought to myself, "boy, we sure could have used this setup back then." It sounds however, like the one you have is not anywhere near a B3 and that's okay - but it would help to have the separate cab, especially with the speaker problems you mentioned. And, it looks heavy, so even if you get the separate cab, I don't look forward to carting something like that to gigs and practices. On the other hand, young people might be okay with that...