Anyone know a nitrogen fixing tree native to southeast Virginia?

I’m working on a design around two peach trees. I have sandy clay soil type and I’m looking to improve the soil quality by adding a nitrogen fixer along with amendments like worm castings and cover cropping. I have a lot of space to work with plus I’ve wanted to add a tree to the backyard anyway. I planted the trees two years ago and have seen a few flowers early in spring but to significant growth in size. The trees are a few feet away from my garden which attracts A LOT of pollinators over the spring and summer. My goals are to nurture the trees long enough to bare fruit.

I’m open to any suggestion generally too so please give pointers.

Also this is my first Reddit post ever so be kind!