Definitely I would spend all my money on getting peonies
trees. I would get one of every color. They are expensive. But they are beautiful and long lived. Well worth the expense if I could just come up with the dollars.
If that isn't of interest here are some other choices.
Actaea pachypoda - Dolls Eyes: All parts of this
plant are toxic so not good if you have animals or children.
Alchemilla erythropoda - Dwarf Lady's Mantle
Anemone nemorosa 'Flore Pleno' - Anemone, Wood: as a flowering ground cover I think this is really nice
Arisaema triphyllum - Jack-in-the-Pulpit
False Spirea - there are a number of colors and heights of these and they are great looking. I love anything that looks like a feather.
Pigsqueak - I know the name is off putting but if you like pink I would definitely look into some of these.
Clintonia umbellulata - Speckled Wood
lily: The blooms are white and it would fit in well with what you already have but it wouldn't provide a contrast so maybe not.
Lilies of the valley: Pink or white
Chinese Winter Hazel: As a small
tree this is great.
Hope something here is useful for you.