The spice and fruitiness of a not-too-intense port wine complements this cake well.
Heat oven to 175°C/350°F. Place some of the butter in pan and place it in oven until butter melts, approx. 5 min. Remove pan from the oven, swirl butter around the pan, and sprinkle brown sugar over it evenly. Meanwhile, pit and slice plums and chop thyme.
Arrange plum slices over brown sugar in concentric circles so that bottom of pan is covered in a single layer.
Whisk together flour, baking powder, baking soda, salt, and chopped thyme and set aside. In another large bowl, cream remaining butter with remaining sugars and lemon zest. Add eggs one at a time, beating well after each, until mixture is light and fluffy. Stir in vanilla extract.
Mix in half of flour mixture, then add half of crème fraîche and mix until just combined. Repeat until all flour and crème fraîche is incorporated. Pour batter into cake pan and spread it evenly over plums. Transfer to preheated oven and bake at 175°C/350°F until a toothpick inserted into center of the cake comes out clean and center is set, approx. 40 to 50 min.
Cool cake in pan for no more than 10 - 15 min., then invert onto a wire cooling rack to cool before serving. Enjoy!