Some of your problem could be your stirrups too long, your reins too short. This can make you sit forward. Another problem could be tension through your shoulders. Try having a lunge lesson with out stirrups, and on loose reins, will help with your core strength and teach you to balance through your buttocks. Try singing to yourself as your going around will help you relax in the saddle. Sometimes we over think which can cause tension through our shoulders and spine.
Something I was told when I was younger was to try to sit on my back pockets of my pants. It helped as a mental and physical reminder instead of just hearing the generic "sit back". And heels are importantly too there is a good chance that if you are to far forward your heels aren't down enough.
Sitting up straight has a lot to do with core strength. I have the same problem sometimes. Heels down, shoulders back, chest out is what my trainer tells me, but I've noticed a huge difference in my riding since I've started doing ab workouts