Mixing Uncommon & Rare Gems

I am trying to only use rare gems for nice pieces I may be keeping for a bit and uncommon gems for those that maybe quickly replaced. However, when I set best in bags to uncommon gems, because I already have some rare gems, it keeps suggesting rare gems. When I exclude the rare gems, it refuses to exclude them.