When you do Best in Bags, is your Mistweaver spec at the top priority (on the left), or below your other specs? The top priority spec gets first pick at gems, enchants, and Azerite Powers. Then if a lower-priority spec needs to share that item, it has to use it with the main-specs gems, enchants, and AZ.
I suspect that MW isn’t your top spec, but if you put it in the top spot, it would pick the INT gems.