I love AMR, but I also love Raidbots; my dilemma is that when I sim my character (Arathor-US / Worvi) each platform gives different results (most noticeable in gear rating and recommendations due to stat weight differences).
Has anyone else noticed this and figured out why the two are different? Furthermore, which one is right/best?
I am very familiar with the differences between the two. If you want to ask about a specific example that would be best - since every spec in SimC is developed independently.
One of the most common reasons people see differences is because it is non-trivial to set the two tools up to do exactly the same thing. Make sure to use a Training Dummy type sim in AMR to compare to raidbots. Also, we use different rotations/apls. I’m happy to explain the thinking behind any of the differences. In general, I use simpler rotations on purpose. The complexity in many of the SimC APLs is not what players will actually be doing in-game in many cases. And another common case is that they make use of knowledge that only a simulator would have to make spell choices. We intentionally limit AMR to what a very skilled human player can actually do.
There are also differences in the core engines of the two simulators that lead to divergence for a couple specific specs. It’s hard to say which one is definitively “better” - but I would argue that the AMR simulator will give you results that are closer to what you can do in the actual game. Some people (like me) consider that “better”, but there are some people who prefer a more robotic/theoretical approach.
A lot of that makes sense, but what I am having trouble wrapping my head around is the stat priorities that I get when looking at the two tools. The sim parameters more closely resembling a human’s results (despite this being Ask Mr. ROBOT!!! 
) shouldn’t, at least I wouldn’t think it would, impact which stats brought greater value.
Both (for BM Hunter) show Crit as top, but AMR says Haste>Vers>Mastery where RB says Mastery>Vers>Haste (admittedly, the resultant weightings are all within a few hundredths of a point, so… maybe nothing to see here?).
Your comment about limiting AMR to what a human would know rather than perfect knowledge is interesting; do you have a specific example this is being done?
I’m not sure why SimC is preferring less haste for BM than we are. also, we don’t really use “stat weights” anymore because they’re pretty coarse and basic compared to our more advanced gear ranking techniques.
In general: If you do a single simulation with AMR and SimC you will get a result that is very close/similar. This is expected if both simulators have the game modeled correctly. The stat weights that simc calculates uses a very, very basic method that only looks at 2 data points per stat. We generate thousands of data points and from that create a predictive function for the stats. It is a much better predictive model of what stats will give you the best DPS - and our function can adapt to your item level. SimC/Raidbots puts almost zero effort into actually analyzing the data. AMR is all about analyzing the data.
We have some forthcoming blog posts where we’re going to try to explain it… but, using single data points from simulations to compare gear isn’t a statistically valid comparison. If you simulate set of gear A and set of gear B and A does 1% more damage than B in your two results… that doesn’t tell you to use set A. That tells you it doesn’t matter if you use set A or B. It doesn’t matter what margin of error(iterations) you use for that test, they’re too close to distinguish. Simulators are good to generate large sets of data to be analyzed for trends. They are not good for “point comparisons”. The main reason for this is because WoW simulators in particular do not exactly model the game they are simulating. It is not possible to measure the error of the simulator compared to the game.
An example of perfect knowledge causing what I believe to be unrealistic sim results is how SimC APLs make liberal use of functions that check when the fight will end to decide how to time up/sync cooldowns. A lot of that logic depends on knowing exactly when a fight will end. Players cannot know this exactly. At best… maybe you can guess within 15 seconds? Depends on your raid group. But there are lots of rotation tweaks in SimC that end up getting one more fractional use of a cooldown because of this knowledge. I could do that in the AMR simulator, but I don’t. It’s not real.
Another difference is that we make all of our gearing strategies on fights with realistic amounts of movement.