My setup on the left continually sims better than the BiB suggestion. Why would BiB suggest a gearing strat that sims lower?
Zackster@Ghostlands
How much lower? And are you using a custom gearing strategy?
I ran about a dozen or so originally and the equipped gear was simulating around 810-815 and the suggested gear 785ish. I am using Machine Learning and no custom. I just ran a dozen or so more and the results are varying quite a bit. I may have to load up the local client to do some more advanced testing (many more runs of each). Quick (stupid) question. How do I tell it to run multiple runs instead of a single run?
Change the sim type to “custom batch”. Then you can run various combinations of things.
If you’re talking about the same simulation setup giving you different results on each run… maybe you need to set a lower margin of error under “More Options”? 0.25% is usually more than sufficient.
I ran each with .05% and the original was 814 and BiB 811. Not as much lower as originally but still lower, Is this just margin of error?
Yeah – out of 814k DPS that’s only a 0.36% DPS difference, it would be extremely difficult to get any closer than that with consistency in an optimizer.
You want to think of it as a tradeoff between speed and accuracy.
The most accurate way to figure out the best setup would be to go in-game and try them out enough times to average out the game’s inherent randomness (crits, procs, etc.). But that is more or less impossible… it would take until the end of the universe in real game time, and chances are you would get an upgrade or two before finishing, making it moot anyway 
The next most accurate way we have come up with is simulation. That’s pretty good – for a given setup, it can get a pretty accurate answer in about 5 seconds, give or take. But when it comes to best in bags… there can be upwards of a trillion combinations worth trying if you have a lot of good gear. To simulate all of those we’re back to around 100,000 years (actual estimate, not made up)… once again, not that useful.
So we create “gearing strategies” which are predictive models based on a small sample of simulation data. They lose accuracy (we are in the process of reducing and actually measuring how much accuracy is lost, but it’s around 1-2% right now), but they can get an accurate answer for any given set of gear almost instantly. But even then, it would take a long time to do a trillion combinations… maybe around 10 minutes, because it’s only almost instant. That’s a long time to wait when you press the “best-in-bags” button, not to mention how expensive it would be to run – we’d have to charge you ridiculous prices to host that.
So the final piece is our combination optimizer. It knows how to sort through all those combinations intelligently and find the best one without actually having to test all of them. You lose a very small amount of accuracy from this, it is actually no loss in over 95% of cases, and less than 0.25% for the rest. But now you can do best-in-bags in under 1 second for most cases.
So to go from 100,000 years via simulation down to 1 second via best-in-bags, you end up losing around 1-2% accuracy right now, and we’re always working on improving that. Our ultimate goal is to always be within 1% of the best simulated setup for your character, though most of the time it will be much closer than that. We feel that is the practical “floor” for accuracy, and we are closing in on reaching it. Anything lower would be prohibitively expensive to build, and also of dubious usefulness to even the best players (for reasons that I plan to write up sometime soon).
That all said – we created the simulator for a reason! If you are able to find a setup that is better than what best-in-bags suggests, by all means use it. We plan to actually make that easier in the near future – if you want to let your CPU crank away and try to refine best-in-bags further than what the optimizer can do, we’ll give you a way to save and use those results with ease.
Perhaps if the BIB result is only within 1-2% of the gear we already are wearing, it should suggest that instead of what it thinks is better? This would save many of us the time and gold spent revising our gear that may or may not be any better than what we already have? Or, automatically run a quick simulation after the calculation to make sure it checks out before making the suggestion?
At various points in the past we have had ideas to put in some kind of “fuzz” on best-in-bags, or a threshold you would have to pass before it suggests changing things. Invariably though, the feature gets put off because most people prefer the solution that scores higher. We might circle back to it though.
We have some UI updates for ToS in store that should help you determine if the changes are worth the trouble or not.
You can definitely run a simulation to compare your current gear to the best-in-bags gear, there are a couple buttons to make that relatively easy. We plan to make it easier in the future in various ways. It will be an on-demand feature though, would be far too expensive to run a simulation for every optimization that we run.