Weird impact of locking stuff on BiB optimize


#1

Hello,

For the purposes of this topic, I’ll use this strategy, with the following import string:

$50;EU;Ysondre;Maximaw;Pas ltemps de niaiser;11;2;110;10:800,15:207,12:800,13:800,14:700,4:800;1;.s1;28;3112332;291,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1044,37,202,1,1,92;1,1,1,1,1,1,1,3,3,3,3,3,3,3,3,3,1,1,4,1,1,1;140842b1487b3516b3528,142186b1532b3337b3418,140841b1487b1813b3516;.s2;29;0000000;899,2,2,2,1,2,1,1,1,1,1,17,421;1,1,1,3,3,3,3,3,3,3,3,1,1;140837b1487b3516b3528,143685b1517b1824b3528,;.s3;30;3212233;1102,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,235,37,304;1,3,4,3,3,3,4,3,3,3,1,1,1,1,1,1,1,1,1;140839b1472b1813b3514,139250b1507b1806b3336,139263b1487b1805;.q1;128935s16b744x140842y1344z-1345;1s17;4701s12b808b1785b199e5427;843s3b-1984b256b1529b199x-10593e456;2555s8b-1725b1644b3;67s9b-1647b1644b3;1239s10b-1966b1844b180;2s1b-2029b326b1703;3s5b-2024b1844b180;26s15b-2014b311b1705e-447;839s6b-2001b289b1531;121s2b-1820b289b1531e3;1468s13b-1845b1844b180;9s14b-2019b1839b181;62s7b-1995b1815b108;24s11b-1938b1830b179;.q2;128819s16b745x140837y2848;54s17;3593s7b1066b1644b3;1339s6b-1906b1785b199;518s11b-1994b266b1720b45;157s3b-2021b256b1529b199x-13437e5883;2925s9b-1994b1795b75;936s10b-1920b1844b180;2s1b-2029b326b1703;3s5b-2024b1844b180;26s15b-2014b311b1705e-447;867s2b-2006b294b1530e454;1566s13b-1849b326b1703;62s8b-2024b1844b180;30s12b-2024b316b1709b11x-26e-461;3362s14b-69b71;.q3;128911s16b746x140839y-1589z13;23s17;3510s2b1065b1644b3x-9041e5437;1193s12b-1906b1785b199e-10;168s6b-1984b1785b199;675s3b-1984b256b1529b199x26e456;2555s8b-1725b1644b3;370s9b-1916b1795b75;936s10b-1920b1844b180;2s1b-2029b326b1703;3s5b-2024b1844b180;26s15b-2014b311b1705e-447;2420s13b-2021b316b1632;13s14b-1958b326b1703;66s7b-1994b1815b108;24s11b-1938b1830b179;.r;_;.inv;6218;38;692;31734;6356;7972;633;17440;9972;5088;1256;22294;867;996;0;2032;1916;911;1684;373;2473;366b604b903b167b1800;1307;1489;0;1;1;0;1;1;1;0;4;1;0;1;0;1;0;1;0;1;0;0;3;0;0;0;0;313;0;0;0;0;0;0;0;0;0;0;0;0;0;1;0;0;0;1;0;0;1;0;1;0;1;0;2562;33;644;153;1;5;3;0;3;507;466b-2729x140837y2848;92b1x-2846y-1589z13;89;100;828;977;904;117;1;517b1065b1644b3x-9041e5437;22b-1647b1644b3;1104;2;16;0;1;0;1;1;0;1;1;0;14;30b-1936b291b1696;43;1;0;0;0;22;1;13;2;1;1;3;31;50b-1957b1785b199;179;176b-2931b902b167b1800;3b-1967b1829b137;28b-1951b1815b95;57b-2828b913b1819b138;31b-1967b167b1800;44b-1932b266b1720b45;85b-3532b1496b276b1723;87b-2024b301b1604;34b-1895b291b1697;1282b-2905b867b1956;226b-2825b894b1839b149;305;391b-1953b1805b95;592b-3390b1480b291b1697;24b-1998b301b1604;56b-1875b1795b75;14b-1910b225b1609;19b-1819b1819b78e-10;204;39;430;109b-1922b315b1530e2;1b-1830b299b1530e7;900b-1799b1800b80;54;37b-1915b304;2b-304b304;2b-304b304;1b-304b304e447;3b-304b304;2b-304b304;22b-254b1785b199;78b-2034b304;1b-304b304;7b-1766b1462b304b2x-3;88;374;17;32;189b-306b1834b137;139;23;379;16;205b-1976b316b1632;12b-1983b1874b184;1b-2033b326b1703;50b-2029b326b1703e-447;35b-2029b326b1703;4b-2029b1849b108;0b-3403b1441b331b1631;2b-1967b1859b178;1b-2022b316b1709b11x3e-7;3b-2061b1869b184;0b-2033b326b1703;0b-2029b326b1703;296;128;1;121;159;47;465;55b-2044b336b1645x26e-2;35b-1971b1854b117;8b-1936b1820b81e9;86b-1926b1845b117;34;0;1;65;19b-1957b311b1699x-26;101b-2010b2010;1;259b-2020b326b1649;548;149;1;182;20;51b-1651b1718b1;527b-71b71$

The BiB advises those results. It sims at 781k DPS

Here comes the weirdness. If I lock Prydaz’s gem and enchant information to what the optimizer suggests, and then save and update, the results are different, by a fair amount (~10% mastery)

(can use image tag because of newbie restrictions) /uploads/db4666/original/1X/cff5e06442275ec2d7cc3680057f412d069014d9.jpg

This ‘constrained’ setup sims a bit higher (altough with the margin of error indicated by the simulator, it’s real) than the free one, at 787k DPS: (begin redacted because of newbie restrictions) simulator/report/779b279421e7437bb9d30722c807c3ed)

I am not really sure what to make of it. Should I lock a few key gems/enchants to ‘help’ the simulator?
What’s your take on this?

Thanks and regards,


#2

My take is that 7k out of 780k or so is very small… 0.8%. From the perspective of an optimizer, it will be hard to distinguish between those two setups without dramatically increasing run time.

I can try it out later to see if anything is going on, but I suspect not.

In the near future I plan to add a way for you to use a simulation result to refine an optimization easily. As in this case it will not make much difference, but some people might like trying to squeeze out that half or one percent by letting a batch of sims crank.


#3

The thing is, it is really reproductible.

If I start the optimization unlocked, I get the first result, every single time.
If I lock the gem and enchant, I get the second one, every single time.

I don’t know if you are caching optimization results, but this is beyond two local minimums, I think.


#4

The optimizer is deterministic, so given the same starting conditions, it will always produce the same result.

It looks like in this case, locking the gem/enchant causes it to find a different solution that scores almost exactly the same. (The score isn’t shown right now, but I’m able to see it on my dev server. With nothing locked it finds a set that scores 783775, with the gem/enchant locked it finds a different set that scores 783698.)

Firstly, you will notice how close these are to your simulations

Set 1: 784k predicted, 781k actual
Set 2: 784k predicted, 787k actual

Secondly, you will notice that as far as the optimizer is concerned, it has found the better of the two solutions when you don’t constrain it.

Thirdly, you will see that the predicted value is only 0.38% different than the actual value for each set.

Getting an optimizer that has any better margin of error than that would be… kind of crazy. At that point you would just have to start simulating many more combinations directly, rather than trying to predict it based on a smaller sampling of simulations. This post might be of interest to you as well: BiB Suggestion Sims Lower Than Original Gear

In this case, you were able to do some direct simulations that show a theoretical gain for one set over the other, so go ahead and use it (and as mentioned earlier, we want to make these “user refinements” more easily integrated with BiB in the future). The optimizer is getting you really close though, to the point where most players won’t feel it, without having to do any manual tweaking or simulation.