Title move heuristic progress statistics to Evaluator
Priority feature Status resolved
Superseder Nosy List guillem, jendrik, malte
Assigned To jendrik Keywords
Optional summary
part of issue727.

Created on 2017-11-29.17:17:16 by jendrik, last changed by jendrik.

part of issue727.
msg6670 (view) Author: jendrik Date: 2017-12-01.10:03:33
Yes, the experiment uses the usual randomization. I merged and pushed the code.
msg6668 (view) Author: malte Date: 2017-12-01.00:33:31
Did you run base and v1 at the same time in the usual randomized manner?

We have a clear systematic speed improvement in the blind search code:
improvement of score_total_time in 44 domains and decrease in no domains. Some
of the runtime improvements in the detailed results are astonishing for such an
innocuous change, on the order of 10% of total planner runtime (!).

As usual this gets lost in the noise in the configuration where the state space
operations are not the bottleneck, but it's still something that should make us
wonder what is going on here.

After looking at the diff again, I think the explanation (support by some
further research, i.e., googling) is probably that the dynamic casts that we
used to do are quite a bit slower than I thought they would be. Interesting.

In any case, I'm happy to see this merged.
msg6667 (view) Author: jendrik Date: 2017-11-30.22:44:58
I think the experiment results look good. The statistics output is also still the 
same for the two tasks for which I checked it.
msg6665 (view) Author: jendrik Date: 2017-11-30.19:24:16
The results are in:
msg6663 (view) Author: malte Date: 2017-11-30.17:45:45
Code looks great.
msg6659 (view) Author: malte Date: 2017-11-30.17:26:34
Done with the comments on bitbucket.

Regarding tests: I suggest lama-first and blind search. We should also manually
inspect the output to see if it is the same as before. (Doing this for one task
is enough.)
msg6658 (view) Author: jendrik Date: 2017-11-30.16:14:14
I made the requested changes. Is the code fine now? Which configurations would 
you like me to test?
msg6649 (view) Author: malte Date: 2017-11-30.12:11:43
I'm done commenting on bitbucket. Let's continue the discussion offline.
msg6637 (view) Author: jendrik Date: 2017-11-29.20:16:44
Complete link:
msg6636 (view) Author: jendrik Date: 2017-11-29.20:16:06
I made a pull request at
requests/77 .
msg6633 (view) Author: jendrik Date: 2017-11-29.17:17:16
Add enable_statistics instance variable to Evaluator class which can be overriden 
by inheriting classes. Move Heuristic::get_description() to 
Date User Action Args
2017-12-01 10:03:33jendriksetstatus: chatting -> resolved
messages: + msg6670
2017-12-01 00:33:31maltesetmessages: + msg6668
2017-11-30 22:44:58jendriksetmessages: + msg6667
2017-11-30 19:24:16jendriksetmessages: + msg6665
2017-11-30 17:45:45maltesetmessages: + msg6663
2017-11-30 17:26:34maltesetmessages: + msg6659
2017-11-30 16:14:14jendriksetmessages: + msg6658
2017-11-30 12:11:43maltesetmessages: + msg6649
2017-11-30 10:55:48guillemsetnosy: + guillem
2017-11-29 20:16:44jendriksetmessages: + msg6637
2017-11-29 20:16:06jendriksetstatus: unread -> chatting
messages: + msg6636
2017-11-29 17:26:45jendriksetassignedto: jendrik
2017-11-29 17:17:16jendrikcreate