Merged and pushed.

We updated the code comment, and this looks ready to be merged.

Thanks! Nothing concerning about the load factors.

I'll copy your observation about the airport task to issue537, which is concerned 
with making blind search faster again.

The report now aggregates the load factor using the arithmetic mean.

Interesting. I'm surprised to see the timeouts in airport, so I ran Airport #24
on my desktop at home (which is rather crappy, as it is designed not to require
a fan). It ran out of memory in 1000 seconds, which would mean that it is at
least roughly a factor of 2 faster, but I guess this is feasible, as I wasn't
using all cores concurrently and when we last compared desktop and server
machines, the single-core performance of the desktops was much better. Still,
I'm a bit surprised to see memory not being a bottleneck in blind search, and it
would be interesting to do a profile on such a problem at some point. But of
course this is not what this issue is about.

Regarding this issue, it's hard to analyze the load factor from the report
without going through all instances individually due to the way it is
aggregated. Would it be possible to give averages rather than sums?

Otherwise I guess all that is left is reading the comment.

I don't see any other change between v2 and v3 apart from the bit-twiddling which 
I'd expect to be responsible for the increase in speed.

The experiment uses no timeout for the translator (it uses an older version of Lab).

Here is the diff: https://bitbucket.org/jendrikseipp/downward/branches/compare/issue694-v3..issue694-v2#diff

Unfortunately, Bitbucket does not do a good job of formatting it.

Nice to see these improvements. :-) Also nice to see how close 32 and 64 bit are
now (at least for blind search) in terms of memory score.

I'm somewhat surprised to see so many timeouts in airport. Does this use
separate translator and search timeouts, or does it use an overall timeout?

I would still like to have a look at the code comment, but I don't think I'll
have time today. Feel free to remind me tomorrow.

Can you send a link to a diff between v2 and v3? I'm curious if the time
improvements are solely due to the modulo to bit-twiddling change, or if there
are other significant code differences.

Here are the experiment results (they were run on the new grid):

http://ai.cs.unibas.ch/_tmp_files/seipp/issue694-v3-opt-issue694-v2-issue694-v3-compare.html

Unexplained errors: 
- 8 parcprinter tasks write more than the soft limit to stdout. We already know about this problem.
- 2 scanalyzer tasks and 1 satellite task can't be translated since they need too much memory. Since we run 4 
algorithms, this explains the 12 "slurmstepd: error: Exceeded step memory limit at some point." messages. We 
should investigate why the translator needs more memory on the new grid than on the old grid. The first step, 
however, should be to limit the memory for the translator, which I just implemented in downward-lab.

Coverage stays the same, total_time decreases and memory increases slightly.

The only thing I didn't do yet is read the main comment, which I want to do
later. Because we now do the bit-twiddling, we probably need to run a second
round of experiments? If yes, feel free to run them now. My possible future
comments on the comment shouldn't lead to another round of experiments. ;-)

I'm done with the changes we discussed. Could you please have a look again?

Done with the code review (mostly offline).

Here are the results of the experiment:

http://ai.cs.unibas.ch/_tmp_files/seipp/issue694-v2-opt-issue694-v2-base-issue694-v2-compare.html

I think the memory, time and coverage scores look good. One important parameter is the maximum 
distance between an entry's ideal and actual bucket. In the experiment above the maximum distance 
is 32. I also ran an experiment where the maximum distance is 64:

http://ai.cs.unibas.ch/_tmp_files/seipp/issue694-v1-opt-issue694-v1-base-issue694-v1-compare.html

While the memory scores are slightly higher than for maximum_distance=32, the time scores drop. 
Coverage scores are almost unaffected.

I propose to stick with maximum_distance=32 for now.

I copied the code for the new hash table from issue213, dusted it off and prepared a new 
pull request at https://bitbucket.org/jendrikseipp/downward/pull-requests/72 . 
Experiments for blind() and lmcut() are running. I'll add this to the review queue.

Split off from issue213.