Issue638

Title Fix implementation of "interesting" patterns
Priority bug Status resolved
Superseder Nosy List florian, jendrik, malte, silvan
Assigned To florian Keywords
Optional summary

Created on 2016-03-14.19:47:30 by florian, last changed by florian.

Messages
msg5988 (view) Author: florian Date: 2016-12-22.23:56:06
Merged. Step 2 is in issue702.
msg5986 (view) Author: malte Date: 2016-12-22.22:55:26
OK.
msg5985 (view) Author: florian Date: 2016-12-22.22:43:55
I suggest we open a new issue for step 2 and close this one.
msg5984 (view) Author: malte Date: 2016-12-22.22:16:04
How do you suggest that we proceed?
I won't insist on "step 2" if you think we shouldn't do it.
msg5983 (view) Author: malte Date: 2016-12-22.22:12:14
The translator behaves in unreproducible ways on trucks-strips because we hit
the invariant synthesis timeout for some instances. So presumably different
configs work on different finite-domain representaitons here. Last time I
checked, it was the only domain with this issue.
msg5981 (view) Author: florian Date: 2016-12-22.22:08:43
Ahh ... right. I actually had the parser for that but it didn't run and I forgot
about it. I updated the report and added some scatter plots:

http://ai.cs.unibas.ch/_tmp_files/pommeren/issue638-v1-issue638-base-issue638-v1-compare.html
http://ai.cs.unibas.ch/_tmp_files/pommeren/issue638-v1-issue638-base-issue638-v1-num_interesting_patterns-pho-sys3.png
http://ai.cs.unibas.ch/_tmp_files/pommeren/issue638-v1-issue638-base-issue638-v1-num_interesting_patterns-pho-sys4.png

The one thing I was surprised by was that num_sga_patterns changes for some
trucks-strips tasks. This shouldn't happen because the only code change is on a
code path that is only entered after computing sga patterns. Any idea whats
going on there?
msg5976 (view) Author: malte Date: 2016-12-22.18:26:45
It seems a bit odd to me to make a change whose main purpose is to change the
set of patterns that is generated and then not measure the number of generated
patterns. Can we add this to the analysis?
msg5973 (view) Author: florian Date: 2016-12-22.14:59:10
For the larger patterns, there is a difference between the versions. The
coverage goes down by a few tasks, but the number of expansions is never larger,
showing that we get dominating heuristics by considering more patterns as
interesting and that these patterns are actually interesting.

I don't think we have to worry about the lost coverage here. All size 3 and 4
patterns are often too many, so when adding more, losing coverage is not surprising.

http://ai.cs.unibas.ch/_tmp_files/pommeren/issue638-v1-issue638-base-issue638-v1-compare.html
msg5953 (view) Author: florian Date: 2016-12-21.00:50:35
Ahh, yes, the patterns are only merged if they are disjoint. With size 2
patterns, there is no difference. I actually looked this up, when we first
discovered this bug. This is the reason why the published results in our papers
are unaffected by this.

I'm now re-running the experiment for size 3 and 4 patterns.

We should probably update the documentation about "shared variables" as well.
msg5948 (view) Author: malte Date: 2016-12-20.21:30:46
If I see it correctly, the experiments only look at patterns of size 2. Can
there be any difference in the wrong vs. correct definition of "interesting"
patterns for such small patterns?

I tried to follow the definition in msg5187, but ended up confused, so I find it
difficult to answer the question myself. The text first says that the two
patterns two combine must "share a variable that is a connection point", which
to me suggests that it should be part of both patterns ("share"). But then
condition 2 seems to suggest that the connection point is not part of one of the
patterns.
msg5946 (view) Author: florian Date: 2016-12-20.18:30:17
The experiments showed no significant differences:

http://ai.cs.unibas.ch/_tmp_files/pommeren/issue638-v1-issue638-base-issue638-v1-compare.html
msg5907 (view) Author: florian Date: 2016-12-19.16:13:19
I created a pull request for the first step (fixing the existing implementation)
on bitbucket. Experiments for the PHO and the canonical heuristic are running.

https://bitbucket.org/FlorianPommerening/downward-issues/pull-requests/27
msg5187 (view) Author: florian Date: 2016-03-14.19:47:29
In the systematic pattern generation algorithm, we restrict attention to
"interesting patterns", i.e., those that might carry more information than the
combination of their subsets. We do this by first generating single goal
ancestor (SGA) patterns and then combine multiple SGA patterns to get the final
patterns. Two SGA may be connected if they share a variable that is a
"connection point" of one of the patterns. This is documented and implemented as
follows:

   A variable is a connection point if it satisfies the following
   criteria:
     1. We can get from the pattern to the connection point via
        an (eff, pre) or (eff, eff) arc in the causal graph.
     2. It is not part of pattern.
     3. We *cannot* get from the pattern to the connection point
        via an (eff, pre) arc.

I believe the direction of the arcs in condition 1 is wrong and should be "(pre,
eff) or (eff, eff)" instead.

Malte also suggested a faster way to compute interesting patterns that does not
rely on SGA patterns as basic building blocks but on patterns that have exactly
one goal variable. This is currently in the repository for our follow-up work on
the PhO paper:

  ai-repos/helmert/xxx-drafts/jair13-pattern-collection-heuristics

I suggest we fix this in two steps: let's first implement the direct fix, that
just reverses the arc direction in the existing implementation and then
implement Malte's algorithm as a second step.

We could do the first part in a subissue and merge it before we start improving
the algortihm.
History
Date User Action Args
2016-12-22 23:56:17floriansetstatus: reviewing -> resolved
2016-12-22 23:56:06floriansetmessages: + msg5988
2016-12-22 22:55:26maltesetmessages: + msg5986
2016-12-22 22:43:55floriansetmessages: + msg5985
2016-12-22 22:16:04maltesetmessages: + msg5984
2016-12-22 22:12:14maltesetmessages: + msg5983
2016-12-22 22:08:43floriansetmessages: + msg5981
2016-12-22 18:26:45maltesetmessages: + msg5976
2016-12-22 14:59:10floriansetmessages: + msg5973
2016-12-21 00:50:35floriansetmessages: + msg5953
2016-12-20 21:30:46maltesetmessages: + msg5948
2016-12-20 18:30:17floriansetmessages: + msg5946
2016-12-19 20:49:39floriansetstatus: chatting -> reviewing
assignedto: florian
2016-12-19 16:13:19floriansetmessages: + msg5907
2016-03-20 14:08:35silvansetnosy: + silvan
2016-03-14 19:47:30floriancreate