Issue 188: Evaluate impact of new invariant synthesis on m&s - Fast Downward issue tracker

Title	Evaluate impact of new invariant synthesis on m&s
Priority	feature	Status	resolved
Superseder		Nosy List	gabi, malte
Assigned To	gabi	Keywords
Optional summary

Created on 2011-01-05.19:21:06 by gabi, last changed by gabi.

Files
File name	Uploaded	Type	Edit	Remove
lmcut_paper-d-abs.html	gabi, 2011-01-10.13:27:33	text/html
mas_paper_pipesworld-notankage-d-abs.html	gabi, 2011-01-10.13:27:26	text/html
mas_paper_pipesworld-tankage-d-abs.html	gabi, 2011-01-10.13:27:17	text/html
mas_paper_psr-d-abs.html	gabi, 2011-01-10.13:27:04	text/html
pipesworld-notankage-msg1131.html	gabi, 2011-01-11.15:46:00	text/html

Messages
msg1141 (view)	Author: gabi	Date: 2011-01-11.15:56:32
This issue already has been referenced in issue187 (strengthening the invariant synthesis), so I mark it as resolved.
msg1140 (view)	Author: malte	Date: 2011-01-11.15:54:31
Thanks! Looks like we're done here? Then it'd be good to reference this one from whatever related issue is open (e.g. the one on strengthening invariant synthesis) and mark it as resolved.
msg1138 (view)	Author: gabi	Date: 2011-01-11.15:46:00
Here is an overview of the coverage results for different N values for pipesworld-notankage (old vs. new): N = 500 16 15 N = 1000 18 15 N = 1500 22 15 N = 2000 23 15 N = 2500 23 15 N = 5000 23 13 N = 7500 23 11 N = 10000 23 11 Detailed results are attached.
msg1131 (view)	Author: malte	Date: 2011-01-10.15:38:57
> pipesworld-notankage (N = 2,500) 23 15 > pipesworld-notankage (N = 10,000) 23 11 I wouldn't be surprised if smaller N values than 2500 would be best here. I guess that part of the worse performance is that we have now so many variables that heuristic generation takes a very long time. Reducing N might help with that. Can you do a test run with N=500, N=1000, N=1500, N=2000? Actually, the sweet spot might also be higher than 2500, even though I doubt that. Still, experiments with N = 5000 and N=7500 would not hurt (only in this domain).
msg1129 (view)	Author: gabi	Date: 2011-01-10.13:46:04
I ran the configurations of M&S from the original paper and the LMCut-Paper for the critical domains. The detailed results are attached. Here is an overview over the coverage results (old vs new): openstacks-strips (N = 10,000) 7 7 pipesworld-notankage (N = 2,500) 23 15 pipesworld-notankage (N = 10,000) 23 11 pipesworld-tankage (N = 1,000) 14 13 pipesworld-tankage (N = 10,000) 14 14 psr-small (N = 10,000) 50 50 psr-small (N = 200,000) 50 50 Obviously, the performance on pipersworld-notankage is now really bad, so I am going to emphasize this domain in issue187. We also lost an instance in pipesworld-tankage.
msg1051 (view)	Author: gabi	Date: 2011-01-05.19:21:05
The new invariant synthesis finds much less invariants on a few domains which can be critical for m&s and related configurations. Thus, we should empirically evaluate the impact of the change on the configurations used in the papers on the topic for the following domains: * Openstacks-STRIPS * Pipesworld-NoTankage * Pipesworld-Tankage * PSR-Small * Woodworking-SAT and -OPT (not currently supported by M&S)

History
Date	User	Action	Args
2011-01-11 15:56:32	gabi	set	status: in-progress -> resolved messages: + msg1141
2011-01-11 15:54:31	malte	set	messages: + msg1140
2011-01-11 15:46:01	gabi	set	files: + pipesworld-notankage-msg1131.html messages: + msg1138
2011-01-10 15:38:57	malte	set	messages: + msg1131
2011-01-10 13:46:04	gabi	set	status: chatting -> in-progress priority: urgent -> feature messages: + msg1129
2011-01-10 13:27:33	gabi	set	files: + lmcut_paper-d-abs.html
2011-01-10 13:27:26	gabi	set	files: + mas_paper_pipesworld-notankage-d-abs.html
2011-01-10 13:27:17	gabi	set	files: + mas_paper_pipesworld-tankage-d-abs.html
2011-01-10 13:27:04	gabi	set	files: + mas_paper_psr-d-abs.html
2011-01-05 19:21:06	gabi	create