Issue338

Title Revise exit status from search
Priority wish Status resolved
Superseder Nosy List florian, jendrik, malte, rpgoldman
Assigned To jendrik Keywords
Optional summary

Created on 2012-05-30.19:33:31 by rpgoldman, last changed by jendrik.

Messages
msg2449 (view) Author: jendrik Date: 2013-05-17.15:34:18
Merged.

The exit codes are explained at 
http://www.fast-downward.org/PlannerUsage
msg2448 (view) Author: malte Date: 2013-05-17.14:21:21
Yes, please do.
msg2447 (view) Author: jendrik Date: 2013-05-17.12:50:32
I accidently commented on the wrong issue (issue376). Pasting the comment here for completeness:

"From Florian's and my point of view the branch can be merged. I also conducted an 
experiment which confirmed that the logic in portfolio.py works and the planner 
is correctly stopped in out-of-memory and timeout situations."

Are you ok with merging this branch as well, Malte?
msg2435 (view) Author: malte Date: 2013-05-12.23:48:04
> EXIT_UNEXPLAINED_ERROR = 1,
>
> We currently only use this for a coin exception in the planner. However, we
> will also use it in the portfolio to signal that something seriously went
> wrong in one of the runs. E.g. one run exits with a segfault.

I'd rather call this a "critical error" then. It's not really "unexplained" at
any place where we explicitly raise it in the planner, after all.

We might also want to add an error code for unsupported inputs (this could e.g.
be used in verify_no_axioms_no_cond_effects).

> EXIT_TIMEOUT = 6,  # Probably only for portfolios
>
> Semantics: All configs reached time limit, no plan was found, none proved the
> problem unsolvable.

Is there a good reason to use this only in portfolios? I think consistent exit
codes for all planner invocations would generally make the user's life simpler.

> EXIT_MEMORY_OUT is used in the same way, but also for single planner runs.

OK, but the word "memory-out" still doesn't exist. ;-) Please stick with
"EXIT_OUT_OF_MEMORY".
msg2434 (view) Author: jendrik Date: 2013-05-12.23:31:58
EXIT_UNEXPLAINED_ERROR = 1,

We currently only use this for a coin exception in the planner. However, we will also use it in 
the portfolio to signal that something seriously went wrong in one of the runs. E.g. one run 
exits with a segfault.

EXIT_TIMEOUT = 6,  # Probably only for portfolios

Semantics: All configs reached time limit, no plan was found, none proved the problem 
unsolvable. EXIT_MEMORY_OUT is used in the same way, but also for single planner runs.

EXIT_MEMORY_AND_TIMEOUT = 7 # Only for portfolios

Semantics: As above, but both timeouts and memory-outs happened.
msg2433 (view) Author: malte Date: 2013-05-12.23:02:56
This is a good starting point. Some questions:

#    EXIT_UNEXPLAINED_ERROR = 1,

I'm not sure what this is supposed to mean.

#    EXIT_USAGE_ERROR = 2,  # Includes wrong sas files

Then it should probably be renamed to EXIT_INPUT_ERROR or some such, since
"usage" usually refers to the command line (only) in this context.

#    EXIT_MEMORY_ERROR = 5,

I'd rename this to "EXIT_OUT_OF_MEMORY", since it's not really an error as such
(or, put differently, "memory error" is a technical name to refer to an out of
memory condition in certain contexts).

#    EXIT_TIMEOUT = 6,  # Probably only for portfolios
#    EXIT_MEMORY_AND_TIMEOUT = 7 # Only for portfolios

I don't understand these two. What are the intended semantics, and why they are
only applicable to portfolios?
msg2432 (view) Author: jendrik Date: 2013-05-12.22:39:19
Florian and me have agreed on the following error codes, what do you think?

    EXIT_PLAN_FOUND = 0,
    EXIT_UNEXPLAINED_ERROR = 1,
    EXIT_USAGE_ERROR = 2,  # Includes wrong sas files
    EXIT_UNSOLVABLE = 3,  # Provably unsovable
    EXIT_UNSOLVED_INCOMPLETE = 4,  # Algorithm reported no solution, but is not complete
    EXIT_MEMORY_ERROR = 5,
    EXIT_TIMEOUT = 6,  # Probably only for portfolios
    EXIT_MEMORY_AND_TIMEOUT = 7 # Only for portfolios
msg2244 (view) Author: rpgoldman Date: 2012-05-30.19:33:30
Currently, the exit status of fast downward's search is non-zero in (at least) 
two cases:

1. Fast downward has searched completely (which may not be semantically 
completely), exhausting its search space without finding a plan.
2. Fast downward has encountered some form of error.

The first of these two cases is arguably not an error condition, and it would be 
desirable for scripts that use fast downward to be able to detect exhaustive 
search failure using the exit status.  Currently distinguishing exhaustive 
search, no error, from error can only be done by parsing output, and this is 
likely to be heuristic only going forward, as different messages could be 
introduced into standard output and standard error.

Malte has proposed the following reorganization:

1 - no plan found 
2 - usage error 
3 - everything else (maybe distinguish these further) 

From the standpoint of a script calling fast downward, this would allow for easy 
detection of:
* successful planning
* exhaustive search: no plan found
* error

Malte mentions a complication:

"...what happens on out-of-memory conditions is currently up the OS. The planner 
doesn't currently catch failed memory allocation at all, so there is no explicit 
setting of an exit code that we could change. So whatever outside behaviour you 
see is not (currently) under the control of our code."

Such conditions would have to be caught and brought under control of the search 
program so that an appropriate exit status could be set.

Possibly "catch out of memory conditions" should be added as a separate issue, 
with some cross-reference to this one.
History
Date User Action Args
2013-05-17 15:34:18jendriksetstatus: reviewing -> resolved
messages: + msg2449
2013-05-17 14:21:21maltesetmessages: + msg2448
2013-05-17 12:50:32jendriksetstatus: chatting -> reviewing
assignedto: jendrik
messages: + msg2447
2013-05-12 23:48:04maltesetmessages: + msg2435
2013-05-12 23:31:58jendriksetmessages: + msg2434
2013-05-12 23:02:56maltesetmessages: + msg2433
2013-05-12 22:39:19jendriksetstatus: unread -> chatting
messages: + msg2432
2013-04-24 14:52:31jendriksetnosy: + jendrik
2012-07-16 11:34:04floriansetnosy: + florian
2012-05-30 19:33:31rpgoldmancreate