Alright, I went ahead and merged and pushed the code.

If you think the results look good, I don't want to look at them.

I agree. 

I have run the requested experiments:

Blind search with limits 300s and 1GB for the newest branch revision. For one copy of the 
configuration, limits are set by lab, for the other limits are set with fast-downward.py.
http://ai.cs.unibas.ch/_tmp_files/seipp/issue546-v1-limits.html

Setting limits with lab for the FDSS portfolios:
http://ai.cs.unibas.ch/_tmp_files/seipp/issue546-v1-opt-fdss-issue546-base-issue546-v1-compare.html
http://ai.cs.unibas.ch/_tmp_files/seipp/issue546-v1-sat-fdss-issue546-base-issue546-v1-compare.html

I'd say the results look good. Setting the limits with the planner script seems to work identical to 
setting them with lab. The optimal portfolios still solve exactly the same number of tasks. There is 
some fluctuation for the satisficing portfolios, but I think this can be attributed to randomness. I 
had a look at one trucks-strips task where we lose coverage and in both versions the first 
configuration ('--heuristic', 'h=ff()', '--search', 'eager_greedy(h,preferred=h)') is given the same 
timeout, but in one case it finds a solution after 5 seconds, in the other it doesn't solve the task 
in 373 seconds. 

What do you think?

The last option looks best to me.

I am done going over your code review comments and will run experiments now.

Another thing we haven't really considered, I guess, is whether the option names are 
what we want them to be. I'm not sure which ones I prefer. Do you have an opinion?

1) --search-timeout, --search-memory (current)
2) --search-time, --search-memory (shortest)
3) --search-max-time, --search-max-memory (corresponds to search options)
4) --search-time-limit, --search-memory-limit (most speaking)

I think it would be good to have more verbose descriptions of how the limits
work, though, ideally as part of the --help documentation. This should answer
questions such as: what is the interplay between the overall limits and other
limits; what if anything special happens when using portfolios (e.g. I assume
portfolios just keep going if a search component exceeds its memory limit?); how
are external limits treated. Basically, everything a user needs to know about
the feature.

I'm done looking at the changes. I very much like the refactoring, in particular
the new `limits` module, but due to the moved I couldn't really follow the logic
100%, so don't take this as a sign that I think it works. ;-)

If a before/after test with some portfolio configs turns up no weird behaviour
and if an experiment with the new options doesn't cause trouble, I think that
would be proof enough that it's OK.

If you find the time, Malte, it would be great if you could have a look at the 
pull request:

https://bitbucket.org/jendrikseipp/downward/pull-request/30

We'd like to add the following parameters to the planner script:

--{translate,preprocess,search,overall}-timeout TIME
--{translate,preprocess,search,overall}-memory MEMORY

TIME can be given in seconds or with the suffixes s, m, h.
MEMORY can be given in MB or with the suffixes K, M, G.