issue625: fix naming convention for existing namespaces. 

Sub issues for splitting off individual namespaces:
  evaluators (issue599)
  heuristics (issue599)
  landmarks (issue599)
  lp (issue599)
  merge and shrink (issue599)
  open lists (issue715)
  operator counting (issue599)
  options (issue626)
  pdbs (issue599)
  potentials (issue599)
  search engines (issue603)
  utils (issue607)
  task_utils (issue720)
  ... (create sub issues for separating a new module)

I suggest closing this. We haven't acted on it for a while, and there are no
specific open tasks. I would still like to eventually have a clearer notion in
the code what is or isn't he planner core, but we can open an issue if and when
we want to work on this.

Any objections? I'll mark it as resolved now so that we don't have to do
anything further if we're happy with resolving it, but if you think there is
value in keeping this open, feel free to reopen.

It's hard to answer this without going over all files. Just a quick grep for
files that don't contain the word "namespace" (which is of course not a
sufficient test for testing that everything is OK) shows quite a few files
related to search algorithms that I don't think should be part of the core.

Perhaps it would also be useful to either introduce a namespace for the core, or
to at least have some way of communicating that something is considered part of
the core and hence should not have its own namespace. Right now it's difficult
to tell if it's intentional when something is not put into a namespace.

Can anybody think of additional code that we'd like to put into its own namespace? 
If not, we can probably close this issue.

cross-reference issue720

cross-reference issue715

The options namespace was introduced in issue626.

We decided to change the naming convention for namespaces to use lower case
names (like file and directory names) to avoid classes with the same name as
their namespace. Such classes lead to problems with forward declarations, e.g.,
we cannot use "using Options::Options;".

Updated list of current namespaces and links to issues.

We delayed introducing a namespace for the option parser from issue606, because
it currently would introduce a lot of conflicts. We first have to uncouple the
code a bit more.

We discussed to stick with two directories for heuristics and evaluators today.

> Should we make this into a meta issue and create individual issues for the
> parts? For example, one for the "easy" part and one each for search, state,
> task and utility classes?

Sounds good to me. I wouldn't create all of them at the same time, but rather in
a more piecemeal fashion as we get to work on them. For example, "utility
classes" was meant as an example of how one *might* separate out part of the
code into a somewhat coherent component. It wasn't meant as a well-considered
suggestion.

On 1): Ideally, I would like to discuss the Heuristic-ScalarEvaluator merger in
a live discussion with everyone who has an opinion on the matter. For me, the
most difficult question at the moment is how one would name the joint class. If
we can easily come up with a plan for how to merge them, then I'd be in favour
of moving to the final directory structure straight away. If it looks more
complicated, I guess it's OK to start with two directories "evaluators" and
"heuristics" even if they turn out to be short-lived. But as we discussed
before, moving files around is somewhat disruptive w.r.t. merging etc., and
perhaps it's not wise to do it if we can already expect moving them again soon.

On 2): no objections to creating the lp directory. I just wanted to ask the
question because it's a possible objection that some of us might feel more
strongly about. (I don't.)

Splitting off one component at a time sounds good to me. Should we make this
into a meta issue and create individual issues for the parts? For example, one
for the "easy" part and one each for search, state, task and utility classes?

On your questions:

1) I'm assuming we want to change all evaluators into heuristics or all
heuristics into evaluators, and that we don't want to do this all at one time. I
suggest to leave evaluators and heuristics in two directories as long as they
are different. If we make some evaluators into heuristics (or the other way
around) we can move them from one directory to the other. This way we could see
which parts are already translated and which ones aren't. If my assumption is
wrong, a common directory might make more sense.

2) I don't have a strong opinion on having the lp in a subdirectory, but it
could make sense for two reasons: we planned to make the interface more
efficient (issue506) which will probably increase its size, and it is a separate
component that most people can ignore when they start to understand the core
code ("Don't look into directory 'lp' unless you need an LP solver").

I updated the suggestion in the summary.

I understand your concern regarding the search code. We want to avoid mixing
"core code" and "plug-in code". I think that the problem here is that the code
was never designed with "search engine" as an actual abstract interface in mind.

Our heuristics/evaluator interface was designed to make these components
replaceable, but the search code was originally monolithic, and the current
design of SearchEngine was created mainly to reuse code between lazy and eager
search, not to provide a well-thought out general abstraction for search algorithms.

As a consequence, the base class already contains what are essentially
implementation details of lazy and eager search. Some components, such as the
search space, probably shouldn't be there. (I can't see it being useful for
something like symbolic search, if we ever wanted to add that, or to give an
example in the current codebase: I can't see it being useful for iterated search.)


Stepping aside from the concrete discussion about the search code, the more I
think about this, the more uncomfortable I am with us having a discussion about
the directory and namespace structure for all parts of the planner at the same
time. I've now spent around two hours on this comment, editing and deleting
stuff, and thinking about how various parts of the search code work together.
Investing this effort would make sense if we were actually planning to work on
the search code at the moment, but I don't think we are. (But volunteers always
welcome! :-)) There are other parts of the proposed structure that I am also a
bit uncomfortable with, but I am somewhat afraid to mention them because of the
cost of the discussion.


I understand and share the desire to clean up the code by moving files into
subdirectories and creating namespaces, but I think this only makes sense if the
resulting directories are cohesive, i.e., have *strong* internal connections
(thematically and/or in terms of implementation) and *weak* connections to the
rest of the code. In cases like the heuristics or evaluators, this is a given,
and it is easy to move them to a separate place. In many other cases, however,
this cohesion first has to be created because we currently have quite tangled
dependencies. The search code I mentioned at the start is an example: I think
that the "abstract interface" (SearchEngine) currently depends on implementation
details (SearchSpace, SearchNode etc.) in ways that it should not, and
when/before introducing a "search" subdirectory, we should tackle this issue,
e.g. by introducing a new class between SearchEngine and LazySearch/EagerSearch.
(Just as an example -- I suggest we only discuss this in more depth at the point
where we want to work on it.)

To me, the underlying high-level problem is to identify cohesive logical
components in the planner code and then make the code layout (in terms of
directory/file name structure and namespaces) agree better with the partitioning
into logical components than it does currently.

The benefit of this, if done well, is that it can become easier to understand
the overall picture. For example, I would like to be able to say something like
"If you want to implement a new search algorithm, the part of the code that you
want to study in depth is A. Parts B and C are also somewhat relevant, but it's
enough to have a superficial understanding. You don't have to worry at all about
D and E."

I think a more productive way to do this is one component at a time. For
example, we could start with the easy cases. To me, the cases that look easiest
at the moment are the proposed directories "heuristics", "evaluators", "options"
and "lp". All other (new) directories I think need further discussion that I
would rather defer until the point where someone wants to work on this
component. In other words, I'm not happy with the proposed structure for these
others, but I also don't want to start a detailed discussion on them until we're
ready to work on them.


Focusing on the easy cases:

1) Do we want to have a separate directory "heuristics" and "evaluators" or a
single directory for both? I should mention that we are working towards getting
rid of the difference between the two concepts, so they will eventually be the
same thing. (Or at least that's my understanding of our latest discussion on
this topic.)

2) With two relatively small source files, is the LP code large enough for its
own subdirectory? Counting the lines of code for all four suggested new directories:

- evaluators: 470 LOC, 7 cc files
- heuristics: 2968 LOC, 13 cc files
- lp: 674 LOC, 2 cc files
- options: 2165 LOC, 3 cc files

Added cross-reference to issue61 and issue266.

Just for fun, I created a graph of the planner files with edges A --> B if A
includes B. I don't think it will help much with the issue but you can see some
of the plugin dependencies there.

I tried to maintain the 1:1 mapping between namespaces and CMake plugins and I
thought we only wanted two kinds of subdirectories: subdirectories containing
all files of one namespace and subdirectories that group files from namespaces
that are too small to get their own subdirectory.

I tried to update the summary with your suggestions, but I'm not sure if I
understood you completely. Did you want the new subdirectories (utilities for
example) to get their own namespace? their own CMake plugin?

Thanks for the suggestions!

Regarding your last message, I think I understand the suggestion, but I don't
think I understand the underlying rationale. (Or perhaps I don't agree with it.
;-)) I don't think we should have cmake plug-ins just to justify to ourselves
that something like the option parser "is allowed to" live in a subdirectory. (I
have nothing against introducing a namespace for the "options" subdirectory:)

Based on what I've seen so far, I'm not in favour of the "core" subdirectory in
your proposal, because it doesn't seem to be a cohesive unit. I'd rather have
these files remain where they are. Then the main source directory is still a bit
of a mess of unrelated things, but I don't think it's really better to instead
have a "core" directory which is a mess of unrelated things.


I'm sure we can find additional cohesive groupings within these files if we want
to reduce the number of files in the main directory. For example, there are a
bunch of modules that are utility modules in the sense that they provide general
services that you can easily imagine using in any other C++ program that have
nothing to do with planning, such as countdown_timer.cc, rng.cc, system.cc,
system_unix.cc, system_windows.cc, timer.cc, tracer.cc, utilities.cc,
utilities_hash.cc. (Some of the utilities may actually be planning-specific on
closer inspection; this is just a first impression.)

I would package the search-related code a bit differently, including
search_engine.cc, search_progress.cc, search_statistics.cc, search_node_info.cc
and search_space.cc with the search code. (An argument can be made for keeping
search_engine.cc separate, just as with heuristic.cc.)

Regarding directory names, I think the module currently named "search_space" has
more to do with states than with search, and I'd reflect that in the name.

Now that issue119 is merged, we can also designate some CMake-plugins as
"dependency-only" which hides them in CMake GUIs and enables them iff they are
needed as a dependency for other plugins. We already use this for relaxation
heuristics and the LP solver.

We could also use this to avoid splitting the core namespace across directories,
by having a dependency-only namespace/plugin for "Options" and having the core
depend on it. On the other hand, its a bit weird to have dependencies in the core.

I added an initial suggestion for the split into namespaces and directories to
the summary, so we have a basis for discussion. Feel free to edit and comment.
The core namespace is split up across multiple directories, because we wanted
individual directories for tasks, options, etc. I wasn't sure whether we also
wanted individual namespaces for them.

We had some further discussion about this today. I've updated our coding
conventions page with the main points, but some of the main conclusions are:

- We want to finally introduce more namespaces soon.
- There will be a 1:1 mapping between namespaces and CMake plugins.
- Namespaces are no longer 1:1 with directories: "small" plugins (e.g. the blind
heuristic) will get their own namespace, but not their own subdirectory.
- We will use subdirectories in certain cases to group related code, e.g. all
heuristics without their own namespace, or all open lists.

In addition to the namespaces for directories, each heuristic should gets its
own namespace to avoid conflicts for commonly used concepts such as operator and
fact classes at the heuristic level. Class names like LandmarkCutHeuristic could
then be replaced by things like LandmarkCut::Heuristic.

This would also make the plugin mechanism (issue117) stronger by reducing name
clashes.

Each subdirectory should correspond to a namespace, using CamelCase names:

  subdir learning   => namespace Learning
  subdir open_lists => namespace OpenLists

Once we've got the name spaces, we can rename some of the files and classes to
be less redundant. Example:

 file   landmarks/landmark_cost_assignment.h => landmarks/cost_assignment.h
 guard  LANDMARKS_LANDMARK_COST_ASSIGNMENT_H => LANDMARKS_COST_ASSIGNMENT_H
 class  LandmarkCostAssignment               => Landmarks::CostAssignment

Once this is done, we should update the text in coding conventions referring to
namespaces, which currently says "(As of r3821, this has not yet been done.)".