We merged the pull request and updated the wiki pages.

Marking this issue as resolved.

The changes in the repository are ready for review: 
https://github.com/aibasel/downward/pull/193

Once this has been merged there, we still need to make the changes in the wiki 
(removing the pages listed in the Summary and updating all references to them).

With Gabi, Florian and Silvan I created a branch "build-instructions" in my Fast Downward fork, but this is still work in progress:

https://github.com/maltehelmert/downward/tree/build-instructions

Direct link to the new BUILD.md file:

https://github.com/maltehelmert/downward/blob/build-instructions/BUILD.md


We removed some stuff completely, revised other stuff and left comments on further TODOs. The idea is that the current pages ObtainingAndRunningFastDownward and LPBuildInstructions will be removed (with some content migrated to other wiki pages) and that everything we consider worth keeping is currently part of BUILD.md.

Some comments on ObtainingAndRunningFastDownward:

Regarding 1, the readme does not cover the points about performance on the different systems and the support for time and memory limits. I'm not saying that this should stay on the wiki page or should be moved to the readme but I think it should be documented somewhere.

There is also a point in there about bug reports and patches that belongs more into a section "Contributing" in my view.

The "apt install" line under dependencies is valuable too. I think it would make sense to move this to the page QuickStart together with "git clone ...", "cd downward", and "./build.py" to replace the reference to ObtainingAndRunningFastDownward there.

Regarding 3, I also think the section on custom builds has value. In fact, it might be good to add a list of CMake options together with an explanation of what they do, as this is all undocumented behavior so far.

Regarding 5, the first point recommends to run experiments on the head of the development branch. Should we replace this to recommend using the latest release instead?

Regarding 6, maybe we could add a page on running scientific experiments. It would contain this point, the caveats from point 5, and the point about performance and time limits on Windows/Linux from point 1.

Thanks, Silvan! Some incomplete comments:

ObtainingAndRunningFastDownward:

Regarding 3, if this talks about how to do custom builds, I think it would be useful to preserve this information somewhere in the ForDevelopers parts. I need to look this up sometimes. Because of the different target audience, it can be shorted.

PlannerUsage:

Regarding 3, I would like to still say something about the exit codes; to me this part of the usage instructions. But it doesn't have to be a section, could just be a sentence at the end of the initial section.

QuickStart:

This was created when we made our first release and is referenced from the release pages. I would keep it for this reason.

I think it would be unfortunate to not get rid of http://www.fast-downward.org/ObtainingAndRunningFastDownward and http://www.fast-downward.org/PlannerUsage.

I went over the two pages.

ObtainingAndRunningFastDownward:

1) I think we could already remove Supported platforms and Dependencies, which are documented in the README now, except for the --validate option, which we could document there, too. We can keep the LPbuild instructions on the wiki and link to them.

2) Obtaining the code: I don't think we need a section for this. It is enough if the main page of the wiki says something like "this is the wiki page for Fast Downward, hosted on Github".

3) Compiling the planner: I think that it suffices to explain that there is build.py which can be used on Linux systems. I don't think that we should explain how to compile the planner *not* using build.py because our current instructions boil down to explaining how to compile a project using CMake, on different platforms and with different CMake parameters. To summarize, I think everything starting from Compiling on macOS can go, including the section Manual and Custom builds.

4) Running the planner: this can go; it is documented under PlannerUsage and can remain in our "running the planner" documentation (which could also be moved to the README, see below).

5) Caveats: it is certainly good to keep those but I would do so in the "running the planner" documentation.

6) "Known good domains" and "Experiments with Fast Downward": both are related to running experiments with the planner. I don't think we have to say anything about this. I just saw we also have http://www.fast-downward.org/ScriptUsage. We could keep this but I think we can also remove it.

PlannerUsage:

1) Usage: this can be condensed to what Jendrik suggested for the README.

2) Different builds: this is an important feature, which goes together with the explanation of build.py (and build_configs.py).

3) Exit Codes: this is basically a link to the dedicated page, which can be found easily on the main page. I would remove that section, too.

4) Examples: we can keep them, but I would remove the "(previously: "fFyY")" parts which must be very old, I assume.

5) "64 bit mode" and "other questions": both can be removed.


I also just found http://www.fast-downward.org/QuickStart (I don't think I have ever read it before) which also contains some examples of running Fast Downward. We can keep it; I don't have a strong opinion.

I currently don't plan to pursue moving all build instructions to the repo. What I suggest instead is to add something along the following lines to README.md below the software versions table. What do you think?


## Instructions

After installing the requirements (see above), compile the planner with

    ./build.py

and see the available options with

    ./fast-downward.py --help

For more details (including build instructions for Windows), see the documentation about [compiling](http://www.fast-downward.org/ObtainingAndRunningFastDownward) and [running](http://www.fast-downward.org/PlannerUsage) the planner.

I uploaded the patched COIN file to issue295.

I agree with the other points:
* Hosting the docs on GitHub sounds like a good idea.
* We shouldn't invest more time in the current wiki.
* We shouldn't split the docs into multiple places.

Therefore, I think it's best to wait with moving content from the wiki to the repo until we have moved the repo to GitHub.

Out of curiosity, I collected some options for hosting docs. After moving to GitHub, we could discuss the options in a Fast Downward meeting.

* Creating a docs/ directory that can be browsed on GitHub
* GitHub wiki (each repo has a wiki repo at https://github.com/YOUR_USERNAME/YOUR_REPOSITORY.wiki.git, e.g., https://github.com/d3/d3/wiki)
* GitHub pages (hosted from within the main repo under docs/ or in a separate repo)
  * Maintain docs in repo and convert them to HTML for GitHub pages with Sphinx (e.g., https://github.com/automl/SMAC3/tree/master/doc becomes https://automl.github.io/SMAC3/)
* Maintain docs in repo and let https://readthedocs.org/ host HTML versions (e.g., https://lab.readthedocs.io)

> Maybe one option would be to move even more of the documentation into the repo (for example
> as multiple markdown files in a "doc" directory)? For example, we could make the split of
> having user-level documentation in the repo and developer-level documentation in the wiki. We
> could also extend autodoc to generate more of the wiki automatically from the files in the
> repo. I'm not sure these are good suggestions, I'm just saying we should consider the
> different options before we decide on one of them.

I always thought that it would be good if more of the docs were part of the repository, as long as they can still be easily viewed online. The main argument against was the work vs. benefit ratio, and this is still a good argument against.

One point to consider is whether we might eventually replace our self-hosted wiki with GitHub's wiki functionality. If yes (I'd certainly consider this as an option), I would advocate against investing more work in this with the current wiki.

Maybe a good place for the COIN patch would be the issue where the patch was developed (issue295). There is already an attached configure script there but it is not the latest version. In the issue, the script was changed multiple times, and the final version was committed to the repository (but not attached to the issue). We eventually removed the OSI code from the repository and placed the modified file on the wiki.

As a said in msg9232, I think the documentation in the repo should contain information on how to obtain the code, because it can be viewed before you have the code on your disk. This would cover the sections 
* pointer to LegacySubversionRepository
* how to install mercurial
* how to use mercurial
* how to clone the repo

The following sections could fit in a file CONTRIBUTE.md but could also stay in the wiki
* description of our workflow
* how to contribute

Which would leave the information on how to use the planner.
* how to run the planner
* caveats
* known good domains (where to get domains, which domains to use)
* running experiments

Whatever we leave in the wiki should also not just be the current page minus the removed content. For example, "obtaining and running the planner" would have gaps and no clear focus if we just removed the part on how to build the planner and left everything else. I think this could be improved by splitting the remaining part into more pages (for example, splitting off how to contribute).

In general, I'm not so clear on what the end goal of this issue is. When we move part of the documentation into the repo and leave a part on the wiki, then users still have to access both to use the planner (first read BUILD.md to build the planner then go to the wiki to find out how to run it). Is this intended? The argument that the build instructions are more in sync with the planner would also apply to running the planner wouldn't it?

Maybe one option would be to move even more of the documentation into the repo (for example as multiple markdown files in a "doc" directory)? For example, we could make the split of having user-level documentation in the repo and developer-level documentation in the wiki. We could also extend autodoc to generate more of the wiki automatically from the files in the repo. I'm not sure these are good suggestions, I'm just saying we should consider the different options before we decide on one of them.

> Have we discussed the plan for these? 

The plan was to replace the build instructions in http://www.fast-downward.org/ObtainingAndRunningFastDownward by links to the new build instructions in the repo and keep everything else the same.

Regarding the COIN patch: I think for a temporary file like this the location is fine, but I'm happy to move it somewhere else. Do you have a suggestion? I wouldn't add it to the repo since it's quite big.

I saw that you moved the file that was attached to the page to https://ai.dmi.unibas.ch/_files/fast-downward/coinutils-configure.patched

Is this really the best place to host this? It would make our webserver an additional dependency. I'm not sure what a good place for this is, maybe we should add it to the repo?

I also finished my review. It is a bit hard to review this because changes to the content are mixed with changes of the format, so it's easy to miss a change. It would have been better to do the conversion without any changes first and then do the changes.

As far as I can tell, you removed the following sections:
* pointer to LegacySubversionRepository
* how to use mercurial
* description of our workflow
* how to contribute
* how to install mercurial
* how to clone the repo
* how to run the planner
* caveats
* known good domains (where to get domains, which domains to use)
* running experiments

Have we discussed the plan for these? In particular, I think it would make sense to keep everything related to mercurial (and eventually update them to git) because people can see the readme when the see the project on github. In that situation, they have no clone of the repo yet. 

You also added the content of http://www.fast-downward.org/SettingUpVal and I am not convinced that they belong in the main build instructions.

I left a comment on bitbucket. Other than that, happy with the changes.

Florian, could you have a look at the pull request, please?

Silvan, are you happy with the changes?

Please merge if/when Silvan and Florian are happy.

To make the review easier I only made formatting changes to the LP build instructions. Other than that they're identical to the ones from the wiki.

Thanks for the comments! I think all them make sense and took care of them accordingly. Is there anything else that needs changing?

The diff looks good, but from the discussion, it seems that we also would like to include the LP build instructions there. It also makes sense that from the BUILD.md we don't have to link to the website except for how to run the planner (which one wouldn't expect in a file called BUILD anyway). For this reason, I would also suggest to leave the instructions on how to setup VAL in the BUILD.md file.

>> I assume you would not want to include them in the BUILD.md file for
>> readability?
>
> Yes, that was my concern. We could add a file BUILD-LP.md, though.

If it makes sense to move the regular build instructions to the repository, the same applies to the LP stuff. I think it makes even more sense for the LP instructions than for the "main" ones because they are more closely tied to specific code versions (specific CPLEX/OSI numbers recommended for different code versions).

> Does the patch only convert from MoinMoin to Markdown? Can we view the result somehow?

I also removed the part on obtaining the code and adapted some minor things. Here is the original wiki file converted to Markdown and the new version: https://ai.dmi.unibas.ch/_tmp_files/seipp/build-instructions.tar.gz

> I assume you would not want to include them in the BUILD.md file for readability?

Yes, that was my concern. We could add a file BUILD-LP.md, though.

Does the patch only convert from MoinMoin to Markdown? Can we view the result somehow? (Otherwise, I'm fine with it anyway if it is just a conversion.)

I would also leave the run instructions on the wiki where the automatically generated doc lives. I'm not so sure about the LP-build instructions. I see that they are not really about Fast Downward, but then all of the examples are very Fast Downward-specific. I think I would prefer having all build-related instructions at one place. I assume you would not want to include them in the BUILD.md file for readability?

I made a patch at https://bitbucket.org/jendrikseipp/downward/pull-requests/152

The new BUILD.md file contains only the build instructions. I'd keep the instructions for how to run the planner in the wiki for now. My plan would be to remove the build instructions from the http://www.fast-downward.org/ObtainingAndRunningFastDownward page and add a link to the instructions in the repo instead. I'm not sure whether we should also move the LP build instructions to the repo and if yes, where. What's your take on that?

Feel free to change BUILD.md directly on the pull request page by clicking the three dots on the top right and select "Edit file". Of course, normal comments are also fine.

BTW: For converting MoinMoin syntax to Markdown, I recommend clicking "Export to Docbook" on a wiki page and then using "pandoc --to markdown docbook-file.db"

As discussed recently in a Fast Downward meeting, we want to move the build instructions into the repo to have them in sync with the code. This should make it easier for people downloading our Zenodo archives to build the code.