I think a downward-dev announcement makes sense, and this should also be mentioned in the change log.

The instructions should be up to date now and Silvan just tested them on the
grid and locally.

One problem for users with existing installations of OSI is the error we had on
the buildbot: if they don't have zlib on their system but OSI is installed,
CMake will still try to include zlib and fail. Installing zlib or reinstalling
OSI according to our updated instructions fixes this. Should we send an
announcement to downward-dev about this?

Great! Don't forget to also change the non-Soplex LP install instructions, as these change too with the new zlib dependency, right?

I made the change in the change log and merged. 

The buildbot now fails because OSI is compiled without zlib there. I'll install
it on the build slaves and will try the build again.

I'll also update the documentation in the wiki with instructions how to install
SoPlex.

Left a small comment on bitbucket, otherwise looks good to merge.

I just saw that this issue is still open but I think the last commit handled all
remaining open points. Can I merge this?

By the way: the OSI team responded to the issue and suggested a fix. It will
probably end up in a future version of the interface but our code now works with
both the fixed and the unfixed version so it doesn't matter so much for us.

Thanks for the review. I made the changes you suggested and added a change log
entry.

I left some comments; I tried to understand the 90 new lines of CMake as well as
possible, but I know very little about it, so that isn't very thoroughly reviewed.

This needs an entry to the change log. It should include a brief mention of
relative performance. If we want to get fancy, we can discuss the different
solvers on a wiki page. If not, mentioning the experiments associated with this
issue will do. (This low-tech solution is probably the better one; for one, the
issue won't get changed over time, at most added to.)

Great job, Florian :-)

If someone wants to review the changes before the merge, they are very minor and
can still be found at: 
https://bitbucket.org/FlorianPommerening/downward-issues/pull-requests/40

It could belong to both OSI or SoPlex. If the SoPlex API requires the sense to
be set after loading, then OSI should hide that difference. I reported a bug
with OSI. As far as I know the teams of both projects are both at ZIB and may
even overlap, so they can pass it along if necessary.

The experiments with the fix finished over night and the results look
impressive. SoPlex outperforms CPLEX in terms of coverage (-1 with potential
heuristics, +4 with operator counting, +19 with lmcount). While the results vary
by domain and configuration a lot, it seems like a viable alternative.

https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v3.html
https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v3-scatter-total-time-diverse-potentials.png
https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v3-scatter-total-time-opcount-seq-lmcut.png
https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v3-scatter-total-time-optimal-lmcount.png

I think this is ready to be merged.

Thanks, Florian! This may be worth reporting a bug with the SoPlex people. (Or
with OSI, if OSI is involved.)

We now also have a nice experiment illustrating that maximizing instead of
minimizing potential heuristic values is a good idea. :-)

For the record: I'm attaching the test program I used to analyze this behavior.

I finally figured out what was going on. We set the objective sense (i.e.,
whether we are maximizing or minimizing the LP) before loading the problem. The
CPLEX interface handles this correctly but the SoPlex interface silently ignores
it. This meant that the LPs that we want to maximize (potentials and LM-count)
were minimized instead. As a result, we got a very expensive blind heuristic.

Changing the code, so the objective sense is set after loading the problem
should fix the issue. I started an experiment to confirm. I forgot to set the
silent flag, so the "unexplained errors" will appear in this experiment as well.

Very interesting. It looks like CPLEX and SOPLEX give different heuristic
values, based on "expansions until last jump" and similar statistics. Should
they? Perhaps the potential heuristic configurations might use different
heuristics because these depend on which of several equally good solutions we
get from the LP solver, but the third and I think also the second configuration
should be fully deterministic, shouldn't it?

Is there a way to get rid of these "unexplained errors" in future experiments?

Since we switched to native builds, this issue became a bit simpler. We also now
have a compatible zlib module on the grid, so I could run some tests. The
results are not at all what I expected. It seems like while SoPlex is slower
than CPLEX on average, CPLEX is extremely slow in some cases.
I didn't look into the results in more detail yet, maybe there are other
influences. For example, I'm comparing against CPLEX 12.9 which we have not
evaluated yet (see issue925, experiments are still running).

https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v2.html
https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v2-scatter-total-time-diverse-potentials.png
https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v2-scatter-total-time-opcount-seq-lmcut.png
https://ai.dmi.unibas.ch/_tmp_files/pommeren/issue752-v2-scatter-total-time-optimal-lmcount.png

PS: In the past, it was quite common for us to not be able to compile on the
experiment platform due to version issues. That's why we had static builds in
the first place: to build on our own machines, then execute on the experiment
platform. (I don't know if that still works.)

Using a different compiler version for the library and for the planner is
something that I'd recommend against anyway. It's likely to not work, and
possible to lead to things breaking in very subtle ways that manifest in random
crashes etc.

It's much harder than I thought to get this running on the grid. SoPlex requires
a CMake version that only comes in two modules on the grid and these modules are
only compatible with modules for certain g++ versions. Using a newer compiler
version for the libraries makes them incompatible with the old compiler, so we
cannot use the compiler old compiler for the planner. Finding a compatible zlib
version on the grid is an additional problem. I'll stop working on this for
today but next time I'll try the whole process without zlib. I'll also try to
look into the issue with the newer compiler version.

There was a small error in msg6683: The option is called "--with-soplex-incdir"
not "--with-soplex-inc-dir".


For the record, here is how I installed osi with cplex/soplex support on the
grid it required a few extra hoops to jump through with loading the correct
modules and configuring OSI to work with them:

module purge
module load GCC/5.4.0-2.26.lua
module load CMake/3.4.3-foss-2016b.lua 
module load GMP/6.1.0-intel-2017.01.lua

# set in .zshrc
export DOWNWARD_COIN_ROOT64=~/local/opt/coin-0.107.9-64-cplex1251-soplex301
export
DOWNWARD_CPLEX_ROOT64=~/local/opt/ibm/ILOG/CPLEX_Enterprise_Server1251/CPLEX_Studio/cplex
export DOWNWARD_SOPLEX_ROOT64=~/local/opt/soplex-3.0.1-64

tar xvf soplex-3.0.1.tgz
cd soplex-3.0.1
mkdir build64
cd build64
cmake -DCMAKE_INSTALL_PREFIX="$DOWNWARD_SOPLEX_ROOT64" -DGMP_DIR="$EBROOTGMP" ..
make -j20
make install
cd ../..

wget http://www.coin-or.org/download/source/Osi/Osi-0.107.9.tgz
tar xvf Osi-0.107.9.tgz
cd Osi-0.107.9
./configure CC="gcc"  CFLAGS="-m64 -pthread -Wno-long-long" \
            CXX="g++" \
            CXXFLAGS="-m64 -pthread -Wno-long-long \
                      -DTHREADLOCAL='' \
                      -D__STDC_CONSTANT_MACROS \
                      -D__STDC_LIMIT_MACROS" \
            LDFLAGS="-L$DOWNWARD_CPLEX_ROOT64/lib/x86-64_sles10_4.1/static_pic \
                     -L$DOWNWARD_SOPLEX_ROOT64/lib" \
            --without-lapack --enable-static=yes \
            --prefix="$DOWNWARD_COIN_ROOT64" \
            --with-cplex-incdir=$DOWNWARD_CPLEX_ROOT64/include/ilcplex \
            --with-cplex-lib="-lcplex -lm" \
            --with-soplex-incdir=$DOWNWARD_SOPLEX_ROOT64/include \
            --with-soplex-lib=-lsoplex 
make -j20
make install

# Repeat OSI installation with these variables for CPLEX 12.7.1 (change
x86-64_sles10_4.1 to x86-64_linux)
export DOWNWARD_COIN_ROOT64=~/local/opt/coin-0.107.9-64-cplex1271-soplex301
export DOWNWARD_CPLEX_ROOT64=~/local/opt/cplex-12.7.1-64/cplex

> Sounds good. I think the zlib issue is a question of whether it is easier to
> document how to compile OSI and SoPlex without zlib or to document how to
> install zlib so that CMake finds it.

Agreed. Given that zlib is a very common library, I would expect the latter to
be easier.

> Looks like this is working. I prepared a pull request for it:
>
https://bitbucket.org/FlorianPommerening/downward-issues/pull-requests/40/issue752/diff

The patch looks good to me.

If you think it is useful, I can try this out too, but I suggest doing this
after it has been merged and our wiki has been updated. That way we also get to
test the documentation. In that case, please remind me that I wanted to do this
once the documentation is ready.

> What kind of experiments should we run with this? Is A*/h^seq on the optimal
> benchmarks enough?

You know best which kinds of configurations exercise the LP code.

Looks like this is working. I prepared a pull request for it:
https://bitbucket.org/FlorianPommerening/downward-issues/pull-requests/40/issue752/diff

I can currently compile it on my laptop with {release, debug} x {32, 64} as a
static build. I cannot try a dynamic build because I cannot compile SoPlex as a
shared library (but that also doesn't work for CPLEX, so I think we can ignore
it). I didn't try it on Mac or Windows yet, but I can give that a try tomorrow.

What kind of experiments should we run with this? Is A*/h^seq on the optimal
benchmarks enough?
I ran one blocks world instance for testing and the results was closer to CPLEX
than I expected (for the 2260950 evaluations CPLEX took 158.172s and SoPlex
205.312s).

Sounds good. I think the zlib issue is a question of whether it is easier to
document how to compile OSI and SoPlex without zlib or to document how to
install zlib so that CMake finds it. I think both are OK but I tend towards the
second option currently (I'll have a clearer picture when this issue is closer
to completion).

Also just to keep track of what I did:

# Download and extract SoPlex 3.0.1 code
# Compile 64 bit SoPlex:
mkdir build64
cd build64
cmake -DCMAKE_INSTALL_PREFIX="/opt/soplex-3.0.1-64" ..
make
sudo make install
cd ..

# Compile 32 bit SoPlex:
sudo apt install zlib1g-dev:i386
sudo apt install libgmp-dev:i386 (important for the performance of SoPlex but
should work without)
mkdir build32
cd build32
cmake -DCMAKE_INSTALL_PREFIX="/opt/soplex-3.0.1-32" ..
make
sudo make install

# Set up environment variables in bashrc (still had similar variables for CPLEX
from earlier)
export DOWNWARD_SOPLEX_ROOT32=/opt/soplex-3.0.1-32
export DOWNWARD_SOPLEX_ROOT64=/opt/soplex-3.0.1-64

# Download and extract OSI 0.107.9
# Compile 64 bit version of OSI
./configure CC="gcc"  CFLAGS="-m64 -pthread -Wno-long-long" \
            CXX="g++" CXXFLAGS="-m64 -pthread -Wno-long-long" \
            LDFLAGS="-L$DOWNWARD_CPLEX_ROOT64/lib/x86-64_linux/static_pic \
                     -L$DOWNWARD_SOPLEX_ROOT64/lib" \
            --without-lapack --enable-static=yes \
            --prefix="/opt/osi-0.107.9-cplex1263-soplex301-64" \
            --with-cplex-incdir=$DOWNWARD_CPLEX_ROOT64/include/ilcplex \
            --with-cplex-lib="-lcplex -lm" \
            --with-soplex-inc-dir=/opt/soplex-3.0.1-64/include \
            --with-soplex-lib="-lsoplex" 
make
sudo make install

# Compile 32 bit version of OSI
make distclean
./configure CC="gcc"  CFLAGS="-m32 -pthread -Wno-long-long" \
            CXX="g++" CXXFLAGS="-m32 -pthread -Wno-long-long" \
            LDFLAGS="-L$DOWNWARD_CPLEX_ROOT32/lib/x86_linux/static_pic \
                     -L$DOWNWARD_SOPLEX_ROOT32/lib" \
            --without-lapack --enable-static=yes \
            --prefix="/opt/osi-0.107.9-cplex1263-soplex301-32" \
            --with-cplex-incdir=$DOWNWARD_CPLEX_ROOT32/include/ilcplex \
            --with-cplex-lib="-lcplex -lm" \
            --with-soplex-inc-dir=/opt/soplex-3.0.1-32/include \
            --with-soplex-lib="-lsoplex" 
make
sudo make install

# Set up environment variables in bashrc:
export DOWNWARD_COIN_ROOT32=/opt/osi-0.107.9-cplex1263-soplex301-32
export DOWNWARD_COIN_ROOT64=/opt/osi-0.107.9-cplex1263-soplex301-64

I don't think we were planning to use zlib for compressed planner input. I think
for us it makes much more sense to handle compression at the driver layer. But
of course we can have zlib as a dependency for some subset of CPLEX/OSI/SoPlex
if that makes our life easier.

I wouldn't worry too much about 32-bit support at this stage. If we integrate
SoPlex only for 64-bit builts, that is fine.

The installation of SoPlex that comes with SCIP has zlib support. For CPLEX and
OSI this is also the default but so far our build instructions explicitly
disable this. In SoPlex the zlib support can also be disabled but that would
mean a more complex installation of SoPlex. 

Since we already talked about accepting compressed input to the planner I
thought it might be interesting to see how complicated it is to link the planner
with zlib so I changed the CMake files so they now look for zlib if an LP solver
is used. This change would mean that we can remove the "--disable-zlib" line
from the LP build instructions.

Linking with zlib was no problem on a 64bit build but it was getting complicated
for the 32bit version. On Ubuntu several packages provide a 32bit version of the
library in different locations. The (only) one that worked for me was
zlib1g-dev:i386. For the other packages, CMake didn't find the library, for
example because there was only a libz.so.1 symlink and none called libz.so.

On the non-CMake front of this issue, I added options to use the solver to our
interface and everything went relatively smooth. Currently, the solver still
outputs log messages for every solved LP that I'll try to disable next.

I'll also try to compile SoPlex from scratch in 32 and 64 bit to see if both
compiles work. Although, if the 32bit build doesn't work, I'd suggest to treat
it as unsupported since we want to get rid of it anyway.

Sounds great.

SoPlex (http://soplex.zib.de/) is a fast open-source free-for-academic-use LP
solver included in SCIP. It should work behind our existing interface through
OSI. I'd like to add support for it here.