Title make 64-bit builds the default, then remove option to build in 32-bit mode
Priority feature Status in-progress
Superseder Nosy List atorralba, florian, gabi, guillem, jendrik, malte, silvan
Assigned To jendrik Keywords
Optional summary
Blocked by issue213.

Created on 2017-12-01.10:27:16 by jendrik, last changed by malte.

Blocked by issue213.
msg7429 (view) Author: malte Date: 2018-09-13.12:23:08
Great! Looks good to merge when the other issues are ready.
msg7428 (view) Author: jendrik Date: 2018-09-13.12:16:31
The make-ipc-submission script works now.
msg7426 (view) Author: jendrik Date: 2018-09-13.12:07:03
I'll take care of the comment.
msg7423 (view) Author: malte Date: 2018-09-13.11:52:36
I left one comment on bitbucket. Looks good. :-)
msg7369 (view) Author: gabi Date: 2018-09-11.15:43:21
Silvan and I tested this with our existing setups, in particular builds on
Windows, Mac and Linux. We did not find a 32-bit linux or Mac system. We also
could not test the LP support on Mac OS (not available on the build bot) and on
Windows. For windows, we first need to install a CPLEX version that works with
our current version of Visual studio but this is a separate issue where we will
ask Augusto for help (also upgrading Visual Studio).

For some unclear reason, we had to use slashes in the environment variables for
coin in Windows.

A test run of autodoc ran through smoothly.

From my point of view, this can be merged as soon as the rest of issue213 is

Pull request at
msg7359 (view) Author: florian Date: 2018-09-10.17:44:50
Gabi and I created a pull request. We should wait with merging it until issue213
is merged and there are a couple of other things we need to test and update as
well (wiki, autodoc, buildbot, code tests).

One place where we left the distinction between 32 and 64 bit is in the find
script for CPLEX. The reason for that is that the default installation of CPLEX
has different paths on different systems and we need to know the bitwidth, OS,
and release/debug mode to set the correct hint path. This is not a build option
that the user could set, i.e., we do not support cross compilation, it merely is
a more generic find script that supports native builds on more systems.
msg6680 (view) Author: jendrik Date: 2017-12-01.12:56:11
I completely agree, Malte. Sorry for the confusion.
msg6676 (view) Author: malte Date: 2017-12-01.12:03:27
> that broke the camel's necks.

Or its back, rather. I'm sure most camels only have one neck, and that the neck
is not generally overloaded by straw.
msg6675 (view) Author: malte Date: 2017-12-01.12:00:56
Jendrik, we have long decided that the 32-bit build will go away. It is just a
question of when. Indefinite deprecation is not an option. We are happy to hear
everyone's concerns about this decision and discuss how to address them, but
this is not going to change the decision.

The most critical reason why we made this decision was that 32-bit CPLEX has
been discontinued, and it is not really possible to continue using the old
versions unless you already happen to have a working old version of the code
lying around on some machines and are happy to never touch the software stack of
these machines (or move to newer machines) ever again. This is by far not the
only reason (others include the huge effort needed to support 32-bits especially
with the library dependencies and cross-platform builds and the way they
complicate the build instructions by requiring obscure cross-compiling
libraries), but it was the straw that broke the camel's necks.

I should also point out that the reason we still have a 32-bit build is not
"mainly because of the memory performance", but *only* because of memory. The
64-bit build was already faster than the 32-bit build in 2011, and the numbers
I've seen since then have agreed with this. (However, I haven't seen many
numbers, something we should perhaps address.)

Alvaro, the core data structures within Fast Downward used to heavily rely on
pointers, but we have started to replace them one by one by things that are less
wasteful with memory when compiled in 64-bit mode. More details on this are in
issue213. It's been a large effort (issue213 is now six years old, and we've
been chipping away at it for a good chunk of that time), but we're now mostly there.

One of the early examples of this was the mechanism for storing state data,
which used to be based on individual vectors of state data for each state and is
now based on large contiguous bins of packed integers. Another example of this
are the hash tables used to store the state registry, where we have to stop
using std::unordered_map and roll our own instead.

I hope that memory-efficient BDD libraries for 64-bit builds already exist,
given that 64 bits are now ubiquitous. It's been a while since I last looked
into BDD library implementations, but back then the ones I looked at didn't use
pointers internally anyway and would have been fine in 64-bit builds. If this is
no longer case, it is really up to the people providing these library
implementations to step up and modernize their code. "If you want to use our
library, don't compile a 64-bit executable" is not an acceptable stance any more
these days.

When our own changes to make the code more 64-bit friendly are done and we are
happy with the memory performance, we will remove the 32-bit options from cmake.
The removal will perhaps not happen overnight, but I don't think we will wait
for a long time before doing it. Of course it will be possible for forks of  the
code to keep these options in, and they will continue to work for the
foreseeable future if the LP code is not used.

The only scenario where I can see this decision changed is if someone interested
in championing 32-bit builds step up to take over the support of 32-bit builds.
This mainly entails taking charge of the cmake files, documentation, and 32-bit
build slaves.
msg6673 (view) Author: atorralba Date: 2017-12-01.11:13:54
My two cents on this. In my experience, compiling on 32/64 bits has the most 
impact on the performance of data-structures that heavily rely on pointers 
(e.g. BDDs). If something alike is being heavily used by the planner, and 
experiments are performed with a limit of 4GB or less, compiling in 32-bits is 
highly recommended.

But probably this is not a problem since FastDownward does not use this type of 
msg6671 (view) Author: jendrik Date: 2017-12-01.10:27:16
Building in 32-bit mode has several disadvantages:

- It only allows using up to 4 GiB of memory.
- It complicates our build scripts.
- It is not supported by recent CPLEX versions.

Therefore, we would like to remove or at least deprecate the option to build in 
32-bit mode. Currently, 64-bit builds don't match the performance of 32-bit 
builds, mainly because they use too much memory. Fixing this is issue213. We 
won't change the default build mode until issue213 is resolved. For now, we would 
like to collect possible concerns about deprecating or removing 32-bit builds. If 
somebody still needs 32-bit builds, we will probably only deprecate them and 
change the default to 64-bit. Otherwise, 32-bit builds will probably be removed.
Date User Action Args
2018-09-13 12:23:08maltesetmessages: + msg7429
2018-09-13 12:16:32jendriksetstatus: chatting -> in-progress
messages: + msg7428
2018-09-13 12:07:03jendriksetassignedto: jendrik
messages: + msg7426
2018-09-13 11:52:36maltesetmessages: + msg7423
2018-09-11 15:43:21gabisetmessages: + msg7369
2018-09-10 17:44:50floriansetnosy: + gabi
messages: + msg7359
2017-12-01 12:56:11jendriksetmessages: + msg6680
2017-12-01 12:03:27maltesetmessages: + msg6676
2017-12-01 12:01:29maltesettitle: deprecate or remove option to build in 32-bit mode -> make 64-bit builds the default, then remove option to build in 32-bit mode
2017-12-01 12:00:56maltesetmessages: + msg6675
2017-12-01 11:46:14guillemsetnosy: + guillem
2017-12-01 11:13:54atorralbasetstatus: unread -> chatting
nosy: + atorralba
messages: + msg6673
2017-12-01 10:43:56floriansetnosy: + florian
2017-12-01 10:35:09silvansetnosy: + silvan
2017-12-01 10:27:16jendrikcreate