Issue940

Title Adapt bitbucket pipeline to git(hub)
Priority feature Status in-progress
Superseder Nosy List augusto, cedric, florian, jendrik, malte, patfer, silvan
Assigned To silvan Keywords infrastructure
Optional summary
Current status:
- ubuntu actions run for different compilers, python versions, and with CPLEX and SOPLEX
  TODO: how to setup nightly and weekly tests? what about regression tests (they might need to store/access additional data)?
- MacOS: actions run the tests currently run by the buildbot: exit codes and standard configs
  optional TODO: compile with LP solvers; setup valgrind to simply run "all code tests"
- Windows: compilation without LP solvers is possible
  TODO: test the planner; compile planner with LP solvers

General TODOs:
- Add CONTRIBUTING.md file to repo that explains which tests are run and how to run the tests locally.

related issues:
issue970 Find a way to run SoPlex during Github actions
issue971 Find a way to run CPLEX during Github actions
issue972 Find a way to run Gurobi during Github actions

Pull request: https://github.com/aibasel/downward/pull/5

Created on 2019-12-05.17:22:03 by patfer, last changed by jendrik.

Summary
Current status:
- ubuntu actions run for different compilers, python versions, and with CPLEX and SOPLEX
  TODO: how to setup nightly and weekly tests? what about regression tests (they might need to store/access additional data)?
- MacOS: actions run the tests currently run by the buildbot: exit codes and standard configs
  optional TODO: compile with LP solvers; setup valgrind to simply run "all code tests"
- Windows: compilation without LP solvers is possible
  TODO: test the planner; compile planner with LP solvers

General TODOs:
- Add CONTRIBUTING.md file to repo that explains which tests are run and how to run the tests locally.

related issues:
issue970 Find a way to run SoPlex during Github actions
issue971 Find a way to run CPLEX during Github actions
issue972 Find a way to run Gurobi during Github actions

Pull request: https://github.com/aibasel/downward/pull/5
Files
File name Uploaded Type Edit Remove
ubuntu.yml patfer, 2019-12-05.17:23:49 application/x-yaml
Messages
msg9732 (view) Author: jendrik Date: 2020-10-11.19:30:34
I just stumbled over a program that allows to run GitHub Actions locally, which could be very useful for working on this issue and debugging GitHub Actions quickly: https://github.com/nektos/act
msg9731 (view) Author: jendrik Date: 2020-10-02.00:01:48
Add note about CONTRIBUTING.md to summary.
msg9730 (view) Author: jendrik Date: 2020-10-01.23:52:53
We discussed this live today and decided that we want to integrate the GitHub Actions more or less as they're currently implemented. They're not meant to replace the Buildbot, at least for now. Instead the goal is to let them highlight style and compilation problems to contributors early in the reviewing process and let reviewers focus on the design of the code.

We agreed that we don't want to test all combinations of compilers and Python versions on all platforms. Instead, we want to test at most the oldest and newest supported versions of the C++ compilers and Python versions. Which versions to test on which platform is still up for debate and also depends on how easy they can be set up, I think.
msg9596 (view) Author: silvan Date: 2020-07-13.10:03:35
I added them.
msg9593 (view) Author: malte Date: 2020-07-11.12:36:47
> If I'm not overlooking anything, the main remaining unknown for this issue is
> whether we can compile the planner with CPLEX on Windows automatically (i.e.,
> without clicking through installers).  I added a TODO for the corresponding
> subissue.

Silvan's msg9097 lists some other TODOs. If they are not yet done, can they be merged with the TODOs in the summary?

No need to mark Mac OS things that the buildbot currently doesn't do as TODOs, but it would be nice to keep track of them in the summary as optional future tasks (but not blockers for moving to github actions).
msg9588 (view) Author: jendrik Date: 2020-07-10.19:22:02
Is anyone (with some Windows expertise) interested in looking into compiling the planner with CPLEX on Windows for Github Actions?
msg9587 (view) Author: jendrik Date: 2020-07-10.19:17:38
If I'm not overlooking anything, the main remaining unknown for this issue is whether we can compile the planner with CPLEX on Windows automatically (i.e., without clicking through installers).  I added a TODO for the corresponding subissue.
msg9584 (view) Author: silvan Date: 2020-07-10.18:09:10
Update pull request.
msg9443 (view) Author: jendrik Date: 2020-07-03.15:19:08
Thanks, I forwarded the relevant bits to the ZIB guys.
msg9441 (view) Author: silvan Date: 2020-07-03.14:42:56
@Jendrik, the soplex action works, including redacting all output related to URLs. The code contains lots of duplication and should really be considered as a prototype.
msg9438 (view) Author: florian Date: 2020-07-03.13:48:09
> Is that visible to you, too? 

Yes it is. Also, for the final location of the installer, we should remember to pick a place that sounds less "temporary".
msg9429 (view) Author: silvan Date: 2020-07-03.11:11:07
One problem I just stumbled across is that if using the secret URL with wget, the actions fails with an error message giving away the URL. Is that visible to you, too? 
https://github.com/silvansievers/issue490/runs/833765241?check_suite_focus=true

(Let me know if you need to be added to the repo.)
msg9416 (view) Author: silvan Date: 2020-07-02.18:25:53
Update repo for pull request (with latest default).
msg9098 (view) Author: malte Date: 2019-12-13.11:35:13
I see no immediate issues with the CPLEX license that would prevent us from using a cloud-based buildbot replacement as long as access is limited to members of our research group. But I suggest you speak to Florian (who is not on this issue) for a second opinion and also look into this yourself.

Some relevant references:
https://my15.digitalexperience.ibm.com/b73a5759-c6a6-4033-ab6b-d9d4f9a6d65b/dxsites/151914d1-03d2-48fe-97d9-d21166848e65/faqs/agreement
https://my15.digitalexperience.ibm.com/b73a5759-c6a6-4033-ab6b-d9d4f9a6d65b/dxsites/151914d1-03d2-48fe-97d9-d21166848e65/faqs/guidelines
https://my15.digitalexperience.ibm.com/b73a5759-c6a6-4033-ab6b-d9d4f9a6d65b/dxsites/151914d1-03d2-48fe-97d9-d21166848e65/faqs/faqs
msg9097 (view) Author: silvan Date: 2019-12-13.11:25:25
Current status of github actions:
- ubuntu: tests run for different compilers, python versions, including CPLEX
  TODOs: 
  - find out if and how we can legally run CPLEX.
  - nightly and weekly tests
- mac: those tests run by the buildbot run: exit codes and standard configs
  TODOs: CPLEX? valgrind for all code tests?
- windows: compilation runs (without CPLEX)
  TODOs: CPLEX? run some tests?
msg9091 (view) Author: cedric Date: 2019-12-11.13:16:34
> Cedric, are you still working on the Windows yaml file? I saw that compilation works, but the planner is not yet tested. Also, it would be great if you could look into adding CPLEX also for the Windows builds.

Not today. I can look into it tomorrow.
msg9090 (view) Author: jendrik Date: 2019-12-11.12:43:58
Building with different Python versions also has its benefits, because it tests build.py with different versions. However, I agree that we should try to only build the planner for each revision once (or twice if we count the debug build). It seems this is possible by using artifacts:

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/persisting-workflow-data-using-artifacts
https://stackoverflow.com/a/57877438/1277298
msg9089 (view) Author: silvan Date: 2019-12-11.12:31:48
Patrick and I discussed this a bit further and I'll try to come up with a solution using matrix builds. We currently don't see an easy way of avoiding to compile the planner again for all different python versions.

Cedric, are you still working on the Windows yaml file? I saw that compilation works, but the planner is not yet tested. Also, it would be great if you could look into adding CPLEX also for the Windows builds.
msg9088 (view) Author: jendrik Date: 2019-12-11.11:11:07
There are multiple ways how to select which tox environments are run (https://tox.readthedocs.io/en/latest/config.html#conf-envlist). If you use a matrix setup, it's better to select only a single Python version. If the matrix build only provides a single Python version, you can use "tox -e py,style". Otherwise, you can probably set TOXENV=py36,style for the matrix build for Python 3.6.
msg9087 (view) Author: patfer Date: 2019-12-11.11:10:38
What are each scenario (nighly, weekly?)
If the tests differ only in the python version used, I would prefer to have only
a single configuration file (if we run different tests in different scenarios,
those can be in different configuration files).
About the issue that only one python version succeeds, does specifying to tox to
run only a certain python version help? (e.g. tox -e p36 if your matrix is in
the step for python 3.6)?
msg9086 (view) Author: silvan Date: 2019-12-11.10:57:40
The ubuntu setup also works (tested gcc, but not clang, python 3.6 and 3.7, and with CPLEX). I think we should now discuss how to test the different scenarios. In the current version, I used a matrix build to test different python versions, but then the generic test "tox" will fail on each build because only one python version succeeds. Should we address this with the already planned task of having separate tox configs for each scenario? This would however mean that we need to do the matrix setup there, too...
msg9082 (view) Author: jendrik Date: 2019-12-10.15:11:59
I left some comments there.
msg9081 (view) Author: silvan Date: 2019-12-10.14:56:09
The MacOS tests now run successfully.
msg9080 (view) Author: jendrik Date: 2019-12-09.18:36:18
Inspecting your code more closely, the problem is that the code needs to call "pytest" instead of "python". Nonetheless, I recommend to use the official way of obtaining Python 3.8.
msg9079 (view) Author: jendrik Date: 2019-12-09.18:29:12
It seems that using homebrew is not the recommended way of installing Python versions for Github Actions. In case you haven't tried that already, 

here is a list of available versions:
https://help.github.com/en/actions/automating-your-workflow-with-github-actions/software-installed-on-github-hosted-runners#python-2

and here is how to install them: https://github.com/marketplace/actions/setup-python-environment

I'd choose Python 3.8 since this is the version that people will most likely want to install on their machines.
msg9078 (view) Author: silvan Date: 2019-12-09.17:53:56
I tried to get the MacOS Yaml file done, but failed because "python3.7 <filename>" just doesn't do anything.

I tried to give access to aibasel, but this didn't work, so I only invited Jendrik and Patrick as "collaborators" so far. Any ideas on why this does not work are appreciated. Otherwise, we could also first try to have separate tox configurations, because running "tox" worked when I tried it.
msg9068 (view) Author: malte Date: 2019-12-05.17:23:57
Hi all, just include me in the nosy list for all issues. (Same for Jendrik, but he was already included.)
msg9067 (view) Author: patfer Date: 2019-12-05.17:23:49
Attached is the current version of the yml file for the GitHub Actions (for
testing on ubuntu)
History
Date User Action Args
2020-10-11 19:30:35jendriksetmessages: + msg9732
2020-10-02 00:01:48jendriksetmessages: + msg9731
summary: Current status: - ubuntu actions run for different compilers, python versions, and with CPLEX and SOPLEX TODO: how to setup nightly and weekly tests? what about regression tests (they might need to store/access additional data)? - MacOS: actions run the tests currently run by the buildbot: exit codes and standard configs optional TODO: compile with LP solvers; setup valgrind to simply run "all code tests" - Windows: compilation without LP solvers is possible TODO: test the planner; compile planner with LP solvers related issues: issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/aibasel/downward/pull/5 -> Current status: - ubuntu actions run for different compilers, python versions, and with CPLEX and SOPLEX TODO: how to setup nightly and weekly tests? what about regression tests (they might need to store/access additional data)? - MacOS: actions run the tests currently run by the buildbot: exit codes and standard configs optional TODO: compile with LP solvers; setup valgrind to simply run "all code tests" - Windows: compilation without LP solvers is possible TODO: test the planner; compile planner with LP solvers General TODOs: - Add CONTRIBUTING.md file to repo that explains which tests are run and how to run the tests locally. related issues: issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/aibasel/downward/pull/5
2020-10-01 23:52:53jendriksetmessages: + msg9730
2020-07-13 10:03:36silvansetmessages: + msg9596
summary: TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/aibasel/downward/pull/5 -> Current status: - ubuntu actions run for different compilers, python versions, and with CPLEX and SOPLEX TODO: how to setup nightly and weekly tests? what about regression tests (they might need to store/access additional data)? - MacOS: actions run the tests currently run by the buildbot: exit codes and standard configs optional TODO: compile with LP solvers; setup valgrind to simply run "all code tests" - Windows: compilation without LP solvers is possible TODO: test the planner; compile planner with LP solvers related issues: issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/aibasel/downward/pull/5
2020-07-11 12:36:47maltesetmessages: + msg9593
2020-07-10 19:22:02jendriksetmessages: + msg9588
2020-07-10 19:17:38jendriksetmessages: + msg9587
2020-07-10 18:09:10silvansetmessages: + msg9584
summary: TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/silvansievers/issue490/pull/1 -> TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/aibasel/downward/pull/5
2020-07-03 15:19:08jendriksetmessages: + msg9443
2020-07-03 14:42:56silvansetmessages: + msg9441
2020-07-03 13:48:09floriansetmessages: + msg9438
2020-07-03 11:11:07silvansetmessages: + msg9429
2020-07-02 18:25:54silvansetmessages: + msg9416
summary: TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/silvansievers/git-downward/pull/1 -> TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/silvansievers/issue490/pull/1
2020-07-02 17:20:48floriansetkeyword: + infrastructure
2020-07-02 16:52:43silvansetassignedto: patfer -> silvan
summary: TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions -> TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions Pull request: https://github.com/silvansievers/git-downward/pull/1
2020-07-01 12:44:04floriansetnosy: + florian
2020-07-01 12:43:46floriansetsummary: TODO issue970 Find a way to run SoPlex during Github actions issue971 Find a way to run CPLEX during Github actions issue972 Find a way to run Gurobi during Github actions
2019-12-13 11:35:14maltesetmessages: + msg9098
2019-12-13 11:25:25silvansetmessages: + msg9097
2019-12-11 13:16:34cedricsetmessages: + msg9091
2019-12-11 12:43:58jendriksetmessages: + msg9090
2019-12-11 12:31:48silvansetmessages: + msg9089
2019-12-11 11:11:07jendriksetmessages: + msg9088
2019-12-11 11:10:38patfersetmessages: + msg9087
2019-12-11 10:57:40silvansetmessages: + msg9086
2019-12-10 15:11:59jendriksetmessages: + msg9082
2019-12-10 14:56:09silvansetstatus: chatting -> in-progress
messages: + msg9081
2019-12-09 18:36:18jendriksetmessages: + msg9080
2019-12-09 18:29:12jendriksetmessages: + msg9079
2019-12-09 17:53:56silvansetstatus: unread -> chatting
messages: + msg9078
2019-12-06 08:12:08cedricsetnosy: + cedric
2019-12-05 17:31:32silvansetnosy: + silvan
2019-12-05 17:23:57maltesetstatus: chatting -> unread
nosy: + malte
messages: + msg9068
2019-12-05 17:23:49patfersetfiles: + ubuntu.yml
status: unread -> chatting
messages: + msg9067
2019-12-05 17:22:03patfercreate