Strudel2

Strudel2 is an interactive environment to develop supports two types of jobs. You will be able to reach it from https://mlerp.cloud.edu.au/.

It will allow you check the cluster’s usage and your disk quota. You will also be able to launch interactive jobs on the cluster. Strudel2 offers two kinds of interactive jobs: terminal and Jupyter Lab.

Terminal

Terminal jobs give you some requested compute attached to a simple text UI. This is ideal if you need to clone repositories, create conda environments, organise your files or anything else that you can do with a linux shell that doesn’t require a GUI.

You can also launch the web terminal attached to the login node if you just need a quick terminal to manage your files, however this is a shared resource with limited compute.

Jupyter Lab

Jupyter Lab jobs give you some requested compute attached to a Jupyter Lab IDE. This is ideal if you prefer experimenting in python notebooks or visualising data with a GUI.

If you prefer using VS Code to the Jupyter Lab IDE you can also connect to this job with the Remote Development Extension Pack. For more details see our page on connecting via VS Code.

You can use this app will using one of our provided conda environments, or one that you provide. If you’re looking for an envionment that’s not listed here, check that its path is included in the ~/.conda/environments.txt file in your home directory.

We have provided the DSKS (Data Science Kitchen Sink) series of environments for you to explore the platform and get started right away. Each version of the environment is tagged with the month that it was compiled, so that as we update and add new packages you can continue to rely on older versions of the environment for reproducability in your experiments.

If our conda environments don’t meet your needs, you can consider maintaining your own miniconda or mambaforge installation in your home directory. For a custom conda environment to be compatible with this app, you will need to include jupyter and dask related packages during. See this page for more information about creating a conda environment to interface with our environment.

Batch

The Batch app gives you the ability to submit jobs to be picked up by the SLURM queue. This allows you to continue to run your tests while you are away from the computer and check back in later when they’re done. The app currently supports python scripts as well as bash scripts.

Of course if you’re comfortable with manually submitting to the queue you can do so if you need the extra control. Batch jobs are run in the Lion QoS meaning you can have up to 4 running at once.

Any outputs to terminal will be captured and stored in the job’s log file for later viewing. Be sure to checkpoint your code and periodically save any trained models so that you can reload your desired model from any chosen point during training process. Checkpointing also enables you to train your model for longer than the 24 hour wall time, as long as you save everything you need to resume at the end of your job.

VS Code Server

Code Server offers a simpler alternative to connecting your IDE to MLeRP over SSH. Just as you can customise your VS Code experience with the desktop app using plugins, you can install VS Code plugins while using Code Server. See the Code Server documentation for more information.

FAQ

Why can’t I connect to Jupyter Lab?

Jupyter Lab needs to be able to create files to run properly. If your disk quota is completely full, it will create empty files that will confuse the system. To fix this, ssh into the cluster or use a terminal job to reduce your disk usage back under quota, then delete the empty files which will be found at /home/<USERNAME>/.local/share/jupyter/runtime.

There is also a known bug where Brave’s ‘shield’ feature will block the Strudel2 Jupyter application. To bypass this, you’ll need to turn off ‘shields’ for both the Strudel2 home page and the Strudel2 api.

Why is my VS Code not attaching to the remote session?

The SSH certificates that SSOSSH creates are only valid for 24 hours, this means that you’ll need to generate new certificates each day before you connect. VS Code also has a known bug where it can get confused if you have another SSH client (such as MobaXTerm) open when it tries to establish the connection.