We have introduced a new app to Strudel, Open Web UI. In the words of the developer: > Open WebUI is an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline.

What this means for MLeRP users is that we have configured this app to talk to the Ollama server on mlerp-monash-node13, allowing you to chat with the LLMs we have hosted there. All while keeping your chat data contained within MLeRP, rather than sending your data to Open AI or some other provider. The app supports uploading documents for RAG as well as internet search and a code compiler.

We hope that this will allow you to experiment with the models and their capabilities with greater usability than our previous Ollama app, which we will now be retiring, since it’s functionality is similar - albeit with a text UI. Note that the deprecation of the Ollama Strudel app, does not prevent you from using the Ollama server through the ollama-python or through Continue - this functionality will still be supported.

Regards, Mitchell Hargreaves

MLeRP has officially crossed 200 users!

As you may have noticed, MLeRP has just undergone a major upgrade, increasing the number of compute nodes from 6 to 14! This has allowed us to double the size of the Tabby partition, and for the rest - we’re leaving as whole GPU nodes to assist with Batch processing. We will monitor usage of the cluster and redistribute the nodes if we feel we need to at a later date.

As an experiment, we’re setting aside one of these new nodes, mlerp-monash-node13, to run an Ollama server. You can reach this api from any other MLeRP node on port 80. You can use this API to generate LLM responses by calling the endpoints it directly or interact with it through the ollama-python library.

Please note that we do not enable the full Ollama API but rather only allow the following endpoints:

• /api/generate
• /api/chat
• /api/tags
• /api/show
• /api/embed
• /api/version

Ollama can also be integrated with your VSCode workflow through AI assistant plugins like Continue. This allows for AI assisted code generation and code autocomplete without paying for access or sending research data to online services like ChatGPT and Claude.

For more information on Ollama, please have a look at our Ollama documentation.

Regards, Mitchell Hargreaves

ML4AU is planning a series of webinars for AI Month (14 Oct - 15 Nov). As a part of this, we would love to hear from some of you MLeRP Users! If you’re doing some exciting ML or AI work on our platform and would like to share it with the community, please reach out to us.

Our first major change is that we have updated our old Terminal app. The new Terminal runs on xterm rather than hterm which hasn’t been updated recently. It also gives us some more flexibility in that we can we can pass it commands to run other apps - which we have taken advantage of to run our ‘View Log’ feature.

We’ve also used it to develop our new Ollama app, which you may have noticed crop up in your side bar. If you’re interested in playing around with large language models, but aren’t sure where to start, this could be a good way to experiment. For now we’ve preloaded the phi3, mistral, mixtral and two variants of llama3, but more models are available on request. Opening the app will give you a chat window with your model of choice for you to interact with directly.

However, Ollama also supports interaction through a python API. We’ve put together a tutorial showcasing this which involves using a model to summarise Wikipedia articles. If you’re a more advanced user and want to experiment with Ollama more directly, you will be should be able to run the program directly which is installed at /apps/ollama/ollama. Note that doing so will use your home directory to store your model repository, but this can be reconfigured if you need to reroute it to use a project directory.

As always, we hope these changes will improve your experience, please let us know if you find any issues or ways that you think we can improve.

Regards,
Mitchell Hargreaves

The most important change that we’ve making is that we’re revising our policy on GPU reservations. After many requests we are now allowing 20GB Reservations through the Tabby QoS. Why now? The short answer is we realised that there are some packages, such some hugging face libraries, which are built in a way that makes Dask offloading very difficult. As a result we’re now repartitioning the HouseCats nodes into a total of 8 10GB slices and 4 20GB slices. We’ve also used this as an opportunity to rethink how we’d like to position the BigCats nodes, so now we are devoting 2 nodes to whole 40 GB GPUs for batch work, bringing us up to 4 whole GPUs in the pool.

The second most important change that we’ve made is that we’ve updated our version of SLURM. This is important because it patches some errant behaviour we’ve caught on some GPU interactions which would allow some users to use unallocated GPUs. If you were previously using this exploit, please make sure that you request a GPU reservation if you need to use one, or use Dask to offload your work to the SLURM queue. Similarly, if you were a user that needed to manually set the CUDA_VISIBLE_DEVICES environment variable, please note that you likely won’t need to anymore and doing so could negatively impact your job.

To acknowledge our most visible change, in the inbetween months you may have noticed a new app appear, VS Code Server. This app acts as yet one more way for users to log in to cluster. It gives users the ability to run an instance of Code Server on MLeRP and connect through their web browser without the need to manage an SSH connection or connect with their own VS Code instance.

To facilitate another common request that we’ve had, if you’d like to work with R as a part of your data science workflow, now you can do that on MLeRP too. While we don’t support R Studio as a part of Strudel2, VS Code has a robust R plugin ecosystem that can be used as your R development environment. We recommend maintaining your R virtual environment using the renv package so that you can ensure isolation, portability and reproducibility in your environments.

We’ve also been hard at work in refining our API to improve the web experience of Strudel2. Many of these changes are invisible to you all, but this change allows us to specify a tunnelid for each app, taking the guesswork out of which app to connect to. This should patch the bug where Strudel2 connects to the wrong log or app if you try and open multiple apps too close together. We hope that this change will make apps like the batch app much more usable.

As always, we hope these changes will improve your experience, please let us know if you find any issues or ways that you think we can improve.

Regards,
Mitchell Hargreaves

Since we last had an update we’ve been hard at work refactoring some of the software we use to maintain cluster to prepare it for any growth to come.

Our main focus has been on the way that our provisioning code handles quotas. We had some reports from users with group allocations that the code used to meter disk usage wasn’t behaving as expected. To resolve this, we have moved away from the ‘user’ and ‘group’ quota system that we used to have and implemented ‘project’ quotas.

In this new implementation, ‘user’ and ‘group’ allocations are both metered through the disk usage in their allocated directory, rather than attempting to count usage based on file ownership. We hope that this should be less confusing and address the reports we’ve been getting of some files being counted towards multiple allocations. As before, you will still be able to check the disk usage of all your allocations from your ‘Account Info’ app. If you continue to experience anomalies with your allocated quota, please let us know so we can work through things with you.

Speaking of the ‘Account Info’ page, we’ve added a new feature there that will allow you to download your SSH credentials. This will generate and download an SSH key as well as an SSH certificate that you can use to connect to our cluster using any tool of your choice. These credentials have a limited lifespan and will be cycled periodically, requiring you to log back in and redownload them to refresh your access. The main way we expect users to use this feature is to enable Remote Development through IDEs like VS Code. Previously, the only way to do this was through using our commandline tool, SSOSSH, or to generate your own keys using our terminal app. We hope that this will help remove friction associated with using your favourite tools with our service. If this experimental feature proves stable and useful, we’ll update our documentation to recommend this as the primary method to remotely access our services.

We’ve started a rework of our documentation to address some of the common stumbling blocks we’ve noticed users regularly asking us about. This will include new tutorials covering things like ‘checkpointing’, and refining our existing ones of Dask usage which some have found to be counter intuitive. If there’s a specific section of the documentation that you believe needs attention and you have ideas on how we can clarify it, please let us know.

Our first update to the docs has been to rewrite our outdated Hardware page (formerly the Compute page) to better reflect the current architecture and usage advice. The biggest change you might notice is a description of each QoS that we offer, how they differ, and what they’re each optimised for… which includes the addition of our new QoS Panther! The Panther QoS is restricted to CPU only jobs, which lets us be much more lenient with the wall time (7 days!). This makes it the perfect choice as the host for a jupyter notebook that sends larger workloads out to Dask workers. This lets us reserve the Lion QoS for big workloads such as memory intensive data visualisation or batch processing.

Which leads us into our new experimental Strudel2 app - the Batch Job app! This application, which will be backed by the Lion QoS, will allow you to queue scripts that you’ve delveoped to be picked up by the SLURM scheduler without ever having to use the sbatch command. The application will allow you to select any compute requirements from a web form, such as number of CPUs, RAM, GPU number and flavour, and run a script from a path that you specify. At present we support python and bash scripts, but in the future we also hope to support automatically converting the ipython notebooks you’ve been working on in our Jupyter Lab app through nbconvert. We’d also like to support a method for our power users to paste their SBATCH scripts into a text box in the form for full control. Once we’ve solidified the app’s functionality and ironed out any bugs, we will finalise its documentation along with the other documentation work we’re doing.

To support this app, we thought it was important to be able to view the results of your jobs from the Strudel2 web application. We’ve now added a View log button to all job records, running or completed. This feature will be available for all Strudel2 applications, not just the Batch Job app. To keep things a little tidier, Strudel2 logs will now be placed in a dedicated folder rather than filling up your home directory: ~/.strudel2/logs/. Note that this will not apply to any logs generated by Dask jobs since they are not spawned by Strudel2 - if you’d like to control where these land you can specify with the log_directory paramater.

Another big change is that we’ve now deprecated the MLeRP-QCIF region, leaving only the MLeRP-Monash region which will serve all users going forward. If you are an inactive MLeRP-QCIF user, we have taken a copy of the data which is now hosted in the Monash region. If you’d like to revisit the MLeRP service now that it is more mature, we can make your old qcif data available to you as a part of the provisioning process on request.

Finally, the last change that we want to tell you about is an administrative one. Going forwards, we’re going to be changing our help email to mlerphelp@monash.edu. This move away from our Gmail address is largely to help deal with some issues we’ve had with our user communications, such as account provision notifications, being caught by university spam filters. Any mail that is sent to our old address should be forwarded on to the new address, so we shouldn’t miss anything in the transition.

As always, we hope these changes will make the experience on our platform a smoother one as we continue to develop features to enable your research.

Regards,
Mitchell Hargreaves

In the weeks since launching the platform’s open beta, we have provisioned over 60 additional individual accounts for the system, bringing our total to almost 80 users which has certainly kept us busy! We have also provisioned 7 group allocations ranging in size from 2-6 members. As always, if you’d like to ammend your allocation’s quota or would like to consider a group allocation to collaborate with others, talk to us and we’ll see what we can do for you.

Speaking of the launch, if you missed our launch webinar, the event was recorded and is available here. In it we discussed the guiding principles that we used when designing MLeRP along with an overview of the services that we offer as a part of the platform.

To accommodate this increase in users we have expanded into a second file system, bringing our total file storage for user data up to 40 TB. As a part of this refactor, all users will have a link to their allocation from the /home directory, meaning that your code will not need to change depnding on where your allocation actually sits.

We also expanded our applications volume dramatically, now weighing in at 500 GB, to allow us to continue to bring you new installations of software. As a part of this migration we found that the original DSKS environment dsks_2023.05 could not be rebuilt due to some of the packages being deprecated. If you need it to rerun some legacy code, this environment is still accessible as a conda-pack at /apps/conda-packs/dsks_2023.05. Note that we haven’t found a way to make a conda-pack work with our Jupyter App yet so this will only support scripts. If you need to convert your notebook into a script, consider using nbconvert which is available in any of our DSKS environments.

One of the new offerings we’ve developed with one of our users is a new conda environment for our Jupyter App that features the Detectron2 and Segment Anything packages by Facebook Research. You can view the full environment definition and package list here.

If you’ve been working on an environment that you think will be useful to others in your field you want to share with others on the platform, we want to hear from you. Let us know and we’ll help you make it available to everyone.

If you’d like to work with software that we don’t currently have installed on the platform, that does not mean that we won’t be able to support you. It just means that you’ll need to be patient and willing to help blaze the path with us so that we can extend the functionality of the platform for you. You are also free to install anything into your allocation that only requires using user space. Sometimes this just means that rather than using apt or apt-get you will need to get the software from conda-forge.

The increase in scale has also come with a new set of challanges, especially regarding the Strudel2 API. We apologise for the instability that some of you will have experienced in this time. We have since rolled out a new version of Strudel2 which should be more performant and robust, as well as adding additional safeguards to prevent the whole system coming down if some users encounter unexpected states.

If you experience any unexpected behaviour from the file system, the Strudel2 web page or anywhere else on the system, do not hesitate to reach out to us and we will work with you to resolve the issue.

Regards,
Mitchell Hargreaves

MLeRP is now in open beta!

For all of you who participated in the closed beta, thank you for your feedback and continued support of the platform. For those of you who are just joining us, welcome! You can read ARDC’s article about our service here.

We will be running a webinar hosted by the Machine Learning Community of Practice for Australia (ML4AU) on the 28th of November to support the launch of the open beta program. We will be introducing the platform’s capabilities and demonstrating some example use cases. If you’ve been wondering how you can better take advantage of the platform or are interested in knowing more about MLeRP before signing up, this webinar is for you. You can register for the webinar here.

In preperation for the launch we’ve been making some updates to the documentation. We have added a new section of the documentation archiving these release notes. We have also made these release notes available over RSS, so if like me you prefer to get your news through a feed aggregator rather than email you can now do that. We also updated our home page to better acknowledge our collaborators and our about page to better introduce the platform and what makes it different from other HPC offerings.

We had some feedback that the Dask PyTorch tutorial was a too full on for users new to machine learning. We’ve now seperated out the Dask components into its own tutorial from the PyTorch adaptation tutorial to hopefully seperate out the concepts for easier onboarding. Please contact us if there are any aspects of the platform which could be better explained and we will work with you to develop new tutorials and documentation to support these usecases.

We also made some updates to Strudel2 updated the app descriptions in Strudel2 and added a link back to the compute page to make it clearer the difference between the compute flavours. There is also a link through to the documentation in the footer now.

The old DSKS specific Jupyter Lab has now been deprecated. If you haven’t migrated to the new Jupyter Lab app yet, you’ll need to do that now. Through the app’s Conda Environment dropdown, you will be able to access the DSKS environments, any other environments that we provide in the future, along with any environments you’ve created in conda or mamba installations that you maintain.

Regards,
Mitchell Hargreaves

We’re pleased to announce that we now have a better UI solution to use conda environments with our new Jupyter Lab app. Now, rather than each app being tied to a given conda environment and requiring users to create a new app to support their own environments, we are using a dropdown which will allow you to select from any of the environments we provide, or any that you provide for yourself.

The dropdown is populated using your ~/.conda/environments.txt​ file which conda should maintain as you create new installations of miniconda or mamba. This means that the app should automatically pick up any new environments that you create with no input from yourself. If for whatever reason this doesn’t happen and you can’t find the environment that you’re looking for, you can manually append a new environment to the file with this command:

echo /path/to/your/environment >> ~/.conda/environments.txt​

We have also used this opportunity to come out with a new version of DSKS that addresses some of the shortcomings with the previous version. The environment now has more packages to support a wider range of data types and tools, for a full list visit the DSKS environment’s page.

If you still need to use the old version of DSKS, it’s still there just renamed. From now on, we will be versioning the environments that we release, tagging them with the year and month that they were built. The old version of DSKS that you are familiar with can now be accessed as dsks_2023.05, while the new version can be accessed as dsks_2023.10​. Similarly, the DSKS specific Jupyter App will point to this renamed environment. A full list of every environment that we support will be available on this new page of our documentation. There’s only 2 versions of DSKS there now, but we’re sure that the list will grow as we get more requests for packages and functionality. If you have a curated environment that you think would be suitable for other users, let us know and we’ll see what we can do.

We are also going to be deprecating the Base Jupyter Lab application. The Base Jupyter Lab application has always been in a weird place in that it simultaneously has too many packages to be representative of a basic python installation, but not enough for the kind of useful data science that we expect from users in the system. The app was originally written to work isolated within a singularity container, but as we move towards providing environments through our mambaforge installation continuing to maintain this unicorn environment seemed more trouble as it was worth, especially since the DSKS environments offer as a superset of it’s functionality and the base environment that is created in a mambaforge install is more suitable for minimal applications.

Please let us know how the new Jupyter Lab app works for you. If this experiment goes well, we’ll also deprecate the old DSKS specific Jupyter Lab since you can still reach that environment by selecting it in the environment dropdown.

Regards,
Mitchell Hargreaves

Over the last two weeks we’ve been addressing some of the initial feedback regarding onboarding to the service to try and smooth things out where we can.

We’ve had some users be confused about the difference between the QCIF and Monash regions. The two regions have different compute hardware and different file systems. As you all have been given file quota in the Monash region, you’ll need to select the Monash region in the ‘site selector’ on the left to launch your jobs. To help make this clearer to you all we’ve added a message in the ‘Account Info’ tab which will warn you if you’re trying to access the wrong region. You can read more about the differences between the regions here.

While we were at it, we also added a new dashboard to the Monash region which will display what compute resources are available for you to request so that . In the future we’d also like to have it break down GPU slices by type like we have but we had some difficulty with the implementation due to differences in SLURM version giving us different output formats so we will revisit this at a later date.

QCIF:

Monash:

We have also had feedback that DSKS is missing a few key packages for some use cases, such as NLP. We will be addressing this with the next version of DSKS coming soon, but before we implement this we want to work out a better way of displaying different conda environments as our current implementation would require us to have a new app for every version which could get cluttered really quickly. We don’t want to get in the habit of changing the environment under people’s feet, potentially breaking code so we will be putting this change off so we will be putting off this change until we have a good UI solution - which is the next feature we’ll be working on.

At the first user meeting there was a suggestion to support an environment which hosted similar packages to Google Colab’s default packages. We had a go at exporting the package list and trying to build this environment, but we were unable to get this environment to solve. Rather than iteratively trying to remove the problem packages until it builds, we believe we will be able to offer similar functionality through DSKS by adding the key packages that you all want from Colab’s environment into the next version, so if there is something you want, please tell us which packages are missing that you’d like.

In the meantime, if our environments don’t suit your purposes, at this stage you’re unfortunately going to have to maintain your own conda or mamba environment. The definitions of all of our environments can be found in the folder: /apps/conda-envs/. At this time, this includes DSKS as well as a template which has the minimum packages that you need to be compatible with our Strudel apps which you can use as a basis for your environment. To help with this process we’ve written a tutorial for installing one on our system. If there is anything that isn’t clear in this tutorial, please let us know so that we can help you. If you’re comfortable with making your environment, we still recommend that you have a look through the tutorial since it also covers how to point Strudel at your environment so that you can run Jupyter Lab sessions with it.

All of our documentation is publicly available on GitHub. If you feel like our documentation is lacking, we encourage you to submit Issues and Pull Requests to help us make it better for all of you. Even if you’re not contributing, cloning the repository is a great way to get easy access to our notebook tutorials to explore the functionality of the cluster.

Until next time,
Kind Regards,
Mitchell

I hope you are all doing well.

I have received a message from a user asking about getting help with provisioning, so I wanted to clear up that I provisioned all accounts according to your requested usernames and dataset sizes before the launch event. Meaning that you can all log in and test things right now through the login portal. I apologise if that wasn’t communicated clearly. The cluster has been very quiet recently, making now a great time to test the system without any wait time. If you’re not sure where to start, have a look at this tutorial or have a look at our documentation on connecting through the web portal.

Speaking of, we have just improved our documentation on alternative methods for connecting to the cluster through SSH or VS Code remote sessions. If this sounds like something that’s interesting to you, we’d love to get some feedback on this.

Since we last spoke at the launch event we’ve been working behind the scenes on some of your requested features. We’ve updated the user permissions of the home directory to 700 rather than 755. This means that privacy is now the default rather than collaboration. You still have the flexibility to change the permissions on your files if you’d like to do so.

We’ve also been having a think about the best way to implement the requested multiday low resource CPU notebooks (which we’ve been tentatively calling the Panther QoS internally). This is likely going to require changing the way we structure the cluster’s partitions under the hood, so we’ve hold off on implementing things so far but it’s on our radar. We’ll be in touch with more details when we have something for you all to try out.

Kind regards,
Mitchell Hargreaves