Let researchers focus on research with Backend.AI

Eunseong Choi, a researcher in search and natural language processing, began working in machine learning and AI in late 2019 during graduate school and has remained in this field since then. He now works at Liner, a search technology startup, where he is responsible for model training and evaluation as well as building search pipelines.

During graduate school, when BERT was first gaining attention, he trained and fine-tuned models with various architectures. Over time, he scaled up to models of around 70 billion parameters and handled both training and inference. Backend.AI, deployed at the Sungkyunkwan University Supercomputing Center, was the core infrastructure that supported a significant portion of this work.

Want to learn how Sungkyunkwan University operates its supercomputing center with Backend.AI? [View case study]

The lab had around 16 to 20 students. Due to budget constraints, they had no choice but to purchase multiple servers each equipped with only two GPUs. Over time, the number of these servers increased to about 40 to 50 units. Meanwhile, the models they worked with kept growing in size, but two‑GPU servers could not support the multi‑GPU configurations they needed.

To work around this, they frequently had to move GPUs from different servers, consolidate them into a four‑GPU machine, and then dispose of or repurpose the remaining components. This kind of ad‑hoc reconfiguration of server hardware became a recurring task for the lab.

“Servers went down often, and in reality all a student could do was reboot. Each time you walk over and reboot a machine, you lose at least 30 to 40 minutes.”

Managing the software environment was another persistent challenge for the lab. They had enough hardware to allocate small servers to individual members, and they operated shared servers using a Google Sheet to manage reservations. Personal servers were convenient because a single user had exclusive access to the hardware, but this led to low utilization and made it difficult to manage idle machines effectively.

On the shared servers, environments frequently became inconsistent after different people used the same machine. Even a simple change, such as switching the PyTorch version, often broke compatibility with the installed CUDA version and caused various components to fail, and they spent significant time restoring a working setup. Tooling choices also varied among users. Some used Docker, others relied on Conda, and some did not use any environment management tool at all, so there was no consistent baseline for the shared environment and keeping it stable was difficult.

Eunseong Choi recalls that once resources were pooled and managed under a single system, efficiency and convenience improved significantly for the lab. Instead of asking how to extend or reconfigure the hardware they had already bought, they could focus on consuming shared resources in a consistent way. Because the operating environment was unified through Backend.AI, they also spent less time on environment setup. The experience was similar to each user receiving a virtual machine, where they could start from a clean workspace and choose the software environment they needed.

There might be a concern that repeatedly receiving new hardware allocations would make it hard to recover previous settings. To address this, Eunseong prepared a script that runs whenever he first connects to a newly assigned GPU resource. Each time he receives new resources, he runs that script, and his usual environment is ready within about 10 to 15 minutes, so he can quickly continue his work. Compared with the period when the lab operated its own hardware stack, he estimates that the effort required has dropped by a factor of ten or more in practical terms.

Everyday convenience with VFolders and VS Code integration

One of the things they remember most clearly, alongside the consolidation of resources, is how two specific features streamlined their day‑to‑day work. The first is storage. While using Backend.AI, Eunseong Choi found the way VFolders work very convenient, especially the behavior where folders whose names start with a dot are mounted automatically.

Each user in the lab could create a folder under their own name and have it auto‑mounted. This allowed them to access files in a colleague’s folder directly, without using SSH, which made collaboration much easier. They note that their company now uses various external platforms such as RunPod and Lambda Labs, but these either do not provide similar automatic mounting or, when they do, the feature is so slow that it is not practically useful.

Another everyday convenience for them was VS Code integration. At the time, they often wrote code manually and copied snippets from web search results. With Backend.AI, they could launch VS Code directly from the application and access and edit their code right away, which made this workflow much smoother.

Because VS Code could be opened inside the WebUI, there was no need to open an SSH session or create a separate connection just for the editor. While using it, this felt so natural that they did not even recognize it as a special convenience. Only after working with other software platforms did they realize how much easier Backend.AI had made their daily development tasks.

Recommendation for academic environments

“For people who feel burdened by managing infrastructure but still want to do research, it is extremely convenient when the environment you need is created with a simple click, and you can focus only on the research you want to do.”

Eunseong Choi notes that some researchers enjoy working with hardware, such as assembling computers, while others chose this field because they are interested in AI and software itself. The ability to write strong research papers and the ability to manage terminals and infrastructure are very different skills, so it is not realistic to expect every researcher to also become an infrastructure expert.

Having experienced both the pain of operating their own infrastructure and the convenience of a unified system, he has a clear view on who would benefit from Backend.AI. In his opinion, any organization that needs to build an on‑premise server environment, especially universities, should strongly consider it. Once an on‑premise cluster is in place, there must be a way to manage those servers, and from the user’s point of view, working through Backend.AI was very convenient and satisfying, so he believes adopting Backend.AI is a good choice if member satisfaction matters.

“The intuitive UI made it easy for first‑time users to get started.”

Before speaking with the Backend.AI team, ML research scientist Eunseong Choi reached out to former lab colleagues to ask how they remembered using Backend.AI. They told him that what remained in their memory was simply that it was comfortable and easy to use.

Backend.AI was the first cluster environment Eunseong had ever used, yet he could immediately understand how to use it. For him, it quickly became natural that “you use this for that task, and when you press the button, resources are allocated,” without needing any prior experience with cluster systems.