24.03, the first release of Backend.AI in 2024, has been released. This update brings significant improvements to the UI and user experience, making it even more functional to install and operate. It includes the following updates since 23. 9, which include

Backend.AI Core & WebUI

• We've made installing Backend.AI even easier with the addition of a TUI-based installer, which automates the download process and makes it easy for users to install and get started with Backend.AI.

• New: Added trash functionality to vfolders. Files in unused vFolders are now moved to the trash instead of deleting them completely, and are then deleted through a complete deletion process. New: Added an argument value to indicate the state of the vfolder.
• New: Added Backend.AI Model Store, where you can now store, search, and easily utilize various machine learning and deep learning models.
• Added metadata for indexing to vfolders to utilize indexes instead of full directory scans for queries.
• Improved system resource utilization by introducing a limit policy for session creation based on the number of pending sessions and requested resource slots. This new resource policy option helps filter and set the maximum value for resource presets and custom resource sliders in the Session Launcher.
• We've added dark themes to the WebUI, so users can now choose from a variety of options to suit their personal preferences.

• Improved screen tuning for unaligned line breaks, whitespace, and extruded announcements in the WebUI, as well as stability improvements such as session name validation.
• The Session Launcher for Model Serving also limits UI input so that only as much input is available as the allocated resources.
• Added an allowAppDownloadPanel argument to hide the WebUI app download panel in the config.toml file for different UI user options.

Backend.AI is constantly evolving to provide a more powerful and user-friendly experience while supporting a variety of environments in the ever-changing AI ecosystem. We look forward to seeing what's next! Make your AI accessible with Backend.AI!

23.09: September 2023 update

In the second half of 2023, we released 23.09, a major release of Backend.AI. In 23.09, we've significantly enhanced the development, fine-tuning, and operational automation of generative AI. We've automatically scaled and load-balanced AI models based on workload, expanded support for various GPUs/NPUs, and increased stability when managing a single node as well as 100-2000+ nodes. The team is working hard to squeeze every last bit out of it. Here are the main improvements since the last [23.03 July update] (/posts/2023/07/31/Backend.AI-23.03-update).

Backend.AI Core & UI

• The Backend.AI Model Service feature has been officially released. You can now use Backend.AI to more efficiently prepare environments for inference services as well as training of large models such as LLM. For more information, see the blog Backend.AI Model Service sneak peek.
• Added the ability to sign in to Backend.AI using OpenID single sign-on (SSO).
• If your kernel image supports it, you can enable the sudo command without a password in your compute session.

• Support for Redis Sentinel without HAProxy. To test this, we added the --configure-ha setting to the install-dev.sh file.
• Added the ability to use the RPC channel between Backend.AI Manager and Agent for authenticated and encrypted communication.
• Improved the CLI logging feature of Backend.AI Manager.
• Fixed an issue where Manager could not make an RPC connection when Backend.AI Agent was placed under a NAT environment.
• The Raft algorithm library, riteraft-py, will be renamed and developed as raftify.
• Support for the following new storage backends
  • VAST Data
  • KT Cloud NAS (Enterprise only)

Backend.AI FastTrack

• Improved UI for supporting various heterogeneous accelerators.
• Deleting a VFolder now uses an independent unique ID value instead of the storage name.
• Upgraded Django version to 4.2.5 and Node.js version to 20.
• Added pipeline template feature to create pipelines in a preset form.
• If a folder dedicated to a pipeline is deleted, it will be marked as disabled on the FastTrack UI.
• Improved the process of deleting pipelines.
• Added a per-task (session) accessible BACKENDAI_PIPELINE_TASK_ID environment variable.
• Actual execution time per task (session) is now displayed.

Contribution Academy

Especially in the past period, the following code contributions were made by junior developer mentees through the 2023 Open Source Contribution Academy organized by NIPA.

• Created a button to copy an SSH/SFTP connection example to the clipboard.
• Refactored several Lit elements of the existing WebUI to React.
• Wrote various test code.
• Found and fixed environment variable and message errors that were not working properly.

Backend.AI is constantly evolving to provide a more powerful and user-friendly experience while supporting various environments in the AI ecosystem. Stay tuned for more updates!
Make your AI accessible with Backend.AI!

This post is automatically translated from Korean

23.03: July 2023 update

A wrap-up of the ongoing updates to Backend.AI 23.03 and 22.09. The development team is working hard to squeeze every last bit out.

Here are the most important changes in this update

• Enhanced storage manageability: Added per-user and per-project storage capacity management (quotas) with VFolder v3 architecture.
• Expanded NVIDIA compatibility: Support for CUDA v12 and NVIDIA H100 series.
• Extended hardware compatibility: Support for WARBOY accelerators from FuriosaAI.

Backend.AI Core & UI

• Supports CUDA v12 and NVIDIA H100 series.
• Supports the WARBOY accelerator, the first NPU from FuriosaAI company.
• Added storage capacity management function (Quota) by user and project by applying VFolder v3 architecture.
  • However, it is limited to storage that supports Directory Quota.
• Fixed an error that caused multi-node cluster session creation to fail.
• Fixed an error where a compute session in the PULLING state was incorrectly labeled as PREPARING.
• Fixed an error in which the CLONING state was incorrectly displayed when cloning a data folder with the same name when multiple storage devices have the same folder.
• Improved the web terminal of a compute session to use zsh as the default shell if the zsh package is installed in the kernel image.
• Added the ability to know the health status of the (managed) storage proxy and event bus.

Backend.AI FastTrack

• Added the ability to set multi-node cluster mode by task.
• Fixed an error where environment variables set in .env were not applied to the frontend.
• Fixed an error recognizing out-of-date when accessing with a mobile browser.
• Added a field to show the cause message when a task-specific error occurs.
• Fixed other editor-related issues.

Backend.AI is constantly evolving to provide a more powerful and user-friendly experience while supporting various environments in the ever-changing AI ecosystem. Stay tuned for more updates!
Make your AI accessible with Backend.AI!

A recap of the ongoing updates to Backend.AI 23.03 and 22.09. The development team is working hard to squeeze every last bit out.

Here are the most important changes in this update:

• Expanded hardware compatibility: Expanded hardware compatibility with support for ATOM accelerator idle checking and Dell EMC storage backends from Rebeillons.
• High-speed upload enhancements: Introduced SFTP functionality to support high-speed uploads to storage.
• Development Environment Enhancements: Enhanced the development environment by allowing sessions to be accessed in remote SSH mode from local Visual Studio Code.
• Increased manageability: Improved the user interface for administrators to make it easier to set up AI accelerators and manage resource groups.

Backend.AI Core & UI

• Added support for idle state checking of ATOM accelerators.
• Introduced SFTP functionality to support high-speed uploads directly to storage.
• Added ability to force periodic password updates based on administrator settings.
• Added an upload-only session (SYSTEM) tab.
• Added Inference type to the allowed session types.
• Added the ability to connect to a session in remote SSH mode from local Visual Studio Code.
• Added support for uploading folders from Folder Explorer.
• Improved the display of the amount of shared memory allocated when creating a session.
• Added support for Dell EMC storage backend.
• Improved the accuracy of container memory usage measurement.
• Improved the ability to run multiple agents concurrently on a single compute node.
• Added project/resource group name filter for administrators.
• Added user interface for administrators to set various AI accelerators, including GPUs, in resource presets/policies.
• Added a user interface for administrators to display the allocation and current usage of various accelerators, including GPUs.
• Added a user interface for administrators to set the visibility of resource groups.
• Provided a user interface for administrators to view the idle-checks value per session.
• Added recursion option when uploading vfolders in the CLI, and improved relative path handling.
• Added a recursive option in the CLI to terminate sessions with dependencies on specific session termination at once.
• Added a new mock-accelerator plugin for developers, replacing the old cuda-mock plugin.
• Added status and statistics checking API for internal monitoring of the storage proxy for developers.

Backend.AI FastTrack

• Improved searching for vfolders by name when adding pipeline modules.
• Added an indication to easily recognize success/failure after pipeline execution.

Backend.AI Forklift

• Bug fixes and stability improvements.
• Support for deleting build job history.
• Supports pagination of the build task list.

Backend.AI is constantly evolving to support a variety of environments in the ever-changing AI ecosystem, while providing a more robust and user-friendly experience. Stay tuned to see what's next!

Make your AI accessible with Backend.AI!

This post is automatically translated from Korean

23.03: 2023년 3월 업데이트

We're excited to announce version 23.03.0, the first major release of Backend.AI for 2023. Some features will continue to be rolled out in subsequent updates.

Specifically in this update:

• Support for the 'inference' service with a new computation session type.
• Support for 'model' management with a new storage folder type.
• Support for managing storage capacity on a per-user and per-project basis.
• Significant improvements to FastTrack's pipeline versioning and UI.

Backend.AI Core & UI (23.03)

• Added model management and inference session management capabilities.
  • More advanced inferential endpoint management and network routing layers will be added in subsequent updates.
• The codebase has been updated to be based on Python 3.11.
• Introduced React components to the frontend and leveraged Relay to introduce a faster and more responsive UI.
• Full support for cgroup v2 as an installation environment, starting with Ubuntu 22.04.
• Updated the vfolder structure to v3 for storage capacity management on a per-user and per-project basis.
• Kernel and sessions are now treated as separate database tables, and the state transition tracking process has been improved to work with less database load overall.
• Improved the way the agent displays the progress of the image download process when running a session.
• Improved the display of GPU usage per container in CUDA 11.7 and later environments.
• Scheduling priority can be specified by user and project within each resource group.
• Supports two-factor authentication (2FA) login based on one-time password (TOTP) to protect user accounts.
• Support for users to register their own SSH keypair for session access.
• Supports user interfaces for Graphcore IPUs and Rebellions ATOM devices.

Backend.AI Forklift (23.03)

• Added Dockerfile templates and advanced editing capabilities.
• Support for creating container images for inference.
• Extended image management capabilities to work with the Harbor registry.

Backend.AI FastTrack (23.03)

• Storage folder contents can be viewed directly from the FastTrack UI.
• Improved session state synchronisation with Core to event-based.
• You can set the maximum number of iterations for a pipeline schedule.
• If a task fails to execute, the pipeline job is automatically cancelled instead of waiting.
• Added pipeline versioning. You can track the shape history of your pipeline, and you can recall the contents at a specific point in time to continue working on it.
• You can modify pipelines in YAML format directly through the code editor.

개발 및 연구 프레임워크 지원

• Supports TensorFlow 2.12, PyTorch 1.13
• Support for NGC (NVIDIA GPU Cloud) TensorFlow 22.12 (tf2), NGC PyTorch 22.12, NGC Triton 22.08
• Added python-ff:23.01 image, which provides the same libraries and packages as Google Colab

In addition to what we've listed above, we've included many bug fixes and internal improvements.
Stay tuned for more to come!

This post is automatically translated from Korean