May 12, 2024
Enhancing Deep Learning Workflows: PyTorch Ecosystem Tools
Welcome to the thriving PyTorch ecosystem, where a wealth of tools and libraries await, purpose-built to elevate your experience in deep learning as a developer or researcher. The Ecosystem Tools pages host many projects from experts spanning academia, industry, application development, and machine learning.
May 11, 2024
Deep Learning Energy Measurement and Optimization
Zeus is an open-source toolbox for measuring and optimizing the energy consumption of deep learning workloads. Our goal is to make energy optimization based on accurate measurements as easy as possible for diverse deep learning workloads and setups by offering composable tools with minimal assumptions.
May 11, 2024
Introducing depyf: mastering torch.compile with ease
We are thrilled to introduce depyf, a new project to the PyTorch ecosystem designed to help users understand, learn, and adapt to torch.compile!
February 15, 2024
Exploring scientific machine learning pipelines through the SimulAI toolkit
SciML, short for Scientific Machine Learning, encompasses work that merges quantitative sciences with machine learning. It has gained significant traction over the past decade, driven by the widespread availability of specialized hardware (such as GPUs and TPUs) and datasets. Additionally, it has been propelled by the overarching influence of the machine learning wave, now ingrained in the zeitgeist of our times. In this context, we’d like to introduce SimulAI, an open-source toolkit under th...
January 29, 2024
Colossal-LLaMA-2: Low Cost and High-quality Domain-specific LLM Solution Using LLaMA and Colossal-AI
The most prominent distinction between LLaMA-1 and LLaMA-2 lies in the incorporation of higher-quality corpora, a pivotal factor contributing to significant performance enhancements in LLaMA-2. This, coupled with its commercial availability, extends the potential for creative applications of large models within the open-source community.
January 25, 2024
3D rotations and spatial transformations made easy with RoMa
Struggling with quaternions, rotation vectors, right-hand rules and all these stuffs? Try RoMa: an easy-to-to-use, stable and efficient library to deal with rotations and spatial transformations in PyTorch.
January 04, 2024
torchdistill — a modular, configuration-driven framework for reproducible deep learning and knowledge distillation experiments
This article summarizes key features and concepts of torchdistill (v1.0.0). Refer to the official documentation for its APIs and research projects.
December 06, 2023
PyPose: A Library for Robot Learning with Physics-based Optimization
We are excited to share our new open-source library PyPose. It is a PyTorch-based robotics-oriented library that provides a set of tools and algorithms for connecting deep learning with physics-based optimization.
November 09, 2023
How Activation Checkpointing enables scaling up training deep learning models
Activation checkpointing is a technique used for reducing the memory footprint at the cost of more compute. It utilizes the simple observation that we can avoid saving intermediate tensors necessary for backward computation if we just recompute them on demand instead.
October 26, 2023
torch.compile, explained
Have you ever felt overwhelmed by the complexities of torch.compile? Diving into its workings can feel like black magic, with bytecode and Python internal details that many users fail to understand, hindering them from understanding and debugging torch.compile.
July 06, 2023
Unveiling the Power of Semi-Supervised Learning: The Unified Semi-Supervised Learning Benchmark
Machine Learning models thrive on high-quality, fully-annotated data. The traditional supervised learning approach typically requires data on the scale of millions, or even billions, to train large foundational models. However, obtaining such a vast amount of labeled data is often tedious and labor-intensive. As an alternative, semi-supervised learning (SSL) aims to enhance model generalization with only a fraction of labeled data, complemented by a considerable amount of unlabeled data. This...
June 29, 2023
Introducing TorchOpt: A High-Performance Differentiable Optimization Library for PyTorch
Explore TorchOpt, a PyTorch-based library that revolutionizes differentiable optimization with its unified programming abstraction, high-performance distributed execution runtime, and support for various differentiation modes.”
April 04, 2023
Profiling PyTorch language models with octoml-profile
The recent launch of PyTorch 2.0 makes it clear that the community is heavily investing in a compiler-powered future for machine learning. The new OctoML Profiler can help any user realize the full potential of these shifts in the ML landscape.
February 10, 2023
How FASHABLE achieves SoA realistic AI generated images using PyTorch and Azure Machine Learning
Fashable is a company born at XNFY Lab (a joint initiative with Microsoft). The company’s main goal is to revolutionize the world of fashion with ethical Artificial Intelligence (AI) technologies built on PyTorch framework. Fashable is focused on developing AI models that generates synthetic contents for the global fashion industry. The Fashion industry has been criticized in recent years because it generates a lot of waste and is responsible for up to 10% of global carbon dioxide output. Fas...
January 31, 2023
Latest Colossal-AI boasts novel automatic parallelism and offers savings up to 46x for Stable Diffusion 2
As a new PyTorch Ecosystem Partner, we at HPC-AI Tech look forward to working with the PyTorch community to advance AI technologies through our open source project, Colossal-AI. We are excited to join forces with the PyTorch community in this effort.
January 06, 2023
Distributed training with PyTorch and Azure ML
Suppose you have a very large PyTorch model, and you’ve already tried many common tricks to speed up training: you optimized your code, you moved training to the cloud and selected a fast GPU VM, you installed software packages that improve training performance (for example, by using the ACPT curated environment on Azure ML). And yet, you still wish your model could train faster. Maybe it’s time to give distributed training a try! Continue reading to learn the simplest way to do distributed t...
