Our team recently traveled from California to New York for a large language model (LLM) meetup hosted by our partner, Weights & Biases (W&B). During the meetup, our CEO Subutai Ahmad and W&B Software Engineer Anish Shah showed how Numenta and W&B are working together to help enterprises deploy LLMs at scale. In this blog post, I’ll give a recap and highlight how our AI platform integrates with W&B.
A quick overview of NuPIC
The Numenta Platform for Intelligent Computing (NuPIC) is a neuroscience-based AI platform designed to run LLMs efficiently on CPUs – far more efficiently than you can on GPUs. At the core of the platform is our optimized inference server, where you can run a range of LLMs with high throughput and low latency. NuPIC includes a training module where you can fine-tune models for a particular application with your own data.
NuPIC also comes with a model library containing highly optimized NuPIC-base models we’ve pre-trained. The library consists of generative and non-generative models suitable for many different Natural Language Processing (NLP) applications, or you can bring your own model. Since NuPIC can run entirely on CPUs, it supports deployments involving many different types of models concurrently. Each model can be completely asynchronous, and you can run these different models on the same server.
Navigating the complexities of AI model experimentation and tracking
While starting with a NuPIC-base model provides a strong start to your AI journey, adapting it to meet the unique requirements of your application often requires extensive fine-tuning and experimentation. This iterative process usually involves tens or even hundreds of models, and as the number of experiments grows, keeping track of them becomes more and more challenging.
You might find yourself asking: Which models performed well? Which ones didn’t? Which model had that specific architecture? What were the results of the model you ran last Wednesday? Tracking these details manually can be cumbersome and error-prone, leaving you with a mess of Excel spreadsheets, handwritten notes, and forgotten experiments.
This is where W&B comes in.
NuPIC and Weights & Biases integration
For those unfamiliar with W&B, they provide tools for machine learning practitioners to monitor, visualize, and collaborate on experiments. We’ve integrated W&B features throughout NuPIC, and anyone with an existing W&B account can access these features seamlessly.*
At the meetup, we showed how NuPIC users can integrate W&B into their workflow:
1. Experiment Tracking
From parameter changes to performance metrics, W&B Experiment Tracking automatically logs every detail of your fine-tuning experiments from the NuPIC Training Module in real time. W&B also provides powerful visualization tools that allow you to compare different experiments side-by-side, identify which adjustments yield the best results, and understand the trade-offs involved. Moreover, each experiment in W&B is assigned a unique ID, ensuring that no matter how many versions you go through, you can always trace back and understand the evolution of your models and projects.
2. Model Registry
Beyond tracking experiments, NuPIC users can use the W&B Model Registry to keep track of different model versions, variants and metadata, such as configurations, results, and source files, from inception to deployment. With all your models systematically organized and accessible, you can always roll back or reference a specific iteration of your model and quickly deploy any version into production with the NuPIC Inference Server.
*To use these features, you must have a W&B account. Once linked, all your experiments and models will seamlessly populate.
Conclusion
NuPIC, combined with W&B’s robust tracking and registry capabilities, enables simple, scalable LLM deployments. Users can leverage the fantastic features of W&B while enjoying the native functionalities of NuPIC:
- Deploy high-performing LLMs on CPUs at scale: NuPIC makes CPUs the ideal solution for deploying robust LLMs without compromising performance. And W&B allows users to efficiently track and manage multiple experiments and models at scale.
- Maintain complete control over your data: NuPIC is delivered as Docker containers. Everything runs on your infrastructure, and you don’t have to send any data to us. Unlike SaaS services, we don’t get any data about your usage, what or when you’re training, etc. For their part, W&B is cloud-agnostic and offers on-prem and private cloud options. This enables businesses to safeguard their sensitive data and models.
- Deploy LLMs with minimal steps: NuPIC is designed to make LLM deployments as straightforward and hassle-free as possible. With NuPIC, you can fine-tune and deploy LLMs with just a few lines of code. Integrating W&B into your workflow is as simple as adding two command line arguments where you connect the API key and project name. Everything is then handled for you – all you need to do is pick your commands and get your results.
If you’d like to stay informed about product updates, new features, or any NuPIC-related news, sign up here to join our NuPIC mailing list.
If you’re interested in exploring how your company can benefit from NuPIC, you can request a demo here.
Charmaine Lai • Marketing Manager