Microsoft has announced the launch of new Azure virtual machines (VMs) aimed specifically at ramping up cloud-based AI supercomputing capabilities.
The new H200 v5 series VMs are now generally available for Azure customers and will enable enterprises to contend with increasingly cumbersome AI workload demands.
Harnessing the new VM series, users can supercharge foundation model training and inferencing capabilities, the tech giant revealed.
Scale, efficiency and performanceIn a blog post, Microsoft said the new VM series is already being put to use by a raft of customers and partners to drive AI capabilities.
“The scale, efficiency, and enhanced performance of our ND H200 v5 VMs are already driving adoption from customers and Microsoft AI services, such as Azure Machine Learning and Azure OpenAI Service,” the company said.
Among these is OpenAI, according to Trevor Cai, OpenAI’s head of infrastructure, which is harnessing the new VM series to drive research and development and fine-tune ChatGPT for users.
“We’re excited to adopt Azure’s new H200 VMs,” he said. “We’ve seen that H200 offers improved performance with minimal porting effort, we are looking forward to using these VMs to accelerate our research, improve the ChatGPT experience, and further our mission.”
Under the hood of the H200 v5 seriesAzure H200 v5 VMS are architected with Microsoft’s systems approach to “enhance efficiency and performance,” the company said, and include eight Nvidia H200 Tensor Core GPUs.
Microsoft said this addresses a growing ‘gap’ for enterprise users with regard to compute power.
With GPUs growing in raw computational capabilities at a faster rate than attached memory and memory bandwidth, this has created a bottleneck for AI inferencing and model training, the tech giant said.
“The Azure ND H200 v5 series VMs deliver a 76% increase in High Bandwidth Memory (HBM) to 141GB and a 43% increase in HBM Bandwidth to 4.8 TB/s over the previous generation of Azure ND H100 v5 VMs,” Microsoft said in its announcement.
“This increase in HBM bandwidth enables GPUs to access model parameters faster, helping reduce overall application latency, which is a critical metric for real-time applications such as interactive agents.”
Additionally, the new VM series can also compensate for more complex large language models (LLMs) within the memory of a single machine, the company said. This thereby improves performance and enables users to avoid costly overheads when running distributed applications over multiple VMs.
Better management of GPU memory for model weights and batch sizes are also a key differentiator for the new VM series, Microsoft believes.
Current GPU memory limitations all have a direct impact on throughput and latency for LLM-based inference workloads, and create additional costs for enterprises.
By drawing upon a larger HBM capacity, the H200 v5 VMs are capable of supporting larger batch sizes, which Microsoft said drastically improves GPU utilization and throughput compared to previous iterations.
“In early tests, we observed up to 35% throughput increase with ND H200 v5 VMs compared to the ND H100 v5 series for inference workloads running the LLAMA 3.1 405B model (with world size 8, input length 128, output length 8, and maximum batch sizes – 32 for H100 and 96 for H200),” the company said.
More from TechRadar ProRumors, misinformation and lies about the federal government’s response to Hurricane Helene have run rampant since the storm made landfall, especially around FEMA funding.
(Image credit: Mike Stewart)
AI has become very good at holding up its end of a conversation with humans, but a set of new AI features from the digital notetaking app Goodnotes performs an even more impressive stunt by reading handwriting well enough to discuss it and even answer questions about what's been scribbled. Goodnotes, which claims 24 million monthly active users, debuted handwriting editing capabilities along with a math-specific AI helper and the Ask Goodnotes assistant that serves as a kind of secretary for your notetaking.
The handwriting editing tools impressively link human writing to digital understanding. They're based on the proprietary Goodnotes Smart Ink technology, which takes down your handwriting and attempts to turn it into typed text. Now, though, the app lets you edit what you've handwritten the way you would something typed out in a document. That includes aligning notes, copying and pasting some of the handwriting, and reflowing the text to make it more logical when going through it.
That's on top of the Spellcheck and Word Complete tools already available for handwritten text. Goodnotes pitches these AI handwriting editing capabilities as a way to combine the flexibility of pen-and-paper notetaking with the ease of editing offered by digital tools. You can see how it works below.
(Image credit: Goodnotes) AI SecretaryAsk Goodnotes, as the name implies, lets you ask questions about what's in your notes, get summaries, explain concepts you jotted down, and even help put together quizzes to test you on the knowledge. So, if you are a student or at a work presentation, the AI can take your hastily scrawled notes and, days later, explain what you were writing about, including researching any concepts you were too vague about to remember what you meant. It can then help you study for a test on the topic or prepare to talk about it with others.
It works with more than just handwritten notes, so you can augment what you wrote with printed text, images, and PDFs. The answers from the AI are personalized and will link to your notes to ensure you understand the context of what it is saying and what you wrote earlier.
The Math Assist feature zeroes in on helping with mathematical equations written out by hand in your notebook. Math Assist recognizes handwritten math problems and can perform calculations to give the answer. It can also show the steps for solving the problem for anything from arithmetic to calculus. If you don't want the full answer, the AI can also restrict itself to hints so you can solve the equations on your own. Goodnotes is available on Apple devices with up to three notebooks for free. All features are available for $10 a year or a lifetime fee of $30.
"We're constantly inspired by the sheer volume of ideas and knowledge that our users capture in their Goodnotes notebooks. Our aim with Ask Goodnotes is to give users new powers to interact with their notes, documents, and PDFs, and unlock fresh possibilities for productivity, creativity, and learning," said Steven Chan, founder and CEO of Goodnotes. "With our new handwriting editing and math features, we focused on how our proprietary machine learning models could be leveraged behind the scenes to make everyday note-taking and document annotation more seamless and intuitive."
You might also like...The justices also left in place a Michigan state constitutional amendment that barred the use of public funds for private schools.
(Image credit: Kent Nishimura)
Many film critics and comic book fans say they felt the sequel felt short of their expectations. Others argue that was the whole point all along.
(Image credit: Alon Amir)
Mini PC makers are becoming increasingly inventive as they cater to the growing demand for compact yet powerful computing solutions. With consumers and businesses alike seeking more space-efficient devices without compromising performance, manufacturers are pushing boundaries in design and functionality.
Geekom makes some of our favorite mini devices, like the AX8 Pro, and its latest offering, the Mini Air12 Lite, is another great addition to its portfolio. As you can guess from the name, this is a scaled-down - and cheaper - version of Geekom’s Mini Air12.
Powered by a 12th Gen Intel Alder Lake N100 processor, it comes with up to 16GB of DDR4-3200 RAM and up to 1TB of M.2 2280 PCIe Gen 3 x4 SSD storage. You can preorder it on the Geekom US site with 8GB RAM and a 256GB SSD for $199 (down from the usual MSRP of $249). It’s expected to ship in October.
Plenty of portsThe Mini Air12 Lite provides a decent array of ports. On the back, you'll find two USB 3.2 Gen 2 and two USB 2.0 ports, as well as DisplayPort 1.4 and HDMI 2.0 outputs, supporting dual-monitor setups, along with an RJ45 Ethernet port. Wireless connectivity is provided via WiFi 5 and Bluetooth 5.1.
The front panel features two USB 3.2 Gen 1 ports and separate 3.5mm jacks for a microphone and headphones. Digital audio is also an option via HDMI and DP.
What sets the Mini Air12 Lite apart is the 9-pin expansion header on the front. This provides pins for connecting external components or additional functionalities, including a 5V DC power pin, power switch pins, and pins for power and HDD activity LEDs. Since the device already features a power button and built-in power LED, we'd guess the aim is to offer users more customization options, possibly configuring these pins for extra controls or indicators, but we'll have to wait until it's released to know for certain.
More from TechRadar Pro