Meta unveiled its first-generation in-house AI inference accelerator designed to power the ranking and recommendation models that are key components of Facebook and Instagram back in 2023.
The Meta Training and Inference Accelerator (MTIA) chip, which can handle inference but not training, was updated in April, and doubled the compute and memory bandwidth of the first solution.
At the recent Hot Chips symposium last month, Meta gave a presentation on its next-generation MTIA and admitted using GPUs for a recommendation engines is not without challenges. The social media giant noted that peak performance doesn't always translate to effective performance, large deployments can be resource-intensive, and capacity constraints are exacerbated by the growing demand for Generative AI.
Mysterious memory expansionTaking this into account, Meta's development goals for the next generation of MTIA include improving performance per TCO and per watt compared to the previous generation, efficiently handling models across multiple Meta services, and enhancing developer efficiency to quickly achieve high-volume deployments.
Meta's latest MTIA gains a significant boost in performance with GEN-O-GEN, which increases GEMM TOPs by 3.5x to 177 TFLOPS at BF16, hardware-based tensor quantization for accuracy comparable to FP32, and optimized support for PyTorch Eager Mode, enabling job launch times under 1 microsecond and job replacement in less than 0.5 microseconds. Additionally, TBE optimization enhances embedding indices' download and prefetch times, achieving 2-3x faster run times compared to the previous generation.
The MTIA chip, built on TSMC's 5nm process, operates at 1.35 GHz with a gate count of 2.35 billion and offers 354 TOPS (INT8) and 177 TOPS (FP16) GEMM performance, utilizing 128GB LPDDR5 memory with a bandwidth of 204.8GB/s, all within a 90-watt TDP.
The Processing Elements are built on RISC-V cores, featuring both scalar and vector extensions, and Meta's accelerator module includes dual CPUs. At Hot Chips 2024, ServeTheHome noticed a Memory Expansion linked to the PCIe switch and the CPUs. When asked if this was CXL, Meta rather coyly said, “it is an option to add memory in the chassis, but it is not being deployed currently.”
More from TechRadar ProRussia wants to boost its censorship infrastructure and plans to spend almost 60 billion rubles (around $650 million) over the next five years to update its internet-blocking system.
This is the strategy outlined in the federal project 'Cybersecurity Infrastructure' as reported by Forbes. The Russian censor body, Roskomnadzor, aims to build a more effective filtering system for blocking VPN services in particular.
The Kremlin has long been battling against VPN usage across the country. That's because a VPN (virtual private network) is security software that encrypts internet connections and spoofs user IP addresses, granting access to otherwise geo-restricted content – exactly what Russian authorities seek to prevent.
The end of VPN usage in Russia?For the likes of Russia, China, Myanmar, and Iran – countries infamous for imposing strict restrictions on the internet – VPN usage has been a longstanding target. Over time, this has created an endless cat-and-mouse game between authorities and VPN providers to find a way to elude VPN censorship.
Technologies like VPN obfuscation and IP rotation are particularly useful at bypassing these blocks, with companies like Proton VPN continuing to invest in new anti-censorship tools. Now, the Kremlin appears ready to invest big money into reversing this trend.
"Russian authorities seem to be attempting to implement the Iranian model of internet censorship (even though we previously expected the Chinese model), which involves categorizing IP addresses into white, grey, and blacklists," Stanislav Shakirov, CTO of Russian digital rights group Roskomsvoboda and founder of Privacy Accelerator, told me. "However, they have not yet been able to achieve this with the current infrastructure."
Currently, Russia's technical approach to countering threats (TSPU in Russian) filters all Internet traffic to block or slow down certain resources. Yet, as Shakirov explained, the implementation of DPI (Deep Packet Inspection) has technical limitations. For instance, "it lacks sufficient computational power to block VPN protocols and throttle YouTube simultaneously," said Shakirov.
This is where the new strategy comes in. Between 2025 and 2030, Roskomnadzor plans to both modernize the current TSPU and install new ones. Censors expect this will increase the efficiency of VPN blocking by 96%.
In response to the censorship Russians are adopting VPN really fast. The government is blocking at least 42 providers, but smaller services and decentralised protocols are still available pic.twitter.com/EabxUDHUPnMay 16, 2023
Since 2019, all Russian telecom operators have been required by law to install TSPU software to allow Roskomnadzor censors to filter out unwanted online resources. The grip on the RuNet (Russian internet) became even stricter following the invasion of Ukraine. More than 100,000 resources were blocked in 2022 alone compared to around 7,000 during the previous year.
As censorship levels went up, VPN usage in Russia has been skyrocketing as well. Authorities then intensified the VPN crackdown that started back in 2017. The most recent move includes forcing Apple to remove at least 29 VPN apps from its local App Store. Plus, a law enforced in March now criminalizes the spread of information about ways to circumvent internet restrictions.
As data from the DPIDetector project suggests, major VPN providers are currently blocked by IP addresses and service domains – something that can be bypassed with an IP rotation feature.
Shakirov explained that, over the past year, authorities have also targeted VPN apps at a protocol level. WireGuard, for instance, is now blocked by mobile operators and, since August, on home internet connections. OpenVPN has also become challenging to use, but the blocking remains inconsistent. Plus, VPN software using more sophisticated anti-censorship technologies like Roskomsvoboda-developed Amnezia VPN continues to work well in Russia.
He said: "It is clear that the current technical resources are insufficient to block even a basic VPN protocol like OpenVPN, let alone more complex technologies. This is why these equipment purchases are a necessary step for Roskomnadzor, although we cannot say if they will be enough."
Is Qualcomm buying Intel? I saw this question plastered over X (cough, Twitter) today, and it got me thinking: what sort of upside-down world are these people living in? Indeed, even some actual news outlets have posted headlines posing the same question, and I’m here to set the record straight.
No, Qualcomm certainly isn’t buying Intel, and won’t be anytime soon. Despite its recent victories with the new Snapdragon X Elite chip, Qualcomm has not approached Intel with any sort of acquisition plan, and Intel itself still appears to be fully committed to its processor business – especially with how impressive its upcoming Lunar Lake chips look.
It’s been a bit of a wild week in terms of computing industry news: AMD revealed that it’s abandoning the flagship gaming GPU and we got our first glimpse of Apple Intelligence at the ‘Glowtime’ event, but it won’t be coming to iPhones or Macs for a little while yet. Among this, Reuters published a report claiming that Qualcomm has “examined acquiring different pieces of Intel”.
Now, that Reuters article contains the phrase ‘sources say’ multiple times in just the first few lines, and Qualcomm itself has refused to comment, so take all of this with a pinch of salt. But to break it down, all we really know is that Intel might be looking to shed some unprofitable parts of its business as it faces some financial turmoil, and Qualcomm might have been interested in buying those business units.
A new fighter enters the ringThere’s no denying that Qualcomm is riding high right now. I personally reported from Computex 2024 in Taipei earlier this year that Qualcomm was the clear winner of the event, with a rousing keynote address that features guest appearances from multiple laptop manufacturer executives, all throwing their collective hats into the Snapdragon ring.
The Snapdragon X Elite chip has proved to be a huge shake-up for the laptop processor market. (Image credit: Future / Philip Berne)It’s also no secret that Intel has been struggling a bit as of late; the massive upheaval caused by Snapdragon X Elite’s arrival has shaken up the entire laptop market, an area where Intel has historically dominated. AMD continues to offer fierce competition in other arenas, bringing its powerful systems-on-a-chip to handheld PCs like the Asus ROG Ally X, while the competing Intel-equipped MSI Claw A1M was just… whelming.
Investors also appear to be unhappy with Intel’s recent performance. The processor giant’s market capitalization dropped to $80 billion this week, the lowest it has been in over a decade. Meanwhile, Qualcomm’s market cap sits at $176bn at the time of writing, while long-time competitor AMD sits pretty at $224bn and big dog Nvidia’s AI investments have seen massive returns, bringing them up to a staggering 2.62 trillion dollars.
Never bet against IntelI have to say, though… I’m not worried. Even if Qualcomm does end up buying out some of Intel’s business ventures – perhaps its client PC design business, as suggested by Reuters’ sources – there’s plenty of potential for Team Blue to bounce back.
With a confirmed release date of September 24 for the new Lunar Lake CPUs as well as a slew of laptops that will come bearing those chips (plus a swiftly-rebooted MSI Claw 8 AI+), Intel is due a resurgence. We’ve also got Intel’s second-gen Arc ‘Battlemage’ discrete GPUs on the horizon, which could be a welcome addition to the affordable graphics card market even if the first-gen Arc cards failed to make a huge splash.
The recently-revealed MSI Claw 8 AI+ will utilize an Intel Lunar Lake chip for its newly boosted gaming capabilities. (Image credit: Future / John Loeffler)Besides, (without wanting to get political here) Intel isn’t going to go under, because the US Military needs it too much. The chipmaker has numerous long-running contracts and agreements with both the Department of Defense and the Department of Energy, so odds are a government-assisted bailout will occur if those straits begin to look a little too dire. So if you're a Team Blue fanboy, have no fear: even the military-industrial complex has your back.
You might also like...