Great Post. What about Marvell? Can its ASIC chips gain market share in AI given that GPU will loose its market share to ASIC in AI based on Mckinsey recent report.
I'm very curious to hear more about XLA, why/how did NVIDIA have such a stronghold on the space for so long. Doesn't Google have a more challenging problem since they potentially need to work with all hardware or is XLA more optimized on TPUs?
Also how come optical computing is still a niche? Free inference sounds like something everyone would jump on? Any programmer friendly tutorials you'd recommend here? What does programming an optical computer look like and what are the key differences a practitioner should be aware of
I think Nvidia's lead arose from the fact that no one else really cared to try for a long time! CUDA was used for graphics programming and had a robust toolchain by 2012/2013 where a couple research groups figured out you could build really impressive deep networks on CUDA. Nvidia quickly cottoned on and started building custom CUDA kernels for deep learning rapidly. I think Google realized early on that CUDA put them in a bit of a bind, but bootstrapping a new compiler ecosystem is a formidable undertaking and I think it took them time to start. The fact they have a strong need now with TPUs also probably pushed their development. XLA on GPUs compiles to CUDA still I believe, but on TPUs/CPUs does something different. But it means that Google is now insulated from CUDA by one level additional shielding.
Optical computing is really interesting. I think silicon photonics has been around for a long time, but no one really had a killer use case that really brought it into the mainstream. Even now, I think the photonics inference startups face a steep barrier. Nvidia hasn't been standing still and has been moving quickly to make their inference chips better. I think it's at best a 50/50 bet that photonics inference catches on even now. Tbh, I don't really know of any good resources for programmers but will keep an eye out! The issue is like with quantum computers, very few people even have access to an optical chip to program! I imagine that eventually someone will come up with an API that's roughly like TensorFlow declarative style (build your computation graph and it's compiled down to the optical chip)
Great Post. What about Marvell? Can its ASIC chips gain market share in AI given that GPU will loose its market share to ASIC in AI based on Mckinsey recent report.
Great pointer! I actually didn't know at time of writing Marvell was building custom ASICs! For anyone reading along, see https://www.marvell.com/company/newsroom/marvell-enabling-the-next-generation-of-data-center-and-automotive-ai-accelerator-asics.html. I don't know enough yet to say how they will do, but will keep an eye on them and perhaps write-up something in a future issue
Excellent post
I'm very curious to hear more about XLA, why/how did NVIDIA have such a stronghold on the space for so long. Doesn't Google have a more challenging problem since they potentially need to work with all hardware or is XLA more optimized on TPUs?
Also how come optical computing is still a niche? Free inference sounds like something everyone would jump on? Any programmer friendly tutorials you'd recommend here? What does programming an optical computer look like and what are the key differences a practitioner should be aware of
I think Nvidia's lead arose from the fact that no one else really cared to try for a long time! CUDA was used for graphics programming and had a robust toolchain by 2012/2013 where a couple research groups figured out you could build really impressive deep networks on CUDA. Nvidia quickly cottoned on and started building custom CUDA kernels for deep learning rapidly. I think Google realized early on that CUDA put them in a bit of a bind, but bootstrapping a new compiler ecosystem is a formidable undertaking and I think it took them time to start. The fact they have a strong need now with TPUs also probably pushed their development. XLA on GPUs compiles to CUDA still I believe, but on TPUs/CPUs does something different. But it means that Google is now insulated from CUDA by one level additional shielding.
Optical computing is really interesting. I think silicon photonics has been around for a long time, but no one really had a killer use case that really brought it into the mainstream. Even now, I think the photonics inference startups face a steep barrier. Nvidia hasn't been standing still and has been moving quickly to make their inference chips better. I think it's at best a 50/50 bet that photonics inference catches on even now. Tbh, I don't really know of any good resources for programmers but will keep an eye out! The issue is like with quantum computers, very few people even have access to an optical chip to program! I imagine that eventually someone will come up with an API that's roughly like TensorFlow declarative style (build your computation graph and it's compiled down to the optical chip)
Hi Bharath, great articles & insights - could you do a primer on photonics and optical computing if you have the time?
Yes absolutely! Will try to address photonics/optical computing in a future post :)