TL;DR
We announce our recent partnership with Third Rock Ventures and discuss an intriguing recent paper about applying transformers to model DNA sequences
Deep Forest Sciences Enters into Partnership with Third Rock Ventures
We are excited to share that Deep Forest Sciences has entered into a partnership with Third Rock Ventures for drug discovery machine learning. The partnership will provide Third Rock Ventures' portfolio companies with unique access to Deep Forest's scientific expertise and Chiron™ platform, which is our new machine learning toolkit that accelerates drug discovery by building upon the widely used DeepChem open-source project.
Transformers for DNA?
Transformers have had a profound impact on machine learning in the last few years, with dramatic applications in NLP. Transformers have also had a marked impact on modeling protein sequences and small molecule structures. There has been some excitement about applying transformers to modeling DNA sequences, but the problem is extremely challenging because of the very long-range interactions found in DNA. A recent preprint claims to make advances on solving the problem, and presents results that pretrained transformers match or exceed baseline machine learning methods on 15 of 18 genomic tasks after fine-tuning. It is important to note though that models in this work only have an attention span of 6kbp. The discussion suggests that it may be necessary to build new architectures (or hardware) capable of processing a 200kbp attention span to solve hard long-range problems in gene expression prediction.
Interesting Links from Around the Web
https://semiengineering.com/what-does-2023-have-in-store-for-chip-design/: The chip industry has a wide range of predictions for what 2023 holds, from open source to AI/ML to quantum computing. No one appears quite sure which direction, if any, holds the strongest potential for future chip improvements.
https://www.sjsu.edu/diversity/land-acknowledgement/index.php: Silicon Valley has a long way to go in acknowledging the colonial history of the land it resides on. This land acknowledgement by San Jose State University takes an important step. The California Genocide should be emphasized more prominently in our schools.
Feedback and Comments
Please feel free to email me directly (bharath@deepforestsci.com) with your feedback and comments!
About
Deep Into the Forest is a newsletter by Deep Forest Sciences, Inc. We’re a deep tech R&D company building Chiron, an AI-powered scientific discovery engine. Deep Forest Sciences leads the development of the open source DeepChem ecosystem. Partner with us to apply our foundational AI technologies to hard real-world problems in drug discovery. Get in touch with us at partnerships@deepforestsci.com!
Credits
Author: Bharath Ramsundar, Ph.D.
Editor: Sandya Subramanian, Ph.D.