热点文献带您关注AI深度神经网络的最新进展——图书馆前沿文献专题推荐服务（55）

2021-12-16

 

        在上一期AI文献推荐中，我们为您推荐了人工智能在光神经网络领域的热点文献，包括用于光学神经网络的11-TOPS光子卷积加速器，用于光纤非线性补偿的硅光子神经网络，用于人工智能和神经形态计算的光子学，一种实现复值神经网络的光学神经芯片等方面的文献。
本期我们为您选取了4篇文献，介绍深度神经网络的热点文献。我校人工智能学院郭军教授团队与天津大学、悉尼科技大学、伦敦大学学院、丹麦奥尔堡大学等多个单位合作在人工智能顶级期刊《IEEE Transactions on Pattern Analysis and Machine Intelligence》发表的“Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization”提出的贝叶斯Dropout优化的自适应方法可以进一步提高DNN防止过度拟合的能力。“Biologically informed deep neural network for prostate cancer discovery”提出的生物信息深度学习预测模型“P-NET”可以使用分子数据预测癌症状态。“Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops”提出了一种将任意大小的深度神经网络折叠成具有多个时间延迟反馈回路的单个神经元的方法，称为Folded-in-time DNN。“DEEP picker is a deep neural network for accurate deconvolution of complex two-dimensional NMR spectra” 介绍了一种基于深度神经网络的光谱反卷积方法，可实现二维核磁共振光谱分析的半自动化。以上深度神经网络的热点文献推送给相关领域的科研人员。
文献一 贝叶斯Dropout优化的自适应方法
Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization
Xie, Jiyang, etc.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
Due to lack of data, overfitting ubiquitously exists in real-world applications of deep neural networks (DNNs). We propose advanced dropout, a model-free methodology, to mitigate overfitting and improve the performance of DNNs. The advanced dropout technique applies a model-free and easily implemented distribution with parametric prior, and adaptively adjusts dropout rate. Specifically, the distribution parameters are optimized by stochastic gradient variational Bayes in order to carry out an end-to-end training. We evaluate the effectiveness of the advanced dropout against nine dropout techniques on seven computer vision datasets (five small-scale datasets and two large-scale datasets) with various base models. The advanced dropout outperforms all the referred techniques on all the datasets. We further compare the effectiveness ratios and find that advanced dropout achieves the highest one on most cases. Next, we conduct a set of analysis of dropout rate characteristics, including convergence of the adaptive dropout rate, the learned distributions of dropout masks, and a comparison with dropout rate generation without an explicit distribution. In addition, the ability of overfitting prevention is evaluated and confirmed. Finally, we extend the application of the advanced dropout to uncertainty inference, network pruning, text classification, and regression. The proposed advanced dropout is also superior to the corresponding referred methods.
阅读原文  https://ieeexplore.ieee.org/document/9439951

                Probabilistic graphical model of the advanced dropout technique
文献二 生物信息深度神经网络在前列腺癌发现中的应用
Biologically informed deep neural network for prostate cancer discovery
Elmarakeby, Haitham A., etc.
NATURE, 2021, 598(7880): 348–352
The determination of molecular features that mediate clinically aggressive phenotypes in prostate cancer remains a major biological and clinical challenge1,2. Recent advances in interpretability of machine learning models as applied to biomedical problems may enable discovery and prediction in clinical cancer genomics3,4,5. Here we developed P-NET—a biologically informed deep learning model—to stratify patients with prostate cancer by treatment-resistance state and evaluate molecular drivers of treatment resistance for therapeutic targeting through complete model interpretability. We demonstrate that P-NET can predict cancer state using molecular data with a performance that is superior to other modelling approaches. Moreover, the biological interpretability within P-NET revealed established and novel molecularly altered candidates, such as MDM4 and FGFR1, which were implicated in predicting advanced disease and validated in vitro. Broadly, biologically informed fully interpretable neural networks enable preclinical discovery and clinical prediction in prostate cancer and may have general applicability across cancer types.
阅读原文 https://www.nature.com/articles/s41586-021-03922-4

                                Interpretable biologically informed deep learning
文献三 Folded-in-time DNN：仅使用单个神经元的深度神经网络
Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops
Stelzer, Florian, etc.
NATURE COMMUNICATIONS, 2021, 12(1): 5164
Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.
Development of deep neural networks benefits from new approaches and perspectives. Stelzer et al. propose to fold a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops which is also of relevance for new hardware implementations and applications.
阅读原文 https://www.nature.com/articles/s41467-021-25427-4

                                                           Scheme of the Fit-DNN setup
文献四 一种用于复杂二维核磁共振光谱反卷积的深度神经网络
DEEP picker is a deep neural network for accurate deconvolution of complex two-dimensional NMR spectra
Li, Da-Wei, etc.
NATURE COMMUNICATIONS, 2021, 12(1): 5229
The analysis of nuclear magnetic resonance (NMR) spectra for the comprehensive and unambiguous identification and characterization of peaks is a difficult, but critically important step in all NMR analyses of complex biological molecular systems. Here, we introduce DEEP Picker, a deep neural network (DNN)-based approach for peak picking and spectral deconvolution which semi-automates the analysis of two-dimensional NMR spectra. DEEP Picker includes 8 hidden convolutional layers and was trained on a large number of synthetic spectra of known composition with variable degrees of crowdedness. We show that our method is able to correctly identify overlapping peaks, including ones that are challenging for expert spectroscopists and existing computational methods alike. We demonstrate the utility of DEEP Picker on NMR spectra of folded and intrinsically disordered proteins as well as a complex metabolomics mixture, and show how it provides access to valuable NMR information. DEEP Picker should facilitate the semi-automation and standardization of protocols for better consistency and sharing of results within the scientific community.
阅读原文 https://www.nature.com/articles/s41467-021-25496-5


                     Architecture of the deep neural network peak picker (DEEP Picker)
往期精彩推荐


热点文献带您关注AI深度学习与计算机视觉——图书馆前沿文献专题推荐服务（33）

热点论文与带您领略未来通信的热点技术及最新进展——图书馆前沿文献专题推荐服务（34）

热点文献带您关注AI强化学习——图书馆前沿文献专题推荐服务(35)

热点论文与带您领略5G/6G基础研究的最新进展——图书馆前沿文献专题推荐服务（36）

热点文献带您关注AI与边缘计算——图书馆前沿文献专题推荐服务（37）

热点论文与带您领略5G/6G领域热点研究的最新进展——图书馆前沿文献专题推荐服务（38）
 
热点文献带您关注AI技术的最新进展——图书馆前沿文献专题推荐服务（39）

热点论文与带您领略5G相关领域研究的最新进展——图书馆前沿文献专题推荐服务（40）

热点文献带您关注AI视觉跟踪——图书馆前沿文献专题推荐服务（41）

热点论文带您领略未来通信在海空领域研究的最新进展——图书馆前沿文献专题推荐服务（42）

热点文献带您关注AI与医学研究——图书馆前沿文献专题推荐服务（43）

热点论文带您领略未来通信在材料领域研究的最新进展——图书馆前沿文献专题推荐服务（44）
 
热点文献带您关注AI与医学图像——图书馆前沿文献专题推荐服务（45）

热点论文带您领略未来通信在光电材料及信息编码领域的最新进展——图书馆前沿文献专题推荐服务（46）

热点文献带您关注AI与生物学——图书馆前沿文献专题推荐服务（47）

热点论文带您领略未来通信在新材料技术领域的最新进展——图书馆前沿文献专题推荐服务（48）

热点文献带您关注AI与人脸识别——图书馆前沿文献专题推荐服务（49）

热点论文带您领略光电半导体领域的最新进展——图书馆前沿文献专题推荐服务（50）

热点文献带您关注AI在集成电路领域的最新进展——图书馆前沿文献专题推荐服务（51）

热点论文带您领略半导体领域的最新进展——图书馆前沿文献专题推荐服务（52）

热点文献带您关注AI在光神经网络领域的最新进展——图书馆前沿文献专题推荐服务（53）

热点论文带您领略未来通信在新材料技术领域的最新进展——图书馆前沿文献专题推荐服务（54）