Vol. 1 No. 1 (2021): Advances in Deep Learning Techniques
Articles

Residual Networks - Architectural Innovations and Beyond: Studying Architectural Innovations and Applications of Residual Networks (ResNets) for Improving Training and Performance in Deep Learning Tasks

PDF
  • Dr. Alexander Lee
Dr. Alexander Lee
Assistant Professor of Machine Learning, University of California, Berkeley, USA
Cover

Published 27-02-2024

Keywords

  • Residual Networks,
  • ResNets,
  • Deep Learning,
  • Architectural Innovations,
  • Skip Connections,
  • Training,
  • Performance,
  • Computer Vision,
  • Natural Language Processing,
  • Speech Recognition
    • ...More
    Less

How to Cite

[1]
D. A. Lee, “Residual Networks - Architectural Innovations and Beyond: Studying Architectural Innovations and Applications of Residual Networks (ResNets) for Improving Training and Performance in Deep Learning Tasks”, Adv. in Deep Learning Techniques, vol. 1, no. 1, pp. 1–10, Feb. 2024.
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Abstract

Residual Networks (ResNets) have revolutionized deep learning by addressing the vanishing gradient problem, enabling the training of very deep neural networks. This paper provides a comprehensive overview of architectural innovations in ResNets and their applications across various domains. We explore the original ResNet architecture, highlighting its key components such as skip connections and residual blocks. Additionally, we discuss advancements such as pre-activation, wide ResNets, and densely connected networks (DenseNets), which further improve the training and performance of ResNets. Furthermore, we examine the applications of ResNets in computer vision, natural language processing, and speech recognition, showcasing their effectiveness in various tasks. Finally, we discuss future research directions and challenges in the field of residual networks.

PDF

References

  1. Pargaonkar, Shravan. "A Review of Software Quality Models: A Comprehensive Analysis." Journal of Science & Technology 1.1 (2020): 40-53.
  2. Raparthi, Mohan, Sarath Babu Dodda, and SriHari Maruthi. "Examining the use of Artificial Intelligence to Enhance Security Measures in Computer Hardware, including the Detection of Hardware-based Vulnerabilities and Attacks." European Economic Letters (EEL) 10.1 (2020).
  3. Pargaonkar, Shravan. "Bridging the Gap: Methodological Insights from Cognitive Science for Enhanced Requirement Gathering." Journal of Science & Technology 1.1 (2020): 61-66.
  4. Vyas, Bhuman. "Ensuring Data Quality and Consistency in AI Systems through Kafka-Based Data Governance." Eduzone: International Peer Reviewed/Refereed Multidisciplinary Journal 10.1 (2021): 59-62.
  5. Rajendran, Rajashree Manjulalayam. "Scalability and Distributed Computing in NET for Large-Scale AI Workloads." Eduzone: International Peer Reviewed/Refereed Multidisciplinary Journal 10.2 (2021): 136-141.
  6. Pargaonkar, Shravan. "Future Directions and Concluding Remarks Navigating the Horizon of Software Quality Engineering." Journal of Science & Technology 1.1 (2020): 67-81.
  7. Raparthi, M., Dodda, S. B., & Maruthi, S. (2020). Examining the use of Artificial Intelligence to Enhance Security Measures in Computer Hardware, including the Detection of Hardware-based Vulnerabilities and Attacks. European Economic Letters (EEL), 10(1).
  8. Pargaonkar, S. (2020). A Review of Software Quality Models: A Comprehensive Analysis. Journal of Science & Technology, 1(1), 40-53.
  9. Vyas, B. (2021). Ensuring Data Quality and Consistency in AI Systems through Kafka-Based Data Governance. Eduzone: International Peer Reviewed/Refereed Multidisciplinary Journal, 10(1), 59-62.
  10. Pargaonkar, S. (2020). Bridging the Gap: Methodological Insights from Cognitive Science for Enhanced Requirement Gathering. Journal of Science & Technology, 1(1), 61-66.
  11. Rajendran, R. M. (2021). Scalability and Distributed Computing in NET for Large-Scale AI Workloads. Eduzone: International Peer Reviewed/Refereed Multidisciplinary Journal, 10(2), 136-141.
  12. Pargaonkar, S. (2020). Future Directions and Concluding Remarks Navigating the Horizon of Software Quality Engineering. Journal of Science & Technology, 1(1), 67-81.