ResNet-18 comparative analysis of various activation functions for image classification

Gaurav Kumar Pandey, Sumit Srivastava

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Deep neural network and Machine learning are a latestemerging concept in the field of data science. Due to multilayer hierarchical feature extraction in conjunction with control variables like number of hidden layers, activation functions, and variable parameters like learning rates, initial weights, and decay functions, deep network models perform better than machine learning techniques. While most of these parameter control the learning dynamics or complexity of representation a neural network can deal with, it is only activation function which introduces non-linearity in a network and current state of activation function poses multiple challenges to both practitioners and researchers some of which are: • Vanishing & Exploding gradients during back-propagation • Zero-mean and range of outputs • Compute complexity of function • Predictive performance Due to this reason our objective in current work in focused to explain with reasoning and experiments the landscape of activation functions available. According to a recent study, we have enough cutting-edge activation functions to modify the architecture of the well-known deep network model. Building on top of widely adopted ResNet-18 network architecture in this study. Subsequently, we evaluate the effectiveness of ResNet-18 for image classification using various activation functions.

Original languageEnglish
Title of host publication6th International Conference on Inventive Computation Technologies, ICICT 2023 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages595-601
Number of pages7
ISBN (Electronic)9798350398496
DOIs
StatePublished - 2023
Externally publishedYes
Event6th International Conference on Inventive Computation Technologies, ICICT 2023 - Lalitpur, Nepal
Duration: 26 Apr 202328 Apr 2023

Publication series

Name6th International Conference on Inventive Computation Technologies, ICICT 2023 - Proceedings

Conference

Conference6th International Conference on Inventive Computation Technologies, ICICT 2023
Country/TerritoryNepal
CityLalitpur
Period26/04/2328/04/23

Keywords

  • Accuracy
  • Activation Function
  • CIFAR-10
  • Machine learning
  • Neural Networks
  • ReLU

Fingerprint

Dive into the research topics of 'ResNet-18 comparative analysis of various activation functions for image classification'. Together they form a unique fingerprint.

Cite this