Attention has become an important component of many recent machine learning and Deep Neural Network (DNN) models. Even though the conceptualization of attention in these models is an approximation of attention mechanism in the brain, it improves the models in both accuracy and interpretability. This points to the potential benefits that building more biologically plausible attention and information routing mechanisms could have for future models In this workshop, I will introduce some of the relevant literature in Cognitive and Neural models of visual attention. I will also review recent DNN models using attention mechanism for different tasks, ranging from object recognition to caption generation. I will end by making connections between the different perspectives and provide examples of how they can be mutually beneficial.