Cognitive impairment is a major concern in elderly people, and a variety of nonpharmacological therapies (NPTs) have been developed to help with cognitive decline. One of the most popular therapies is cognitive training, which includes pencil-and-paper puzzles, computerized games, or the combination of the two. Training is designed to have participants perform diverse exercises in one or more cognitive domains. Most clinical training trials indicate that well-organized tasks are clinically effective for cognitive improvement. Neural plasticity is a probable explanation for positive training effects. EEG and fMRI research show that the electrical activity and metabolism of specific brain areas are changed, and these changes are retained for a long period after training. Studies on mice to uncover the cellular and molecular changes underlying neural connectivity have found effective changes in brain networks after learning or training. Rac1 and NMDA receptors are thought to be involved in hippocampal neurogenesis, which is induced by learning. Here we review clinical trials of cognitive training, published during the last five years, and summarized some important characteristics of training tasks design. The probable role of neuronal plasticity and molecular mechanisms in training effects also are discussed. Most importantly, we discuss key ways to modifying the design of tasks based on studies we review. This review mainly identifies and discusses the reasons for positive training effects on cognition from clinical and neurophysiological perspectives. Based on the findings and their related mechanisms, further studies should design more effective and specific training tasks.