Besides the catalytic domain, some xylanases contained a non-catalytic domain which is named as carbohydrate binding module (CBM). CBM can be used to improve their binding-ability to insoluble substrates. We illustrated the importance of CBM by reviewing the source of CBMs, type of families, features of binding to insoluble substrates, specific amino acids involved in substrate-binding, linker peptides connecting the catalytic domain, and the effect of CBMs on xylanase thermostability. CBM is important for xylanase to break down complicate carbohydrates. Perspectives on engineering xylanase activity according to the characteristics of CBMs were given.