Binary neural networks (BNNs) have been proposed to reduce the heavy memory and computation burdens in deep neural networks. However, the binarized weights and activations in BNNs cause huge information loss, which leads to a severe accuracy decrease, and hinders the real-world applications of BNNs. To solve this problem, in this paper, we propose the information-enhanced network (IE-Net) to improve the performance of BNNs. Firstly, we design an information-enhanced binary convolution (IE-BC), which enriches the information of binary activations and boosts the representational power of the binary convolution. Secondly, we propose an information-enhanced estimator (IEE) to gradually approximate the sign function, which not only reduces the information loss caused by quantization error, but also retains the information of binary weights. Furthermore, by reducing the information loss in binary representations, the novel binary convolution and estimator gain large information compared with the previous work. The experimental results show that the IE-Net achieves accuracies of 88.5% (ResNet-20) and 61.4% (ResNet-18) on CIFAR-10 and ImageNet datasets respectively, which outperforms other SOTA methods. In conclusion, the performance of BNNs could be improved significantly with information enhancement on both weights and activations.