Mining entity and relation from unstructured text is important for knowledge graph construction and expansion. Recent approaches have achieved promising performance while still suffering from inherent limitations, such as the computation efficiency and redundancy of relation prediction. In this paper, we propose a novel hybrid attention and dilated convolution network (HADNet), an end-to-end solution for entity and relation extraction and mining. HADNet designs a novel encoder architecture integrated with an attention mechanism, dilated convolutions, and gated unit to further improve computation efficiency, which achieves an effective global receptive field while considering local context. For the decoder, we decompose the task into three phases, relation prediction, entity recognition and relation determination. We evaluate our proposed model using two public real-world datasets that the experimental results demonstrate the effectiveness of the proposed model.