Nonlocal quantum entanglement is a fundamental resource for future quantum networks. However, the efficiency of generating nonlocal entanglement between distant nodes is severely limited by the exponential loss incurred when locally generated entangled states are distributed through lossy quantum channels. This limitation becomes more pronounced in practical scenarios requiring the simultaneous distribution of multiple entangled pairs. Although classical multiplexing approaches, such as spatial, temporal, and frequency multiplexing, can increase the nonlocal entanglement generation rate, they do not improve the single-shot transmission efficiency. In contrast, quantum multiplexing, which can be generated by high-dimensional encoding of single photons, allows for the parallel generation of multiple nonlocal entangled pairs in a single transmission round, thereby enhancing the overall efficiency of nonlocal entanglement generation. Quantum multiplexing thus presents a promising route toward scalable quantum networks. This review introduces the mechanisms of generating nonlocal entanglement through quantum multiplexing, and focuses on two main methods: using high-dimensional single-photon encoding and high-dimensional biphoton entanglement distribution. Then it examines how quantum multiplexing can accelerate the generation of nonlocal quantum logical entanglement. Finally, it briefly explores the potential of quantum multiplexing for building large-scale quantum networks.