python 多核并行计算-并行计算机系统结构与可扩展计算

一、使用理由：

通常，现有的计算机包括多个CPU核心。 然而python 多核并行计算，在现实中运行程序时，通常只使用单核CPU，导致CPU资源利用不足。 因此python 多核并行计算，我们可以通过多核CPU并行计算来加快程序的运行速度。

2. 如何使用 2.1。 需要使用的函数

cpu_num = multiprocessing.cpu_count()

proc = multiprocessing.Process(target=single_run, args=(digits, "parallel"))
proc.start()
proc.join()

2.2 示例程序

import numpy as np
import multiprocessing
from sklearn.manifold import TSNE
import time
path = "E:\\blog\\data\\MNIST50m\\"
def single_run(digits, fold="1by1"):
    sum = 0
    for i in range(0,500000000):
        sum = sum+i
    print("sum:",sum)
def one_by_one():
    start_time = time.time()
    for i in range(0,12):
        single_run(digits=[], fold="1by1")
    end_time = time.time()
    print("one by one time:",end_time-start_time)
def parallel():
    begin_time = time.time()
    n = 10  # 10
    procs = []
    n_cpu = multiprocessing.cpu_count()
    chunk_size = int(n / n_cpu)
    for i in range(0, n_cpu):
        min_i = chunk_size * i
        if i < n_cpu - 1:
            max_i = chunk_size * (i + 1)
        else:
            max_i = n
        digits = []
        for digit in range(min_i, max_i):
            digits.append(digit)
        print("digits:",digits)
        print("CPU:",i)
        procs.append(multiprocessing.Process(target=single_run, args=(digits, "parallel")))
    for proc in procs:
        proc.start()
    for proc in procs:
        proc.join()
    end_time = time.time()
    print("parallel time: ", end_time - begin_time)
if __name__ == '__main__':
    parallel()
    one_by_one()