期望实现的demo效果:
多个线程/进程消费一个数据列,能够让各线程/进程去分担压力,避免出现一个线程/进程工作,多个线程/进程围观的情况。
(数据负载均衡演示)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import queue
import threading
import time
import random


def thread_entity_core(thread_q):
while True:
thread_q.get()
time.sleep(random.randint(0,12))


def view_con(t_list):
while True:
print("各线程负载状态: ",end='')
for i in t_list:
print(i.qsize(),end=' ')
print()
time.sleep(3)


thread_list = []
thread_q_list = []
for i in range(6):
thread_q = queue.Queue(7)
thread_q_list.append(thread_q)
thread_entity = threading.Thread(target=thread_entity_core, args=(thread_q, ))
thread_entity.start()
thread_list.append(thread_entity)

tm = threading.Thread(target=view_con, args=(thread_q_list, ))
tm.start()

while True:
queue_dict = {x: x.qsize() for x in thread_q_list}
queue_dict_sort = sorted(queue_dict.items(), key=lambda x: x[1], reverse=False)
queue_dict_sort[0][0].put(1)
time.sleep(1)

执行效果:

取到的数据,优先分配给负载最小的线程,来实现各线程压力均衡。