期望实现的demo效果:
多个线程/进程消费一个数据列,能够让各线程/进程去分担压力,避免出现一个线程/进程工作,多个线程/进程围观的情况。
(数据负载均衡演示)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
| import queue import threading import time import random
def thread_entity_core(thread_q): while True: thread_q.get() time.sleep(random.randint(0,12))
def view_con(t_list): while True: print("各线程负载状态: ",end='') for i in t_list: print(i.qsize(),end=' ') print() time.sleep(3)
thread_list = [] thread_q_list = [] for i in range(6): thread_q = queue.Queue(7) thread_q_list.append(thread_q) thread_entity = threading.Thread(target=thread_entity_core, args=(thread_q, )) thread_entity.start() thread_list.append(thread_entity)
tm = threading.Thread(target=view_con, args=(thread_q_list, )) tm.start()
while True: queue_dict = {x: x.qsize() for x in thread_q_list} queue_dict_sort = sorted(queue_dict.items(), key=lambda x: x[1], reverse=False) queue_dict_sort[0][0].put(1) time.sleep(1)
|
执行效果:
取到的数据,优先分配给负载最小的线程,来实现各线程压力均衡。