A Big Tool Beyond Responsabilities: The Scheduler

The needs to create a system that attends multiples sources at same time produce the implementation of the TaskScheduler and TaskAssignator classes that provide the feature of work as a general conceptualization of a big machine with multiple tasks.

For a specific implementation, for your goal, you have to inherit the Scheduler and implement this funcions and actions.

You have to create in a memory shared space multiprocesing manager the following basic structures:

• list ipt :: list that manage process assignment
• list ico :: list that manage tasks on the active process
• dicctionary assigned_tasks :: dict that collect assigned tasks
• nproc :: int value with amount of assigned process
• dictionary sta_init :: dict with the flags to control task executions.

The following step is the assignment of your loop function in which you run your desired tasks. There are two cases:

• Unidirectional :: create a dictionary with your run_task async function. That function have to initialice the classes for every task and then do the unidirectional work.
• Bidirectional :: create a dictionary with you have to separate the different actions. The ‘run_task’ is for initialize the loop with the actions. The key net2service is the async function that come from network and send to service, the key service2net is the async function that come from service and send to network,

On the main script you have to put the Scheduler object on the Assignator object, in that way every time you active a task the assignator produce a message to activate the task on the Scheduler.

The magic on this clasess are the initial settings, in which you define the size of your machine:

• amount of processes
• amount of tasks by process

With that data the machine put in idle the main process task and by every process also in idle the amount of tasks. When you start to assign the task to do something, the TaskAssignator object start to wake up the idle tasks. So on, is not simple but works.

Explanation, please!

I can say this class is the digievolution of the taskloop functionalities. I need to use those tools to collect an amount of data in real time from a network of stations. If i used only a process the rate to produce a new data is large than the capacity to process all at the right time. Because of that is necesary the use of more simultaneous processes to work with all the assigned tasks.

In relation with the amount of stations is necesary to know

• amount of sources
• frequency of new data (f)
• capacity of number of stations or sources by processor in that frequency (cap(f))

We can define

• Amount of workers (procesors) to work with every source (n)
• Amount of tasks availables by every worker (m)

At the running time, the system is an empty machine with n workers in await status, and every worker with m tasks also in await status.

When the metadata to connect every source arrive to the machine, maybe using an extra worker to attend socket communication, the data is loaded and start to active the tasks, one source is one task.

And that is the idea. Big, happy, useful machines for everyone.