Azure Batch has come out of preview and its technical overview gives details of how it works. More interesting and detail article is provided here and also gives a brief description of Pool and VM Lifetime. The cost charged will be based on number of minutes number of VMs are running in a pool. It talks about two ways to maintain a pool. One is pre-defined and other is dynamic with dynamic aimed at cost efficiency. There are couple of examples posted also on MSDN pages that detail with Pool Creation and then tasks. Other examples that talk about Autopool creation through workitem.
A similarity in these code is using toolbox and wait for predefined period of time for tasks to complete. Below is the code snipped on it where a task monitor is created and wait for 10min.
- client.OpenToolbox().CreateTaskStateMonitor().WaitAll(job.ListTasks(), TaskState.Completed, TimeSpan.FromMinutes(10));
-
- Then stdout and stderr of the tasks are read and displayed in console.
-
- foreach (ICloudTask task1 in job.ListTasks())
-
- {
-
- Console.WriteLine("Task " + task1.Name + task1.GetTaskFile(Constants.StandardOutFileName).ReadAsString() + " :\n");
-
- }
We are going to extend this code for reading the state every 10min, take care of not reading the stdout when task is not complete, print only once per task and remove the vm of the task that is complete.
First we need to add check to above code to avoid exceptions due to our attempt to read a task output before its done. Add a condition inside the for loop to avoid reading the task output before its complete
Next we also add a while loop to handle keep reading the tasks status till all tasks are done. In the same code will remove the vm of task which is complete to save cost
Next to keep the console clean, we do not want to keep reading every task output, so we will use a dictionary to maintain list of tasks that where completed previously and not read them again.
-
-
- ICloudJob job = wm.GetJob(WorkItemName, JobName);
-
- Console.WriteLine("Wait 10 minute for all tasks to reach the completed state");
-
-
-
- string workItemPoolName = job.PoolName;
-
- IPoolManager pm = client.OpenPoolManager();
-
- ICloudPool pool = pm.GetPool(workItemPoolName);
-
- Dictionary < string, TaskState > taskState = new Dictionary < string, TaskState > ();
-
- while (taskState.Count < tasksToAdd.Count)
-
- {
-
- client.OpenToolbox().CreateTaskStateMonitor().WaitAll(job.ListTasks(), TaskState.Completed, TimeSpan.FromMinutes(10));
-
- foreach(ICloudTask task1 in wm.ListTasks(WorkItemName, JobName))
-
- {
-
-
-
- if (!taskState.ContainsKey(task1.Name))
-
- {
-
- if (task1.State == TaskState.Completed)
-
- {
-
- Console.WriteLine("StdOut for Task " + task1.Name + task1.GetTaskFile(Constants.StandardOutFileName).ReadAsString() + " :\n");
-
- Console.WriteLine("StdErr for Task " + task1.Name + task1.GetTaskFile(Constants.StandardErrorFileName).ReadAsString() + " :\n");
-
-
-
- taskState.Add(task1.Name, task1.State);
-
-
-
- VMInformation vmoftask = task1.VMInformation;
-
- Console.WriteLine("Deleting vm: " + vmoftask.VMName + "\n");
-
- IVM vmToDelete = pool.GetVM(vmoftask.VMName);
-
- vmToDelete.RemoveFromPool(TVMDeallocationOption.TaskCompletion);
-
- Console.WriteLine("Deleted vm: " + vmoftask.VMName + "\n");
-
-
- }
-
-
-
- else Console.WriteLine("Task " + task1.Name + " is at" + task1.State + " state \n");
-
- }
-
- }
-
- Console.WriteLine("Wait again 10 minute to get all tasks states. \n");
-
-
-
- job.Refresh();
-
- }