Currently, when running 02-workload-heterogeneous.py in the /examples directory via dragon, I am unable to get the output of the bash ComputeTasks or Python ComputeTasks that include a process template.
Output:
Task task.000001: DONE output: 'nid004193' Stdout: 'Native Function' Return 'nid004193'
Task task.000002: DONE output: '' Stdout: Return 0
Task task.000003: DONE output: '' Stdout: Return [0, 0, 0, 0]
Task task.000004: DONE output: 0 Stdout: Return 0
Task task.000005: DONE output: [0, 0, 0, 0] Stdout: Return [0, 0, 0, 0]
I noticed that mp.set_start_method("dragon") was missing in the file. Adding that in fixed the output capture of all the ComputeTasks that define a process template (Task2-5).
Output after adding the mp.set_start_method('dragon"):
Task task.000001: DONE output: 'nid004193' Stdout: 'Native Function' Return 'nid004193'
Task task.000002: DONE output: 'nid004193' Stdout: 'nid004193' Return '0'
Task task.000003: DONE output: 'nid004193'
nid004193
nid004193
nid004193' Stdout: 'nid004193
nid004193
nid004193
nid004193' Return '[0, 0, 0, 0]'
Task task.000004: DONE output: '0' Stdout: 'Single Function' Return '0'
Task task.000005: DONE output: '[0, 0, 0, 0]' Stdout: 'Parallel Function
Parallel Function
Parallel Function
Parallel Function' Return '[0, 0, 0, 0]'
From my reading into Dragon and Rhapsody, this makes sense, as without the line, leftover artifacts from python's standard multiprocessing are being mixed in.
Also, when a process_template is used, although a ComputeTask may be a Python function, output is only accessible via stdout, not the return value.
For reference: I added a print statement to each of the Python task functions to test stdout:
async def single_function():
import socket
print("Single Function", flush=True)
return socket.gethostname()
async def parallel_function():
import socket
print("Parallel Function", flush=True)
return socket.gethostname()
async def native_function():
import socket
print("Native Function", flush=True)
return socket.gethostname()
And edited the final print-out in the for to be:
for t in tasks:
result = t.return_value if t.function else t.stdout # <-- Need to add a check here for process_templates??
print(f"Task {t.uid}: {t.state} output: '{result}' stdout: '{t.stdout}'")
Currently, when running
02-workload-heterogeneous.pyin the/examplesdirectory viadragon, I am unable to get the output of the bash ComputeTasks or Python ComputeTasks that include a process template.Output:
I noticed that
mp.set_start_method("dragon")was missing in the file. Adding that in fixed the output capture of all the ComputeTasks that define a process template (Task2-5).Output after adding the
mp.set_start_method('dragon"):From my reading into Dragon and Rhapsody, this makes sense, as without the line, leftover artifacts from python's standard multiprocessing are being mixed in.
Also, when a
process_templateis used, although a ComputeTask may be a Python function, output is only accessible via stdout, not the return value.For reference: I added a print statement to each of the Python task functions to test stdout:
And edited the final print-out in the for to be: